Internet piracy and cybersecurity concept. Integrated circuit and virtual digital padlocks.

Keep Calm and Carry On: 5 Data Security Do’s and Don’ts for Everyone

Cyber criminals see opportunity in the pandemic. Some exploit security vulnerabilities in remote working. Others prey on people’s fears by crafting phishing emails and malware-infected websites purportedly about the coronavirus.

You don’t have to be a security expert to fight back. We can all make a difference by following good security practices in our day to day work. These are five data security do’s and don’ts for everyone.

DO be patient – with technology, others and yourself.

Millions of employees suddenly have to learn new tools and work habits. Employers are struggling to support a newly remote workforce. Technology providers are straining to meet demand. It’s a recipe for frustration.

When we get frustrated or upset we’re more likely to take make mistakes or take security shortcuts. Instead, cultivate patience. We need to be patient when tools don’t work, organizations are over-burdened and people – ourselves included – are slow to master new technology.

DON’T forget to update privacy and security settings.

Technology helps us be productive and stay connected out of the office, but we have to use it responsibly. Cyber criminals will exploit any security vulnerability. Public settings on social media and collaboration tools leave the door open to trolls.

First, make a rough inventory of devices and programs you use for work: computers; smartphones and tablets; mobile apps; wireless router; smart (connected) devices like Wi-Fi enabled printers; VPN; cloud storage accounts; webmail; collaborative tools like Slack and Zoom; social media accounts. You may be surprised how long the list is.

Next, review and update privacy and security settings. Pay special attention to social media and collaborative platforms; they often default to less secure settings. Unlink accounts while you’re at it (Facebook doesn’t actually need to be linked to everything in your life).

Finally, revisit your passwords. Use strong, unique passwords for all your devices and accounts.

DO thank your information security team.

The security outlook was already grim before Covid-19. Companies and law firms of all sizes are under constant cyberattack. Ransomware is an ever-present threat. IT departments are short on staff and resources.

Information security is a difficult and stressful job, but it doesn’t have to be a thankless one. Take a moment to tell your data security team how much you appreciate their hard work.

DON’T fall for phishing, smishing or fake news.

A majority of data breaches originate in phishing emails. Other methods popular with cyber criminals are smishing (the text message version of email phishing), fake news links in social media and malicious websites. We are the weak link and there are plenty of depressing statistics to prove it.

By the same token, we’re the front-line defense. We thwart a cyberattack every time we recognize a phishing attack or social media scam for what it is.

Be on the lookout for warning signs:

  • You don’t know the sender, or it’s someone you haven’t heard from in a long time and who has no reason to contact you.
  • The message is out of character or the sender doesn’t usually send files or links by email/text.
  • Content clues like missing or different signature block, spelling errors, grammatical mistakes, not the sender’s usual “voice.”
  • The message violates the organization’s security procedures, chain of command, code of conduct, etc.
  • It involves money in any way.

If an email or text might be legitimate but you have even the smallest doubt, verify first. For news, stick to trusted sites. Constant vigilance is essential. Cyberattacks directed at individual technology users are increasingly sophisticated and topical.

DO what you can where you are.

At the end of the day, it all comes down to doing what we can where we are. Following good data security practices for individuals is the baseline.

From leadership to participation, there are many ways to actively support your organization’s security initiatives. Approving a software purchase, raising the alarm about a possible security problem (even if it turns out to be nothing) and attending technology training webinars are just a few examples.

Are you tech-savvy or a remote working pro? Your colleagues need your help.

Cybersecurity is essential business for lawyers and legal professionals. We have an ethical obligation to safeguard clients’ confidential and privileged information. Moreover, clients – and potential clients – need legal counsel and practical assistance in cybersecurity issues.

Data security in the legal sector is both a duty and an opportunity to provide superior service. By educating ourselves about cybersecurity and following best practices, we can all help safeguard our own and our clients’ confidential information.

ACEDS Welcomes the Orange County Chapter

ACEDS Launches Orange County Chapter

March 27, 2020EAGAN, Minn. – The Association of Certified E-Discovery Specialists (ACEDS), the world’s leading e-discovery training and certification professional association and part of The BARBRI Group, continues to expand its presence with the launch of a new chapter in Orange County, CA

The ACEDS Orange County Chapter, which will serve legal professionals in Orange County and surrounding areas, will create an environment where members can collaborate on and discuss professional development, training and certification as well as issues in the legal industry impacting e-discovery professionals in the global legal community. Many Orange County chapter members are Certified E-Discovery Specialists (CEDS) or are aspiring to earn their CEDS or the eDiscovery Executive (eDEx) certificate.

Among its activities, the chapter will organize networking opportunities, professional development events and training sessions, while also participating in local and global conferences and events. The chapter welcomes e-discovery, information governance, data privacy and other data management professionals in the Orange County area, including law students, new graduates, young legal professionals, and seasoned legal professionals, and will assist aspiring CEDS in earning their certifications.

“We are pleased to welcome the Orange County Chapter to our association. This chapter will serve Orange County, San Diego and the Inland Empire e-discovery community providing education, training, and networking support, and we congratulate the chapter board on its creation,” said Maribel Rivera, Senior Director of Community Relations, ACEDS.

The Orange County Chapter will be led by:

  • PRESIDENT: Marla Mohr
  • VICE PRESIDENT: Grady Glover
  • SECRETARY: Sherri Watts, CEDS
  • TREASURER: Michael Holden,
  • DIRECTOR AT LARGE: Christine Stansall, CEDS, Tona R. Cornelius, Annie Loo, CEDS

“It has taken many years, but we are excited to be launching an Orange County Chapter of ACEDS. I’m thrilled to be working with such a diverse group of people on our board to bring eDiscovery and legal technology education and networking to the Orange County and Southern California area,” said Marla Mohr, Chapter President.

To learn more about the Orange County chapter, subscribe to receive news and updates from ACEDS, or contact the association at You can also join the chapter at and interact with members on the Orange County LinkedIn Group.

The Association of Certified E-Discovery Specialists (ACEDS), part of leading legal education provider The BARBRI Group, is a global member-based association for professionals who work in e-discovery, information governance, compliance and the broader legal community. ACEDS provides training and certification in e-discovery and related disciplines to corporate legal departments, law firms, the government, service providers and institutions of higher learning. Our CEDS certification is recognized around the world and used to verify skills and competence in electronic discovery for organizations and individuals through training, certification and ongoing education. The CEDS credential is held by practitioners at the largest Fortune 500 companies, Am Law 200 firms and government agencies. ACEDS has 23 chapters, with locations in most major U.S. cities, the UK, Ireland, Canada, the Netherlands and South Africa (with Australia and South America chapters coming soon). Our goal is to help professionals and organizations reduce the costs and risks associated with e-discovery while helping to improve and verify their skills and advance their careers and overall technology competence in e-discovery and related fields.

Contact: Cindy Parks

Credit card security. Online Shopping security

Coronavirus Home Office Security – Practical Tips for Securing Your “New” Home Office

I’ve been working as an independent consultant for quite some time. Along the journey, I’ve picked up many tips and tricks to maximize productivity while working from home with great results.  There have been many articles written about this issue and I hope to add some serious security ideas to the discussion that you may not have considered.

Many of you are being told to work from home with no idea on where to start and what matters the most. This article is going to focus on practical information to help you secure your home office. Check with your company to see if there are any protocols in place. If not, start with these basics.

Strong and weak easy Password. Note pad and laptop.

Use strong passwords and a password manager

Passwords should be unique for every account and should comprise a long string of upper- and lower-case letters, numbers, and special characters. Clearly, it’s difficult to remember all these passwords, which is why password managers are such popular tools these days. I use LastPass for saving and accessing passwords but of course there are other products available I particularly like LastPass because anything you save to LastPass on one device is instantly available to you on any other device you use. if you don’t already have a LastPass account, you can get started by signing up for a free trial at I am not affiliated with LastPass in any way.

Set up two-factor authentication

Multifactor Authentication is an added layer of security that you can enable within LastPass and requires a second step before you can gain access to your account. Enabling this security feature helps protect your account from keyloggers and other threats – even if your Master Password was compromised, your account could not be accessed without this second form of authentication.

Virtual private network, VPN, Data encryption, IP substitute.

Use a VPN

A Virtual Private Network, or VPN, is a piece of software that changes your IP address and encrypts all of your internet traffic. This improves online privacy, security, and helps users to bypass online censorship imposed by the government, ISPs or any other organization or person blocking websites. A popular main reason to use a VPN is to protect your online information and to visit websites that can be hard to enjoy locally. When left unprotected, your private data, such as bank account information and credit card numbers, can fall into the wrong hands. A good VPN encrypts your data, so even if you connect to a public wi-fi network, your private data is guaranteed to be protected.

Set up firewalls

A firewall is a network security device that monitors incoming and outgoing network traffic and permits or blocks data packets based on a set of security rules. Its purpose is to establish a barrier between your internal network and incoming traffic from external sources (such as the internet) in order to block malicious traffic like viruses and hackers.

Your home internet provider may already have a firewall in place so check before your bother to set one up

Use an antivirus software

Antivirus software is nearly as crucial as a PC’s operating system. Even if you’re aware of potential threats and practice extreme caution, some threats just can’t be prevented without the extra help of an AV program—or a full antivirus suite.

Antivirus software is critical for EVERY PC or device you use at home. Without it, you risk losing your personal information, your files, and even the cash from your bank account.

AV software can keep your Windows PC safe from spyware, Trojans, malware, and more.

There are quite a few great choices out there for you can easily find with a Google search, or better yet, ask your IT person what he/she recommends.

Man touching a wifi security concept

Secure your home router

Wireless internet or Wi-Fi access has become a necessity in the home and workplace, but it can also open a door to risks from hackers, scammers, and identity thieves. Whether in your home or office, an unsecured Wi-Fi router running on the default manufacturer settings could be a liability when it comes to hackers and Wi-Fi squatters accessing your private information and burdening your broadband.

If your Wi-Fi network isn’t secured properly — a public IP address, no unique Wi-Fi password — you could be letting anyone with a wireless-enabled device to gain access. You might not be worried about someone using your wireless connection, but the real risk is exposing sensitive information you send and receive — your emails, banking information, and maybe even your smart home’s daily schedule — to cybercriminals.

Install updates regularly

Microsoft Update is the online extension of Windows that helps you keep your computer current. Microsoft Update includes updates from Windows Update and from Office Update, in addition to updates for other Microsoft products and for third-party device drivers. Use Microsoft Update to install updates for your computer’s operating system, software, and hardware.

New content is added to the site regularly so that you can obtain recent updates and fixes to help protect your computer and to keep it running smoothly. To use the Microsoft Update site to install all critical updates for your computer, follow these steps.

I choose the other route: automatic updates. By using Automatic Updates, I don’t have to visit the Microsoft Update Web site to scan for updates. Instead, Windows automatically delivers them to my computer and installs them automatically.

Red key with text BackUp and touch finger icon on blue digital laptop keyboard

Back up your data

The main reason for data backup is to save important files if a system crash or hard drive failure occurs. There should be additional data backups if the original backups result in data corruption or hard drive failure.

Additional backups are necessary if natural or man-made disasters occur.

Encrypt your hard drives in Windows 10

Simply locking your PC with a password isn’t enough, as hackers can still find ways to bypass the lock screen. Windows Hello makes the processes a lot harder considering it relies on biometrics, but in cases where your information is stored on a secondary hard drive that can be pulled out, biometrics become largely irrelevant.

The good news is that you can still protect your information on Windows 8 by using BitLocker drive encryption.

In both cases you need the Pro version of Windows, not Home.

BitLocker can be used to secure both internal and external hard drives. It doesn’t only function after signing into Windows, it can also determine if a security threat is present during the boot up process, so you’re fully covered.

MacOS has encryption built in regardless of the version.

Beware remote desktop tools

Remote desktop tools aren’t new but with organizations becoming increasingly international and teams becoming more mobile they’re fast becoming essential.

A search for remote desktop software reveals a myriad of options. It can be overwhelming navigating past rogue tools, confusing interfaces, and buggy services.

For a great overview, take a look at the 10 Best Free Remote Desktop Tools.

The abstract image of the hacker's hand reach through a laptop s

Look out for phishing emails and sites

What is phishing? Phishing is a cybercrime in which a target is contacted by email, telephone or text message by someone posing as a legitimate institution to lure individuals into providing sensitive data such as personally identifiable information, banking and credit card details, and passwords. Don’t buy into these schemes.

The FTC has a great article surrounding phishing. Take the time to educate yourself on this very real threat.

The FBI’s IC3 (Internet Crime Complaint Center) has a chart showing the huge amount of loss victims have been taken for. What’s extremely sad is how the scams focus so much on our elderly.

chart showing the huge amount of loss victims have been taken for.

Use encrypted communications

In the legal profession, encrypted communications could be critical since we’re dealing with attorney client privilege so often. I used the website and checked my email to see if I had been a victim of a breach and sure enough, I have two out of three emails that have been breached. There are several tools to stop this from happening to you (I installed one myself) and you can find a great list at including free, paid and business tools and services.

Check your bar association to see if encryption is required in your state to protect confidentiality.

Locked chain on laptop as computer protection and cyber safety concept. Private data protection from hacker malware

Lock your device

To lock your computer:

Press the Win+L key combination on the computer keyboard (Win is the Windows key). Windows key features the Windows logo.

Click the padlock button in the lower-right corner of the Start button menu. Clicking the padlock icon locks your PC. Why lock your computer? I know you didn’t ask that question, right? For step by step directions, go to

Hopefully, I set out some helpful information and tips for you to assist the transition to your new unfamiliar environment. Working from home can certainly have its benefits, but it also comes with major responsibilities. Take the time to implement those responsibilities.

digital fingerprint login for cybersecurity online on internet

Popular Jewel-Osco Grocery Chain Claims BIPA is Unlawful

BIPA (Biometric Information Privacy Act) was first introduced by Illinois in 2008 and requires informed consent of the collection of biometric data prior to collection, prohibits companies from profiting from biometric data, permits only a limited right to disclose data, mandates protection obligations and retention guidelines and creates a private right of action for any individuals harmed by violators of BIPA.

Albertson’s, which owns the popular Jewel-Osco chain of grocery stores in Chicago IL, disagrees with this law — so they’re taking the state to court. Albertson’s argues that the Illinois Biometric Information Privacy Act should be considered special legislation prohibited by the state’s constitution, because it applies to some companies while improperly leaving others out.

The grocery chain, who is fighting a lawsuit, claims that it shouldn’t have to face litigation for collecting employees’ fingerprints because there is no functional difference between it and the types of businesses excluded from liability under Illinois’ biometric privacy law. BIPA sets up many private employers for huge judgments, while exempting government and financial institutions.

One often forgotten requirement under BIPA is data retention. Like all other privacy laws, data retention is mandatory. BIPA states that PII (personal identifiable information) must be deleted within three years or organizations can face up to $1,000 in fines/violations. Robert Fowler, Director of Strategic Partnerships at Exterro, states that BIPA is just another example of why it’s critical for organizations to know the kind of personal data they are collecting and ensure document retention requirements are enforced around it. In order to meet the requirements of BIPA, conducting a data inventory isn’t a nice to have anymore—it’s a necessity.”

Personal data you don’t have cannot be breached. A clear way to mitigate a lot of organizational risk is to get rid of data that you don’t need. It’s an unnecessary added liability to over-retain personal data that serves no business purpose.

Albertson’s issues surrounding compliance to BIPA start with the organization’s data map, which, for any organization, needs to be built the right way. Keeping the data inventory up-to-date can help prevent running afoul of data privacy laws like BIPA, or for larger organizations with an international presence, the EU’s General Data Protection Regulation (GDPR). Fines for violating such privacy laws can really add up, which makes it all the more important to ensure the only data retained by organizations is accurate and necessary for business purposes.

Juan Di Luca_CEDS Spotlight

CEDS Spotlight – Juan Di Luca

Welcome to our “CEDS Spotlight” where we will feature ACEDS members who have recently become CEDS certified. Every one of our members is unique and so are their e-discovery journeys. We hope this will be a terrific way for you to get to know the ACEDS community.



[Maribel] Hi I’m Maribel Rivera, the senior director of community relations at ACEDS. Today I’m delighted for us to showcase Juan Di Luca. Juan is the founder of Data Analysis Services and cross border e-discovery project manager and he’s also an attorney. Juan is located in London and he’s here with me today just to give us a little bit of information and background about him. Hi Juan, how are you doing?

[Juan] Very good, very good. And you?

[Maribel] I’m doing great, thank you. Thank you so much for being with us. I know we’ve got a bit of a time difference, so I’m happy that we could get together and do this spotlight for you.

[Juan] Thank you so much.

[Maribel] Great. So, Juan can you share a little bit about your background with me?

[Juan] Yes, yes. I’m a lawyer from the civil law country from Venezuela. I moved to the U.S. did my LLM there, did a paralegal degree there, and then I moved to the UK for work and then life have me here for the last ten years.

[Maribel] Wonderful, and can you share just a little bit with me about your expertise and how you fell into e-discovery?

[Juan] Yes, as soon as I finished my LLM in Washington D.C. an organization was looking for lawyers with international background and languages and that was me. I had a background both in civil law and common law and speak Spanish and they were looking for lawyers with different languages and Spanish was one of those. So they asked me to help them with an investigative document reviewer. Then that investigation finished and they realized that I was also Italian citizen and I was able to work in Europe and they have a portion of the investigation in Belgium, Brussels. And I came to Europe for work and met with other organizations who were looking for people who could help them to grow the e-discovery business and to, actually knowledge, in Europe about e-discovery. Back then in 2007 not too many people in Europe knew about this area, and they were looking for people to help them to develop the e-discovery as a business, as a business model, and that was me. I started working with them, learning, and I started my career there basically, as a, in the document review area, became project manager and then I moved into business development and eventually serving a client in Switzerland opened an opportunity for me to start my own company, and that’s what I have been doing since then.

[Maribel] That’s amazing. So you have a lot of cross border experience it sounds like, just dealing with e-discovery in general.

[Juan] Yes, that’s the case. I guess my take on this is that the U.S. is always the starting point and I guess it’s where the e-discovery area is more mature and more developed. And I think having that experience to start with is, plus my experience working as a foreign legal consultant in the U.S. with law firms really helped me to have a good base, and I brought all of that experience and then in Europe the same needs that we had in the U.S. We found those in here in different locations and different countries and in differences of laws. But the basics were the same and we managed to provide services in different locations including, I have even worked in the Middle East.

[Maribel] Can you share a little bit with me about how you heard about ACEDS and why you decided to get certified?

[Juan] Yes, it’s a nice certification everywhere, especially in the U.S. I think in Europe there is a still little bit more of need for people and professionals in my area to learn about ACEDS, and also to learn about the value of the certification. They always tell, when people ask me about this certification I always explain to them that this is an agnostic approach to e-discovery and it’s not married to any specific technology. And that’s a huge value because every other certification out there in the market usually it relates to one specific technology. Which is great, but in a way it’s quite narrow and as technology is moving so fast and it’s changing faster than we can actually comprehend, it’s always difficult to keep up getting certification with different technologies after a few years are obsolete and we need to. So the good thing about ACEDS is that it’s a good base, it’s a good foundation, while it’s strong in knowledge and it doesn’t relate to any specific portion of technology or tool, so it brings a broader idea about the area instead of one specific part.

[Maribel] So it kind of gives you that foundation that you’re looking for.

[Juan] Exactly. Yes, and I would say more than a foundation because it is quite intense and quite deep in knowledge. But you’re right, I take it as a foundation in the sense that it can apply to, you can use that as a start point to work with every other technology in the market for any area within the e-discovery model.

[Maribel] Well, given that you’ve mentioned that. So, can you share a little bit about how long it took you to prepare for the exam and your thoughts after you took the exam. How you felt?

[Juan] Oh the exam is quite intense. I was surprised. As lawyers, you always think that there is nothing harder than law school and if you managed law school it should be fine to take any exam in the future. But yes, I was knowing, I guess you lose practice but, it’s something that you really need to put your head around. It’s not something that you can take lightly. The best approach is to try to study all the material in the shortest period of time possible and take the exam. But having said that, it took me three months to prepare for the exam.

[Maribel] Oh wow, okay.

[Juan] Yeah.

[Maribel] Three months.

[Juan] But obviously. Yeah, you’re sharing work-life. You know, work and personal life and all that. It took me a little longer than what I expected to do some reading again, especially of the material I covered in the beginning. So I guess

[Maribel] Okay.

[Juan] if I would have to do it again I would say no more than two months would be ideal. Eight weeks.

[Maribel] Is there anything, is there anything that you can, you gained from taking the certification that you didn’t know before?

[Juan] Oh, as far as knowledge, the material itself.

[Maribel] Yeah.

[Juan] Yes, definitely the material is structured according to the EDRM model so basically you follow, the reading goes together with the model we use as a service, so it makes perfect sense. But, it also goes deep into the tells in every single stage, and more. So, there is extra information on every single step of the EDRM where you can go deeper if you want. So, yes definitely I learned a lot about the different steps, and I see every other professionals, each one have all specialized, mostly specialized, in one or two areas of the EDRM so we knowledge of all the rest but we can work on that. So, because I am a lawyer I tend to work more on the later part of the EDRM because the earlier part of the EDRM is more technical. Where they gather the data from computers and they work with hard drives, so that specific portion is usually done by people with more technical background. But having said that, there is people there with legal background and the lawyers managing the investigation, I’m in full control of that. So don’t take my experience of that, but my experience specific experience is that I work mostly on the end with the service, once the data is already in the platform for review. So with the examination, I have the opportunity of learning about the earlier part of the model, so that was quite important.

[Maribel] I mean overall it sounds like anyone who’s doing e-discovery you would suggest that they take the training and the certification, but for attorneys do you think that that’s important for them to understand that and get that foundation as well?

[Juan] Completely, completely. And I think it makes perfect sense for attorneys coming from the common law system either in the UK or the US, attorneys with that background only, it’s more natural because the discovery part of the legal process is what they learn in law school. So this is just the application of technology to that portion of the legal process, so for them it’s more natural. It’s basically getting the certification to understand how technology applies to the exchange of evidence or the analysis of evidence, so it’s easier. I think it’s also super important for lawyers coming from civil law countries, or with civil law backgrounds for this era and this date any middle sized organization or big organizations anywhere in the world. Especially, in Latin Americas that we have a whole continent there, we have Spain and so on. All of those countries, although they don’t exchange evidence as they do in common law countries they are part of a global market. In a way, they have to learn. They think they don’t need it but they do to learn about it. So I think in a nutshell I think it’s important for everyone should at least understand the basics. And the certification I think is the best way to know the whole process and take it from there. Yeah.

[Maribel] That’s excellent, thank you so much. How does it feel to be part of the ACEDS community? Can you share? I’ve been part of the community for five or six years now.

[Juan] It’s amazing.

[Maribel] So how does it feel having that community?

[Juan] Oh I’m happy to do the ACEDS certification was already been busy. I tried one dinner, it really helps. Remember that the e-discovery area is new, it’s not that new the technology and all this I think has been in place for over 20 years, but having, even though we have been working on this for that many years there is not any specific career or school for e-discovery. So there is a huge gap and basically knowledge comes through experience, more or less, and it’s really hard for organizations and for people to really measure backgrounds. So it’s always, it’s an empirical process where you need to prove that you know. And it’s a difficult area because there is a huge need for professionals but there is not a specific structure, place where people could learn about the area. And I think it’s one of the few, I think it’s the only one, but I’m not sure if that’s the case. But as far as I know it’s the only one, it’s the only certification that has an agnostic approach to e-discovery and I think that’s the real value because if you have somebody who wants to learn about e-discovery or wants to get certified, ACEDS can provide that combination. Can give you a certification that talks about the whole process and it doesn’t relate to one specific technology or company. So it has a little more value because you guys, don’t, you are not suggesting if you get the certification you should work with this technology or the other technology. You guys don’t do that. You just teach the knowledge and the basics and the structure and then each professional takes it from there depending on who they are working for or the tools they’re working with.

[Maribel] Excellent, thank you so much. So, I’m not going to keep you too much longer but I have one last question for you. What, right now are you reading that you think our community should read?

[Juan] Well I’m not sure if the community should read it.

[Juan] I love an author called Paulo Coelho. He’s an author that basically talks about his journey in life and somehow I tend to identify with his books. I’m reading “The Alchemist” and it’s one of those books that you can read many times and always get something different each time that you have the opportunity of reading that. Yes, so that’s what I’m doing now.

[Maribel] I have to agree with you I think it’s, his books it’s where you are in your own journey depends how you read that book. So I can totally relate to that. Great.

[Juan] Great that’s it.

[Maribel] Well thank you, thank you so much for being with us Juan. I really appreciate it and I really thank you for being part of our community.

[Juan] Thank you so much for the opportunity and looking forward to keep learning from the ACEDS and being part of the community.

[Maribel] Thank you.

[Juan] Thank you so much.

Like what you’ve read? Keep up to date on the latest #CEDSCommunity updates by subscribing to our blog or by following us on LinkedIn or Twitter.

Young woman looking on the black board with mathematical formula

There Is No One Size Fits All Sample Size Appropriate for TAR Validation (Part II)

In the first part of this piece I pointed out why it can be difficult to validate TAR using control set metrics. When the overall proportion of responsive documents is very low, it becomes increasingly burdensome to randomly sample enough documents for useful estimates of recall and precision. For this reason and others, elusion testing is joining or even displacing control set metrics as the industry’s preferred approach to validation.

Elusion testing is often preferable because it is easier and more intuitive to infer meaningful estimates from the resulting proportions, allows one to include all or nearly all reviewed material as training data, and may reduce the burden of reviewing a large volume of non-responsive documents for validation purposes.

In the context of eDiscovery, an elusion test sample is a random sample of all documents not slated for production or human review. The proportion of responsive documents in the sample serves to approximate the number of responsive documents in that discard pile. This is usually undertaken at the end of the review process as a means of confirming that a party has made a reasonable and proportional effort to satisfy their document production obligations.

Because recall and precision are calculated using only a subset of documents in a random sample, it can be difficult to determine how many total documents to include in a control set sample. However, since the proportion we’re estimating with elusion testing is taken from the full sample set used for validation, it is far easier to calculate an appropriate sample size.

That said, it’s just as dangerous to fixate on a certain sample size or confidence interval for elusion testing as it is to do so for a control set. Consider the following examples.

Imagine we’re faced with a review database consisting of:

150,000 Total Documents

30,000 responsive documents prioritized by TAR software and coded with perfect accuracy by attorneys 20,000 not responsive documents prioritized by TAR software and coded with perfect accuracy by attorneys 100,000 documents categorized as unlikely to be responsive and not reviewed by attorneys

Since our software indicates that the remaining 100,000 are not responsive, we decide to validate using elusion testing. We use the sampling feature of our software to specify a 95% confidence level (CL) and 5% confidence interval (CI) or margin of error (MoE) to randomly select 383 documents for our elusion test.

The review team codes with perfect accuracy and identifies 5 documents of the 383 as responsive. A conservative interval estimate using a 95% confidence level suggests that .4% to 3% of the not reviewed documents are responsive. This means we can expect to find an additional 400 to 3000 responsive documents in the not reviewed data.

Remembering that recall is the proportion of all responsive documents correctly identified as responsive, we can now estimate a range for this proportion using simple arithmetic.

proportion using simple arithmetic

This gives us:


Those numbers look pretty good. However, consider a similar situation to see where we may run into issues using the exact same sample size for elusion testing.

This time we’re looking at:

150,000 Total Documents
4,000 responsive documents prioritized by TAR software and coded with perfect accuracy by attorneys 46,000 not responsive documents prioritized by TAR software and coded with perfect accuracy by attorneys 100,000 documents categorized as unlikely to be responsive and not reviewed by attorneys

Once again, as given by our 95% CL and 5% MoE settings, we sample 383 documents of the 100,000 not reviewed. Coincidentally, we find the exact same proportion in this sample as in the other—5 responsive documents. As in the prior scenario, with 95% confidence we can infer that there are an additional 400 to 3000 responsive documents in the documents categorized as not responsive by our software.

As in the prior scenario, with 95% confidence we can infer that there are an additional 400 to 3000 responsive documents in the documents categorized as not responsive by our software.

However, low richness foils our efforts toward efficiency once again because this time we’re looking at the following lower and upper bounds:


With these numbers we’re back to validation metrics that may be less than defensible. It’s also worth noting that we should be transparent about calculating this range for recall as a function of the elusion rate rather than as a confidence interval around a sample proportion because the method of calculation implies certain assumptions, which I’ll save for a deeper dive on probability theory.

Now that we’ve demonstrated the problems with using a fixed sample size or even a fixed confidence interval for validation, we’ll use algebra to answer the question implicit in all this.

What sample size is appropriate for my elusion test?

In order to solve for our unknown (an appropriate sample size), we need to know the values of the following variables:

Total number of responsive documents identified
Total number of documents not slated for review or production
Target minimum recall
Expected elusion rate

The formula for the elusion rate is:

Formula for the elusion rate

We’ll revisit the second scenario posed above and aim for 75% minimum recall as we determine our sample size. This sets the variables at:

4,000 = Total number of responsive documents identified
100,000 = Total number of documents not slated for review or production
75% = Target minimum recall
.65% = Expected elusion rate

The expected elusion rate can usually be estimated from other metrics, but if the elusion rate is unknown, 0 can be used for the smallest possible sample or use half of the value in step 2 below for a reasonable estimate. Also, note that of course you should continue fine-tuning your models and reviewing additional documents before sampling for elusion if your expected elusion rate is higher than the value in step 2.

Step 1) Solve for the maximum number of eluded documents permitted by your target recall:


Step 2) Calculate maximum elusion rate given by step 1:


Step 3) Solve for smallest possible sample size that allows us to estimate maximum elusion rate given by expected elusion rate and step 2 as upper bound. You can use the Wilson score interval formula below to solve for n—one day I’ll explain why that’s my preferred approach—or you can use an algorithm to solve for the sample size given by a Clopper-Pearson interval if you prefer.

Wilson-Score Equation

A sample of 1078 documents with no more than 7 responsive documents in it allows us to infer with 95% confidence that the actual elusion rate is below 1.33%.

Returning to our low richness scenario, our review team codes the elusion sample of 1078 documents with perfect accuracy and identifies 3 responsive documents in the sample. This indicates that with 95% confidence we can estimate the true elusion rate is .06% to .81%, thus it is reasonably likely there are 60 to 810 responsive documents in our discard pile.

Considering that,


Our elusion test allows us to estimate recall at 83% to 99%. While that range is quite a bit wider than one given by a 5% margin of error, it should be more than adequate for defensibility purposes.

Supposing that the true number of responsive documents in the total set for review was 4,500 out of 150,000 and our true recall was 89% (4,000 out of 4,500), we probably would have needed to sample around 5,000 documents for a control set to estimate recall with a 95% CL and a 5% MoE.

While very low richness still complicates validation, this approach to elusion testing gives us a more reasonable way to estimate recall.

Judge gavel and tablet computer on table

Basics of Presentation Technology for Legal Advocacy

Lawyers use presentation technology to advocate for their clients more effectively and efficiently. Low-cost electronics have brought technology within reach of practically any budget. Technology may now be deployed in every manner of matter and setting, from lengthy jury trials to short arbitrations.

No one should be intimidated by presentation technology. However, it isn’t simply a matter of bringing your computer to court either. Like every aspect of legal work, using presentation technology has best practices and potential pitfalls. Planning and preparation are the keys to avoiding common mistakes and presenting your best case.

Technology and Paper: Not an Either/Or

Presentation technology is equipment and software for the electronic presentation of evidence and demonstratives in a legal proceeding.

This table summarizes traditional presentation means and the corresponding technology means:

EXHIBITS Paper Screen display
DEPOSITIONS Read into record Video playback
DEMONSTRATIVES Trial boards PowerPoint

In talking to lawyers about presentation technology, I often encounter two misconceptions. First, presentation technology is not “the paperless courtroom.”

When technology is used, electronic presentation is the primary means of display – but not the sole means. A strategic approach to presentation selects the best tool available for each exhibit, deposition and demonstrative. For example, you might use a PowerPoint in conjunction with a trial board or hand the witness a paper copy of an exhibit while everyone else views it on a screen.

Presentation Technology: Not Just for Trials

The second misconception is that presentation technology is only for trials.

Electronic evidence presentation became associated early on with high-value jury trials where the amount at stake justified the expense of equipment rental. However, it’s been many years since the federal courts spurred greater technology use by equipping courtrooms with monitor display systems.

State courts on the whole trail their federal counterparts in this respect, but they’re catching up fast. The situation also varies tremendously from courthouse to courthouse. While hardly the norm, some state courts have installed state-of-the-art technology that outdoes the federal court next door.

In addition, the cost of electronic equipment has dropped dramatically. If there is no or inadequate technology on-site in a courtroom or other venue, parties may be able to procure a basic equipment set-up for a very modest cost.

These trends combine to make presentation technology a feasible option in a wide range of matters beyond trials. For example, preliminary injunction hearings, administrative proceedings and arbitrations.

Deploying Technology Requires Software, Equipment and People

Deploying presentation technology requires the right software, equipment and people:

  • Software – Software programs to display each content type.
  • Equipment – Computers to run the software and a display system.
  • People – Trained staff to use the software and manage the equipment.

Neglecting any of these will inevitably lower the quality of your presentation.

Put positively, with careful planning and preparation you can deploy the software, equipment and people you need for a high quality, efficient and cost-effective presentation.

Part 2 of this three-part series will cover presentation software, equipment and staffing in greater depth. Part 3 will review three common mistakes in using presentation technology and how to avoid them. The good news for legal professionals is presentation technology is more about good case management than advanced computer literacy.

Zapproved ACEDS Partnership

ACEDS Announces Zapproved Renews as Affiliate Partner

March 3, 2020EAGAN, Minn. – The Association of Certified E-Discovery Specialists (ACEDS), the world’s leading e-discovery training and certification professional association and part of The BARBRI Group, is pleased to announce that Zapproved, an e-discovery software provider that empowers in-house legal teams to more effectively manage litigation response, investigations and information requests, has renewed as an ACEDS Affiliate Partner.

As an ACEDS Affiliate Partner, Zapproved plays an important role in the global ACEDS community, leveraging and contributing to the community’s collective knowledge and expertise, and the association’s education, marketing, training and professional development resources. Employees and partners of Zapproved are able to access ACEDS’ breadth of job tools and networking forums, a global chapter network and events, and a best-practice-oriented worldwide community of professionals.

“The Zapproved team is known for its passion and innovation in leveraging software to transform the corporate legal process,” said Maribel Rivera, Senior Director of Community Relations at ACEDS. “We’re excited about the enhanced partnership which allows us to collaborate and share e-discovery best practices with our community.”

“We are so thrilled to strengthen and solidify our long-standing relationship with the ACEDS community. We look forward to working with the new board of directors and celebrating the organization’s continued growth,” added Melissa Tatham, Zapproved Head of Marketing.

The Association of Certified E-Discovery Specialists (ACEDS), part of leading legal education provider The BARBRI Group, is a global member-based association for professionals who work in e-discovery, information governance, compliance and the broader legal community. ACEDS provides training and certification in e-discovery and related disciplines to corporate legal departments, law firms, the government, service providers and institutions of higher learning. The CEDS certification is recognized around the world and used to verify skills and competence in electronic discovery for organizations and individuals through training, certification and ongoing education. The CEDS credential is held by practitioners at the largest Fortune 500 companies, Am Law 200 firms and government agencies. ACEDS has 23 chapters, with locations in most major U.S. cities, the UK, Ireland, Canada, the Netherlands and South Africa (with Australia and South America chapters coming soon). Our goal is to help professionals and organizations reduce the costs and risks associated with e-discovery while helping to improve and verify their skills and advance their careers and overall technology competence in e-discovery and related fields.

About Zapproved
Founded in 2008, Zapproved builds e-discovery software designed to help corporate legal and compliance teams reduce costs and mitigate risk. Our software empowers teams to manage legal holds and collect, process, and review data with ease. We have more than 350 corporate customers, a 99% retention rate, and an unwavering commitment to keeping our customers ridiculously successful.

Contact: Cindy Parks

Justice and Law concept image

Sean O’Shea: Tips for Paralegals and Litigation Support Professionals – February 2020

2/1/2020: Illinois Supreme Court: Biometric Information Privacy Act Protects Rights Even If No Injury
Illinois Supreme Court rules that Biometric Information Privacy Act does not require actual injury when fingerprint collected without notice or consent.

2/2/2020: Relativity Pricing
Since 2019, Relativity has only charged 33% as much for data stored for early case assessment as it charges for active data, and only charges users that work in RelativityOne (its SaaS software as service) 60% of regular user rates.

2/3/2020: Relativity Pay As You Go Pricing a New User Interface
Aero UI will allow for faster load times.

Relativity Pay As You Go Pricing a New User Interface

2/4/2020: Kubernetes
Kubernetes (K8s) is a system that automatically deploys applications to different web connected computers or hosts.

2/5/2020: Relativity and FedRAMP Certification
Relativity announced this month that it expects to achieve the FedRAMP cloud security authorization later this year.

2/6/2020: findstr command to filter out files with given extension
dir | findstr /v “.pdf”

2/7/2020: NIST Study on Collection of Cell Phone Data
The study showed that Cellebrite Physical Analyzer could not collect stand-alone files and GPS related data on the HTC One Mini.

2/8/2020: NIST Study on Collection of Cell Phone Data – Paraben Electronic Evidence Examiner
NIST/DHS study on collection of data from Android phones using a Paraben tool shows when certain types of data are missed.

2/9/2020: JTAG, Chip Off, and ISP Extractions of Data from Mobile Phones
JTAG data extraction: soldering wires to ports; ISP data extraction: soldering wires to board; Chip Off: the memory card is removed and inserted in a specialized reader.

2/10/2020: Bloomberg’s Brief Analyzer
You can download cases cited in a brief faster than with the Lexis Get & Print tool.

2/11/2020: Exchange Litigation Hold Can Max Out Recoverable Items Folder
The default max size is 30 GB.

2/12/2020: SEC Letter: Produced Exchange Order Audit Trail Info Not Covered By Regulation SCI
Regulation Systems Compliance and Integrity covers all national securities exchanges and requires that their trading systems have adequate security, capacity, and integrity.

2/13/2020: Zero Trust
Zero Trust refers to an approach to cyber security that operates on the premise that an organization should never trust any outside user, device, network, or application.

2/14/2020: RBAC
RBAC stands for role based access control.  This is a security protocol which is commonly used in large organizations.  The key concept is that permission to perform operations and access to objects are assigned to specific roles, rather than individuals.

2/15/2020: Nevada Court of Appeals: No Plain Error in Admission of Facebook Profile Exhibits Without Showing Acquisition with Factual Specificity
Nevada Court of Appeals rejects abuse of discretion claims in admission of Facebook, Text Message, and Phone Tracking App ESI.

2/16/2020: Clearing iPhone RAM
iPhone RAM can be cleared by going to Settings . . . General . . . Shut Down and holding the home button for 5 seconds.

2/17/2020: E.D. Tex.: Cloned Discovery from Another Case Is Improper
Cloned discovery is only permitted when the fact of the production of the documents in another case is relevant to the present case.

2/18/2020: Excel DATEDIF formula to show time gap between two dates
show day, month, or year gap between two dates in





2/19/2020: Excel formula to check how much two cells match




ROW(INDIRECT(“A1:A” &C2)))))

2/20/2020: Edit Video Clips in Windows 10
Don’t miss that you can easily edit video files in Windows 10.

Right click on a video file in Windows Explorer and select Open with . . . Photos

2/21/2020: Avoid Zero Result with ISBLANK
In Excel use the ISBLANK formula to avoid a 0 result.

2/22/2020: App to communicate securely with witness
A judge ruled the WitComapp does not comply with New York state law.

WitComapp image

2/23/2020: PowerShell Command to Create a Zip File
Compress-Archive -Path F:\MyData -DestinationPath F:\foofolder

2/24/2020: Get List of Restore Points
PowerShell Command Get-ComputerRestorePoint

PowerShell Command Get-ComputerRestorePoint

2/25/2020: SQL Fiddle
You can use the free online tool SQL Fiddle to practice your SQL scripting.  See:

2/26/2020: New Jersey Appellate Court Denies Motion to Suppress Warrantless Disabling of Cell Phone Lock
The New Jersey Superior Court Appellate Division held that putting a cell phone in airplane mode and disabling the lock on the phone was unlawful.  However, citing the decision of the Supreme Court of United States in Riley v. California, 573 U.S. 373 (2014), the Court found that there was no Fourth Amendment violation because no data was taken from the phone.

2/27/2020: Auto animate bullets individually
In PowerPoint you don’t have to animate each bullet point one by one.

Image of Auto animate bullets individually

2/28/2020: Full Duplex
Full Duplex is a term used to refer to data transmission that operates at the same speed between two devices in both directions simultaneously.

2/29/2020: The Official Regex Search for Email Addresses
There is an official regex to search for any email addresses permitted under the RFC 5322 protocol.