Meeting All Three Goals of Early Case Assessment with Sampling: A Powerful Yet Underused Tool

Most legal teams are underusing sampling, and it’s costing them, as sampling methods provide a solution to an otherwise costly fulfillment of a discovery request, which may not yield relevant information.

Sampling is one of the most powerful approaches to meeting all three goals of early case assessment (ECA); however, it remains largely underutilized. This can be attributed to several factors, with relevance and complexity of this method to name but a few.

When it comes to executing a successful ECA process, there are quite a few ways to find materials you would expect to be in a collection of electronically stored information (ESI); these include: searching tools, email threading tools, duplicate handling tools, advanced analytic tools, and sampling tools.

While there are many tools lawyers and broader legal teams can use for ECA, sampling is a persistently potent tool for all three ECA goals we discussed in a prior article.

This article explores sampling for ECA, including its different varieties, why it remains underutilized and most importantly, its powerful capabilities for meeting the three goals of ECA.

Understanding Sampling in Early Case Assessment (ECA)

Sampling, or data sampling, traditionally refers to analyzing and inferring information about a full population based on observations of a subset of the population.

In the context of eDiscovery and ECA, sampling involves selecting a smaller, representative subset of ESI to analyze and make inferences about the larger population of ESI. This also includes scrutinizing a small set of documents for relevancy to determine how many relevant documents may exist within an ample data set.

This technique serves as a useful tool in the overall ECA toolbox. There are several reasons behind its utility, namely because it works via the premise that by reviewing a representative subset of documents, you can draw accurate conclusions about the entire data set.

The Pertinent Examples of Sampling in ECA

While there are various kinds, sampling for legal ECA has two main varieties, those of judgmental sampling and formal sampling. Understanding both will help make your ECA process as effective as possible. It’s key to understand that there are also other sampling methods, such as stratified sampling, cluster sampling, or systematic sampling, for example.

Judgmental Sampling

Judgmental sampling is an informal and intuitive process; this involves looking at some randomly selected materials to get an anecdotal sense of their contents. This is for whether you sample from a particular source, from a particular search’s results, or from a particular time period. With this kind of sampling, you’re not reviewing a specific number of documents or taking a defined measurement with a particular strength. Instead, this involves getting an impression with which you then use to make an intuitive assessment with. Essentially, this involves checking through a few sets of documents to get a feel for the data set. This approach doesn’t rely on formal validity but can help identify patterns and red flags early in the process.

Formal Sampling

Formal sampling is the opposite, as it involves reviewing a specified number of randomly selected documents with the goal of taking a defined measurement with a particular strength to draw more reliable conclusions. This measurement can include either how much of a particular thing there is within a collection, (such as with estimating prevalence), or how effective a particular search is, such as that of testing classifiers. It’s more structured and delivers insights with measurable confidence.

The difference between these two categories is mathematical rigor, as judgmental sampling is anecdotal and intuitive, while formal sampling is statistical and measured. Simple Random Sampling is one of many kinds of formal and judgmental sampling that might occur

Why Sampling is Underutilized in ECA

Given that sampling helps you draw accurate conclusions and that its varieties help you ascertain different aspects of ESI, it may seem far-fetched that this tool is underused when it comes to ECA. Despite its benefits in discovery, we continue to hear our clients are not using sampling as a step in their process. Here are some of the reasons why:

1. There is a lack of knowledge about sampling techniques and how they can be leveraged among legal teams.
2. There is a perceived lack of time to undertake a robust sampling effort before diving into primary review.
3. Perceived complexity: Given that sampling includes selecting a representative portion of a dataset for a detailed review, this may discourage legal teams relying on this technique, as it’s regarded as complex and difficult.
4. Continuation of more traditional tools: Some legal teams prefer taking traditional, linear review processes, in which they review documents one by one, rather than looking at smaller representative subsets.
5. A lack of key resources: Some legal teams lack the expertise and resources required to effectively perform an ECA through sampling.
6. Misconceptions about missing information: Some legal experts may falsely perceive this method to miss relevant information, and then lead to incomplete or inaccurate case assessments, despite the fact that a sample is representative of the larger population. As such, some people may not detect any benefits or values from it.

The Power of Using Sampling for Early Case Assessment (ECA)

Sampling is one of the most powerful tools in your ECA toolbox. The following goes into depth of the power that sampling brings to eDiscovery, namely ECA.

Why Sampling Matters

First off, sampling enables your legal team to expeditiously learn about the materials you have, what they may contain, and how best to uncover more of what you need. It operates without the risk of missing any key unknowns that may appear when you rely solely on searching.

By reviewing a randomly selected cross section, you will see some of everything in your larger population and some of all the different terms and phrasing your custodians use to help you make informed decisions on your following ECA steps.

Sampling helps you estimate costs and risks, as it involves estimating the scope of your case, the potential costs of discovery, and the likelihood of finding relevant evidence. By focusing on a sample rather than all the documents, legal teams can then estimate the overall cost of eDiscovery, including collection, review, and production.

By analyzing a representative subset of data, you can also help develop effective search terms to be used in the eDiscovery process. This way, your team will only pull relevant information.

Moreover, sampling offers time savings, as attorneys and legal teams don’t need to comb over every single document, but rather a smaller subset. This helps attorneys build a stronger case and anticipate potential counterarguments without having to sift through documents that may not have any relevance to the case. While time savings are a benefit of sampling, sampling itself is not a substitution for review. That’s because sampling is one of many steps in a review preparation process that can be used to reduce and/or prioritize the review population.

What Sampling Unlocks

More informed decisions:

Sampling results in a clearer picture of the data you interpret, as it informs strategic choices about settlement considerations, resource allocation, and a case’s direction. This helps enable legal teams to focus on the more pertinent information, thus reducing the risk of overlooking critical evidence.

Stronger analyses:

Sampling enables legal practitioners to analyze a representative subset of data, cancelling out the use of anecdotal guesses and replacing them with statistically grounded insights. As such, this method is not only crucial for technology-assisted review (TAR), as it also offers significant benefits throughout different phases of eDiscovery, including litigation and investigations. By examining a small, manageable portion of data, legal teams can draw accurate conclusions about the entire dataset, leading to stronger analyses.

Speedier Insights:

Sampling quickens the identification of key documents, patterns, and anomalies, allowing legal professionals to prioritize their review efforts effectively and efficiently. This kind of expedited assessment is key in time-sensitive situations, including internal investigations or compliance reviews.

Cost Benefits:

Using sampling can significantly reduce the volume of documents requiring a full review, therefore leading to substantial cost savings.

All in all, sampling is a powerful tool that legal teams would be remiss not to incorporate into their ECA efforts. It helps manage the difficulties of eDiscovery, make better decisions about case strategy, and improve the efficiency of the litigation process.

How Sampling Contributes to the Three Goals of ECI

Sampling plays an important role in ECA, particularly for its three goals. When it comes to the first goal of traditional ECA, sampling informs early risk assessment and helps shape strategy before heavy resources are committed.

1. Traditional Early Case Assessment: Evaluating Case Risk and Merits

In the ECA phase, sampling helps traditional ECA, as it enables teams to gain an early, data-backed understanding of the case’s factual landscape before diving into full-scale review. By drawing statistically valid samples from the collected dataset, legal teams can:

• Gauge case exposure early by seeing representative examples of key documents (namely emails, contracts, communications).
• Validate assumptions about the strength of evidence supporting or undermining claims.
• Estimate potential review volumes and costs, as sampling provides early insight into how much relevant or privileged material might exist. This fosters better budgeting and strategic decisions.

2. Early Data Assessment (EDA): Assessing Data Completeness and Quality

In EDA, sampling is primarily utilized for data validation and quality control. Moreover, sampling in EDA is diagnostic, as it builds confidence in the integrity and scope of the dataset before deeper analysis or review. As such, teams can:

• Verify data completeness by sampling across custodians, file types, or collection sources to ensure that all the relevant data has been gathered.
• Detect anomalies, and this includes missing metadata, corrupted files, or unexpected data types. All of these can complicate processing or review.
• Estimate the proportion of responsive or privileged material in the dataset, helping calibrate filters, search terms, and culling strategies.

3. Review Preparation: Optimizing Review Efficiency and Strategy

During review preparation, sampling enables testing, validation, and process optimization before full-scale attorney review begins.

Teams can test search terms, analytics, and workflows by sampling results to confirm they capture relevant documents while minimizing false positives.

They can also validate machine learning or TAR (Technology-Assisted Review) models through control or validation samples to confirm acceptable accuracy levels. In addition, teams can forecast review effort and staffing by sampling to estimate document relevancy rates and likely review speeds.

Sampling in this goal is tactical; it refines review strategy, reduces error, and improves predictability in downstream workflows.

Mastering ECA with the Use of Legal Experts

While sampling is a mighty tool for ECA, you shouldn’t go at it alone. Our team of legal experts are here to help with all of your ECA tasks and concerns, including sampling.

Sampling is a strong tool, but with the sheer volume and complexity of today’s data, you need more than just tools—you need a partner who brings experience, technology, and insight to the table.

In addition, we offer a suite of tools as part of Aurora, our trailblazing Digital Enterprise Platform that changes how legal teams manage data, eliminating the compromise between standardizing on a single platform and fragmenting across multiple review solutions.

With Aurora CoreECI, we help you excel at early case assessment. Specifically, Aurora CoreECI offers powerful early case assessment capabilities with advanced searching and filtering, enabling rapid, data-driven decisions early on. By leveraging machine learning and natural language processing, CoreECI surfaces key insights and trends, helping you make data-driven decisions from the outset of your matters.

That’s how Consilio stands out. From our team of legal specialists to our suite of tools, such as Aurora, Consilio packs everything you need to manage your legal cases and issues.

Work with us today.

Get a deep dive of ECA fundamentals: