Solving the ‘AI Trust Gap’ with Expert-Led AI

A substantial execution challenge has arrived with the broad Generative AI access that has swept the globe since 2022. While most organizations and professionals driving mission-critical outcomes see GenAI’s potential, they are also largely still early in understanding how to effectively harness it.

We saw this in our recent Consilio global survey centered around legal services professionals: while 90% of respondents confirmed they are experimenting with AI, only 10% have adopted it in substantial production manners. A lack of trust in AI to deliver quality results is clearly a key factor in this lack of adoption. This trend also extends across other industries, as applying AI to work that requires a high degree of accuracy takes a remarkably high level of care.

Being that we at Consilio are the largest global provider of tech-enabled legal services, we’ve found a unique way to tackle this trust gap to get the most from AI as it evolves. Given that we offer eDiscovery, document review, flexible talent solutions and legal transformation services, we’re no strangers to AI, especially due to our decade-plus experience and millions of documents processed using GenAI’s precursor, machine learning analytics that in legal services are called Technology Assisted Review (TAR).

The Consilio signature approach is to pair seasoned subject matter experts with the technology even after it is deployed – going beyond model building to the hands-on execution of AI-driven work. We call this process “iterative prompting,” and it has generated market-leading quality and efficiency for clients.

In highly nuanced fields like eDiscovery, where sifting through the right documents from a stack of millions is a nuanced, specialized skillset, we have found that applying subject matter expertise to the custom, innovative models our team deploys is successful, in that it mitigates the very broadly known issues AI has with hallucinations and other inaccuracies. Guiding Consilio’s “multi-model validation” process – in which multiple AI models validate each other's output, these “AI Guide” subject matter experts are how we at Consilio are able to deliver trustworthy AI performance.

“Guided AI Review” is the first product Consilio launched using this expertise-driven approach. The product has delivered industry-leading “precision” and “recall,” which translates into higher quality AI output. Consilio has over 130 advisors trained in this, and a total of 250 million documents have been analyzed across all its analytics engagements.

Across all industries, the success Consilio has seen placing hands-on subject matter experts in the midst of GenAI activity bears insights. Given the known gaps in AI that will continue for some time, and given how the technology will continue to shift radically, expert-led AI creates important learning and validation for organizations as they continue to learn.

Interested in elevating your eDiscovery experience with AI?

‍