Elevate’s Jeremy Pickens recent article “Why More Accurate (Gen)AI Doesn’t Always Reduce Total Document Review Costs” raised several good points using different eDiscovery review scenarios. That inspired me to adapt some of my own tools into a free review budget calculator where you can use your own collection parameters, rates and assumptions to project the potential accuracy, cost and time to completion for counsel’s risk/cost decision.
Recap for those who TLDR:
- Regrave LLP case study “Generative AI for Complex Document Review” provides rare, scholastic metrics and insights comparing accuracy and time of CAL vs. GenAI.
- My blogs riff on the study parameters and create a free tool to add costs into their results.
- Jeremy Pickens (link above) further expands the discussion with example scenarios that show how CAL can actually be more cost effective depending on collection parameters.
- The eDiscovery Journal Tools page – eDiscovery Review Budget Calculator Excel workbook that allows you to input your own parameters and assumptions.
Tool creation, research and notes:
Your AI of choice can generate a cost model workbook in minutes. Referencing prior tools, the Redgrave Study and Pickens article gave Claude some context for the first version. But the ‘Devil is in the details’ when creating any kind of data model. Redgrave LLP, Pickens and many other peers have relayed that GenAI relevance prompt refinement can day literal billable days. So did the multiple iterations, checking and tuning to get the review budget calculator to factor in different steps and approach options.
That tool tuning sent me down a couple research paths (See Reference Benchmarks worksheet) to somehow model the relationship between collection-scope complexity and how much CAL review may be required before ‘model stability’ is reached (i.e. you can quit reviewing). Claude was stumped, but summarized the available study results. Within those I spotted a correlation pattern that I used to create an estimated TAR 1 stability volume based on richness multiplied by a 1-5 complexity scale.
Example: a collection with a 10% richness and a very high complexity (5) could require CAL review of 50% of the total volume to reach stability. The Redgrave LLP study was well out of bounds (77% reviewed), but you can easily modify the relevant formula to more closely resemble your prior reviews.
This tool could easily be extended to cover more of the eDiscovery lifecycle. I have created many similar matter budget models for clients based on average custodial metrics during the hold phase. AI speeds up the process, but it requires serious supervision and expect to scrap and restart a few times. It does not have our experience and context to draw on. So it will leave out factors and options while looking great. Have fun with the tool and love feedback or variations.
I am still booking briefings and peer conversations, so grab a time slot if you are joining me in Chicago if you have recovered from ILTA!
Greg Buckles wants your feedback, questions or project inquiries at Greg@eDJGroupInc.com. Reach out for a free 15 minute ‘Good Karma’ call if he has availability. He solves problems and creates eDiscovery solutions for enterprise and law firm clients.
Greg’s blog perspectives are personal opinions and should not be interpreted as a professional judgment or advice. Greg is no longer an investigative journalist and all perspectives are based on best public information. Blog content is neither approved nor reviewed by any providers prior to being published. Do you want to share your own perspective? Greg is looking for practical, professional informative perspectives free of marketing fluff, hidden agendas or personal/product bias. Outside blogs will clearly indicate the author, company and any relevant affiliations.
Greg’s latest nature, art and diving photographs on Instagram.