Migrated from eDJGroupInc.com. Author: Greg Buckles. Published: 2015-02-17 19:00:00Format, images and links may no longer function correctly.
I had one of those ‘What the heck does this mean?’ moments when I read my morning web feed. The headline is, “Electronic Discovery Leader Discovia Achieves 99/7% Accuracy Rate.” Don’t get me wrong, I am all about using metrics and measurements to improve process efficiency and quality. Discovia obviously has invested in automation, formal procedures and documentation (required for ISO 27001 certification). I see this kind of over-simplified marketing spin on metrics all the time since predictive coding (PC) became a selling point. Metrics and statistics can be used or abused to support many assertions. In the case of PC, I see providers using ‘accuracy’ to up-sell analytics without acknowledging the complex trade offs in precision and recall that impact the production size and composition. Attorneys have always known that if you want to produce 100% of the relevant collected documents, then forget relevance review and produce everything. Misunderstood or oversimplified metrics have become eDiscovery ‘black magic’ that reminds me of the early days when techno babble could be used to obfuscate human or process failures from a baffled bench. Getting back to translating this particular press release, let us look at the language:
“…it achieved a 99.7% Accuracy Rate on over 14,000 client service requests in 2014.”
I interpret this to mean that they had roughly 42 ‘internal and client-identified errors’ in 2014. Definitely an impressive rate of reported ‘errors’, but what qualifies as an error? A typo on a CD label has negligible impact compared to accidentally producing privileged docs. I probably would not have had such a strong reaction to the headline if it were not followed up by:
“Company is First Industry Services Provider to Publish Quality Metrics” and “Discovia is the first eDiscovery services firm to deliver a fixed-price managed services solution…”
eDiscovery consumers are some of the most cynical buyers that I have ever seen. They ought to be when any provider uses the words ‘first’, ‘only’ or ‘best’ in any context. I wish that marketers did not feel the need to hype real, credible news to grab our attention. The concluding quote does a great job of communicating the practical take-away of Discovia’s 2014 quality metrics without over inflating them.
“Our accuracy rate is the result of our corporate culture and commitment to quality,” said Talin Andonians, Discovia’s Vice President of Operations. “We employ an exceptional team of professionals who focus on continuous improvement and documentation of our processes. To that end, we have made a significant investment in building a robust quality system featuring proven methodologies and very stringent protocols.”
Greg Buckles wants your feedback, questions or project inquiries at Greg@eDJGroupInc.com. His active research topics include analytics, mobile device discovery, the discovery impact of the cloud, Microsoft’s 2013 eDiscovery Center and multi-matter discovery. Recent consulting engagements include managing preservation during enterprise migrations, legacy tape eliminations, retention enablement and many more.
Blog perspectives are personal opinions and should not be interpreted as a professional judgment. eDJ consultants are not journalists and perspectives are based on public information. Blog content is neither approved nor reviewed by any providers prior to being posted. Do you want to share your own perspective? eDJ Group is looking for practical, professional informative perspectives free of marketing fluff, hidden agendas or personal/product bias. Outside blogs will clearly indicate the author, company and any relevant affiliations.