Practical QC in eDiscovery

Migrated from eDJGroupInc.com. Author: Greg Buckles. Published: 2011-12-27 05:02:47Format, images and links may no longer function correctly. One key element for transforming your eDiscovery from an ad hoc reactive fire drill into a mature, proactive business process is the development and implementation of formal Quality Assurance (QA) and Quality Control (QC). I have always viewed QA as tackling ongoing process improvement such as regular cross case comparisons, while QC tends to be checking on did the process perform properly. Basically, how can we make the process better versus did everything work right? When interviewing corporate client eDiscovery teams, everyone is conscious of the need for QA/QC, but the vast majority seem to feel that it is impractical or unrealistic given their tight deadlines, lack of resources and typical fire-fighter mentality. Some law firm clients have swung to the opposite extreme, with elaborate workflow, check lists, physical chain of custody forms and more. Their QC has grown out of reasonable proportion and their productivity suffers because their overall QA has been neglected. So how do we achieve a reasonable quality process without bringing the legal process to a halt?

The key here is the term ‘reasonable’. There are no absolute standards in eDiscovery, because every matter may have unique requirements, ESI characteristics or other factors. However, we can assess the ‘typical ’ matters and ESI to establish baseline requirements and build in early check points where counsel is briefed and can determine any appropriate deviations. Given the recent interest in legal hold searches on corporate archives, I thought that a hold scenario would be a good practical example of quality controls with relatively few parameters.

Legal Hold Scenario:

• Counsel has initiated a legal hold on a typical contract dispute
• The eDiscovery team conducts early interviews and identification searches that produce a list of 15 key custodians for preservation
• The lead paralegal checks the custodian names and email addresses for known variations and logs the hold scope with their matter management system
• The lead paralegal executes preservation searches on the companies system (we are making this technology agnostic, so it could be directly on the archive or through another enterprise search system)

Now we come to our quality check example (though there are QC steps prior to this)

When the initial search results come back, our paralegal needs to look at the results and be able to document that QC checks were performed to defend this part of the legal hold process. For the purposes of this scenario, we are excluding local, offline, legacy and other ESI sources. The quick table below is an example of the fields that would be tracked in Sharepoint, your matter management system or even on a centralized spreadsheet.

Jumping down to the QC checks, let’s look at what they mean.

• Error Check – Check the search system to make sure that there were no errors thrown during the search. Most systems will alert you, but our paralegal should be given guidelines on understanding typical errors and how to report/document significant errors. For example, in large enterprise environments, there are always indexes being updated or sources that are offline, but many of these are not relevant to typical matters.
• Size/Scope Check – Was the total number of hits appropriate in size for the number of custodians and the date range? During early acceptance testing, the eDiscovery team established an average number of email/custodian/year along with guideline on acceptable deviations (+X%). So our paralegal does a quick calculation to compare the average number of email per year per custodian. This will vary by the date range (i.e. it grows per year), but it will help spot custodians with missing ESI.
• Date Check – Most systems will allow our paralegal to view the number items per year, quarter or month in the search filters or navigation facets. Others will create a date distribution report that can be used to quickly spot spikes or gaps that may need to be investigated.
• Custodian Counts Check – Just like your Date Check, this step uses navigation facets or reports to verify that our custodian ESI is evenly distributed. Counts by Author or Recipient can help you spot the kinds of name/address changes that happen in dynamic corporate environments. This is also a secondary QC check on the initial identification scope definition. Our scenario does not include search terms for preservation, but a quick search with a few unique terms/phrases can identify missing players within the social network.
• Source Check – Now our paralegal looks at the distribution of results across the ESI sources to spot gaps or missing sources.
• Name Variation Check – As an extra check, our paralegal reviews the variations of Author and Recipient names across the date range. Counsel is not worried about this, but our paralegal knows that the company had a major merger two years back and many users got new accounts at that time.

This is just a simple example scenario, but I hope that you can see how these checks could be performed and documented with minimal effort. QC is important. We work in an industry with an expectation of high quality and a documented process. You always want to be able to answer a challenge well after the fact with hard proof that you thought about quality, created a quality process and actually performed your QC checks.

Also I wanted to congratulate the winners of the eDiscoveryJournal Holiday Drawing.  I hope you were able to enjoy our gift to our readers.