ESI Culling

This technical note discusses how to perform culling prior to document review.

General

The process of culling (aka reducing or filtering) electronically stored information (ESI) is designed to defensibly reduce a large collection of documents to a smaller set for further review.  Depending on how documents were collected and other factors, culling can reduce data sets by as much as 90% or more (if broadly collected).  

Culling requires advanced litigation support/legal, case, and technical knowledge.  Quality Control procedures are important for each step in the process.  Custom training sessions and Professional Services are available (billable at the standard hourly rate).

Prior to Culling

Prepare separate upload batches by custodian and assign the custodian to the upload batch.  If documents have been uploaded without custodian assignment, custodians may be assigned subsequent to upload by filtering on an upload batch or custodian-specific portions of the SourceFilePath.  Use multi-doc edit to assign a custodian. See Custodians for more information.  

Do not begin culling until all processing has completed. There should be no active processing in the case (by any user) during culling. 

Prior to culling the user may wish to remove exact (hash) duplicates from the case.  See Deduplication for more information. 

Culling by Custodian

Documents may be broadly collected from custodians initially.  Courts often limit discovery to a determined/negotiated number of custodians.  It is appropriate to collect and preserve ESI from custodians who may later be culled from the case.

Assuming custodial assignment, as described above:

Set up a Temporary Culling Custom Field.  Go to Management->Manage Custom Doc Fields (Account Admin or Power User) and set up a custom section entitled Custodian Culling with a Custom Checkbox field entitled Custodians to Cull (delete).

Tag Custodian Documents to Cull.  Go to Browse and show the Custodian field and the custom field Custodians to Cull (delete). Apply filter for a specified Custodian.  Select all documents in the view and use multidoc edit to code all to the custom field Custodians to Cull (delete). The custom field Custodians to Cull (delete) should populate.  Repeat for other custodians.

Filter on All Documents from Custodians to be Culled. When coding for custodians to cull is complete, go to Browse and apply a filter for the custom field Custodians to Cull (delete).  Only the custodian documents tagged for culling/deletion will display.

Download a Log of Documents Culled by Custodian.  If needed for preservation, download a log of all documents before deletion.  See Log of Documents Deleted in a Cull below.  

Download Archive (if needed).  If needed, select all documents (with the cull filter applied) and Export to Briefcase to retain a local copy for archive purposes.  Wait for the Briefcase to complete before continuing.  

Delete Documents to be Culled by Custodian.  Using the same filter, select all documents and delete all.  The documents in the case will be reduced by the number deleted. 

Delete Temporary Section and Field.  Delete the temporary section and field created above.

The above procedure assumes no custodian files have been changed to result in partial email families (in normal usage and coding this should not occur).  If this has occurred, before deletion, run the documents through a Review Set, as described below, to recapture email family associations within a single Custodian.

Culling by Date Range

Culling often involves removing documents outside of a specific date range.  Use the Master Date field for this purpose.  This field retains email family associations by assigning the email Date Sent not only to the email body, but also to the associated email attachments.  Any documents with no Master Date should not be deleted in a Date cull as the document date is unknown in the program.  Many documents in a typical ESI collection do not have a date retained in metadata and should not be deleted in a date cull.  To cull by date range:

Set up a Temporary Date Culling Custom Field.  Go to Management->Manage Custom Doc Fields (Account Admin or Power User) and set up a custom section entitled Date Culling and a Custom Checkbox field entitled Outside of Retained Date Range (delete).

Code Documents Outside of Date Range. Go to the Browse page and show the Master Date field.  Apply filter by a Master Date with the retained date range as the start and end date, with the NOT checkbox checked, and tag the documents using multi-doc edit. 

Filter on All Documents from Outside of the Date Range to be Culled. When the user has completed coding all the date range, go to Browse and apply a filter for the custom field Outside of Retained Date Range (delete).  Only the documents outside of the retain date range tagged for culling/deletion will display.  

Download a Log of Documents Culled by Date Range.  If needed for preservation, download a log of documents before deletion. See Log of Documents Deleted in a Cull below.  

Download Archive (if needed).  With cull filter applied, select all documents and Export to Briefcase to retain a local copy for archive purposes.  Wait for the Briefcase to complete before continuing.

Delete Documents to be Culled by Date Range.  Using the same filter, select all documents and delete all.  The documents in the case will be reduced by the number of documents deleted.

Delete Temporary Section and Field.  Delete the temporary section and field created above.

The above procedure assumes no files assigned to the date range have been changed to result in partial email families (in normal usage with the Master Date field and coding this should not occur).  If this has occurred, before deletion, run the documents through a Review Set, as described below, to recapture email family associations.

Keyword Culling

Culling often involves removing documents that do NOT match certain keywords in Search.  Keywords can be negotiated with opposition, independently determined or court ordered.  Keywords are determined from analysis of Requests for Production or Subpoenas, and other relevant court filings or orders. In general, keyword-based culling involves searching and coding all documents returned by keyword searches, and then deleting the documents (after adjusting for partial emails families as described below) not returned by keyword searches. The following steps describe how to cull by keyword:

Set up a Temporary Culling Custom Field.  Go to Management->Manage Custom Doc Fields (Account Admin or Power User) and set up a custom section entitled Keyword Culling and a Custom Checkbox field entitled Keywords Hits to Retain.

Search and Code Keyword Hit Documents to Retain.  Go to Search and show at minimum the custom field Keywords Hits to Retain.   Apply keyword searches to get keyword hits.  After each search, select all documents in the view (the search results) and apply multi-doc edit to code the custom field Keywords Hits to Retain.  Repeat for all keyword search terms.

Create Review Set to Recapture Partially Email Families.  Using the above method of identifying documents to retain, will invariably result in partially coded email families.  The user must use a Review Set to recapture those associations.  See Expand Documents to Retain in a Cull to Avoid Partial Email Families below.

Filter on All Documents not included in Keywords Hits to Retain. Once the documents tagged to the Keywords Hits to Retain have been expanded, go to Browse and apply the filter NOT Keywords Hits to Retain by selecting Keywords Hits to Retain=checked and also clicking Not checkbox at the section level (here Keyword Culling).  Only the documents not tagged by the Keywords Hits to Retain field and intended for culling/deletion will be displayed.

Download a Log of Documents Culled by Keyword Hit.  If needed for preservation, download a log of all documents to be deleted.  See Log of Documents Deleted in a Cull below.  

Download Archive (if Needed).  If needed, select all documents (with cull filter applied) and Export to Briefcase to retain a local copy for archive purposes.  Wait for the Briefcase to complete before continuing.  

Delete Documents to be Culled by Keyword Hit.  With cull filter applied, select all documents and delete all.  The documents in the case should be reduced by the number deleted.  A filter of NOT Keywords Hits to Retain should no longer return documents.  

Delete Temporary Section and Field.  Delete the temporary section and field created above.

Log of Documents Deleted in a Cull

The user may wish to download a log of documents culled for evidentiary purposes.  If needed, prior to deleting documents as part of a cull, filter on the documents and download a log (Export->Export Log to Excel) and retain.  Any available fields can be exported.  The following are field export suggestions: Title, Extension, Source FilePath, Master Date, Date Sent, Date Received, Date Last Modified, Custodian, From, To, Cc, Bcc, Subject, IsEmailAttachment, and custom fields.  The Excel file can include a sheet explaining the date of the cull and the cull criteria used. 

A log may also serve to prepare statistics of the number and type of documents deleted as part of a cull, if needed.

Expand Documents to Retain in a Cull to Avoid Partial Email Families

Do not delete partial email families.  As noted above, partial email families can occur when deletions are based on keywords (e.g., keyword-based culling).  To avoid this, before deleting documents the user should follow the procedure below to add entire email families into the tag that is to be used as a basis for document deletion (e.g., Keywords Hits to Retain).  

Create a Review Set.   Go to Account->Manage Review Sets (Admin or Power User) and create a new Review Set entitled Expand Email Families MM-DD-YYYY.  In this case, this procedure is used just to expand email families.

Add Documents to Be Retained to Review Set.  Go to Browse and apply a filter of the documents to be retained.  For the keyword culling, this would be the custom field Keywords Hits to Retain.  Select all documents in the filter and then select Review Sets->Expand Email Families MM-DD-YYYY. 

Start Review to Expand Email Families.  Go to Account->Manage Review Sets and select the review set Expand Email Families MM-DD-YYYY.  Click Start Review.  This will increase the number of documents in the review set to include any email bodies or attachments necessary to complete partial email families.

Increase Keywords Hits to Retain to Complete Email Families.  Click on the title of the Review Set or go to Browse and filter on the Review Set (Select Filters->Review Set->Expand Email Families MM-DD-YYYY).  Select all documents in the filter and multi-doc edit to apply to the field used, e.g. Keywords Hits to Retain.  

Complete Cull.  Complete the applicable cull.

Delete Review Set.  Delete the temporary Review Set created above.

Quality Control

We recommend applying quality control procedures to sample and test before deleting documents.

Very Large Multi-Doc Edits or Deletes

For very large multi-doc edits or deletes, the user may need break the instruction up into smaller parts.  If you have difficulty with this, contact Professional Services for assistance.