Skip to content

Main Insight

The Future Society, with support from IEEE, has developed a model protocol to assist parties seeking to establish the trustworthiness of advanced tools used to review electronically stored information (ESI) in legal discovery.

Model Protocol for Electronically Stored Information (ESI)

October 2, 2023

The Future Society, with support from IEEE, has developed a model protocol to assist parties seeking to establish the trustworthiness of advanced tools used to review electronically stored information (ESI) in legal discovery. The potential that advanced review technologies hold for advancing a “just, speedy, and inexpensive determination of every action and proceeding” has been clear to litigants and the courts for over 15 years. The recent advent of legal applications based on AI systems, such as large language models (LLMs), brings additional hope for advancing this objective. However, attempts to realize the potential often encounter obstacles, such as costly and time-consuming meet-and-confer processes and (reasonable) uncertainty with regard to the effectiveness of such tools.

This effort, led by Dr. Bruce Hedin, aims to provide a statistically robust method for evaluating the results of an AI-assisted review of ESI in legal discovery, thereby reducing the barriers in the way of the effective adoption and use of such tools. The protocol also serves as a model for the rigorous assessment of advanced tools, including AI systems based on LLMs or foundation models in other domains.

The Model Protocol for Electronically Stored Information publications comprises two documents: the Protocol and Commentary and the Guidelines for Practicioners.

The Model ESI Protocol (“Protocol”) specifies procedures for validating the results of a review effort; the Commentary to that protocol (“Commentary”) provides guidance on implementing those procedures. The Protocol and Commentary will substantially meet the needs of most practitioners.

The Guidelines for Practitioners (“Guidelines”) provide an additional level of depth required in circumstances in which a deeper understanding of the concepts and methods that underlie the approach to validation and the steps to be followed in executing its procedures is required, such as when answering a question raised by opposing counsel or when adapting to a mid-review change in the data landscape.

This work was made possible with generous financial support from IEEE and gracious support from a number of distinguished legal experts. More information can be found in the “On Preparation of these Documents” and “Acknowledgements” sections of both documents.

Team members

Related resources

Springtime Updates

Springtime Updates

A leadership transition, an Athens Roundtable report, and a few more big announcements!

Leadership transition announcement

Leadership transition announcement

Earlier this year, after an extensive search for The Future Society’s next Executive Director, the Board of Directors selected Nicolas Moës, former Director of EU AI Governance, to fill the role.

TFS feedback reflected in historic UN General Assembly resolution on AI

TFS feedback reflected in historic UN General Assembly resolution on AI

Earlier this year, we provided feedback on a draft UN General Assembly resolution on AI. Last week, an updated resolution that reflected our feedback, co-sponsored by more than 120 Member States, was adopted by consensus.

TFS joins U.S. NIST AI Safety Institute Consortium

TFS joins U.S. NIST AI Safety Institute Consortium

TFS is collaborating with the NIST in the AI Safety Institute Consortium to establish a new measurement science that will enable the identification of proven, scalable, and interoperable measurements and methodologies to promote development of trustworthy AI and its responsible use.

Towards Effective Governance of Foundation Models and Generative AI

Towards Effective Governance of Foundation Models and Generative AI

We share highlights from the fifth edition of The Athens Roundtable on AI and the Rule of Law, and we present 8 key recommendations that emerged from discussions.

Launch event for the Journal of AI Law and Regulation (AIRe)

Launch event for the Journal of AI Law and Regulation (AIRe)

TFS is partnering with Lexxion Publisher to launch The Journal of AI Law and Regulation (AIRe), covering key legislative developments in AI governance and providing an impartial platform for discussing the role of law and regulation in managing the challenges and opportunities of AI's impact on society.