Typically, if you hear “technology-assisted review” echoing in a room, you’re probably in a courtroom. Surprisingly, that may no longer be the case. Maura Grossman and Gordon Cormack are taking technology-assisted review (TAR) out of the courtroom and into a new frontier – the science lab.

How Technology-Assisted Review Could Uncover COVID-19 Treatments

Grossman and Cormack, prominent leaders in data science technology for litigation, have teamed up with the knowledge synthesis team at St. Michael’s Hospital in Toronto. The goal is to help scientists sift through and automate searches of all the existing data on previous pandemics, previous clinical studies on different respiratory diseases, as well as the many recent studies from around the world on COVID-19. All of this data holds crucial information, which when pieced together, may lead to better treatment or even a cure for this terrible disease. Lives depend on uncovering key research, ideas, and treatment outcomes quickly and TAR could be part of the solution.

The goal of this effort is to use a type of technology-assisted review to sort through the mountains of studies, research, and clinical reports to quickly gather evidence of useful treatments. The world is still learning so much about COVID-19 in a short amount of time, but to synthesize all this information, to identify related treatments or approaches, and to do a real systemic review takes precious time. To help the team quickly identify clinical studies related to a particular medicine or approach, Grossman and Cormack are providing these medical researchers with a powerful tool from the world of legal discovery – Continuous Active Learning (CAL).

Consolidating Mountains of Data Using Continuous Active Learning

CAL is a supervised machine learning workflow that takes any available example documents provided and stack ranks a collection from most to least relevant. As people review the top ranked documents and determine whether the information is relevant to the current search, the system learns from these decisions and quickly updates the stack. The reviewers can quickly reach the most obvious, relevant documents in the collection and then continue the process to the point of diminishing returns. In legal discovery, CAL is especially useful when the number of relevant documents is very small in relation to the size of the collection. Without CAL, reviewers would have had to manually sift through multiple languages, the screening of abstracts, and even read through full papers, which can take months, but with the use of CAL, the review was accomplished in a matter of weeks.

Discovering Lost Treatments

From previous studies conducted during other pandemics, such as from the SARS outbreak in 2002 or the MERS outbreak in 2012, there is much to be learned. Using CAL to review the data from these previous pandemics, amongst other data from clinical trials done on similar respiratory diseases, officials are able to quickly draw sound conclusions about the treatments and other procedures that are likely to be the most promising, but also, they are able to dismiss those which are not useful. Consequently, decision makers have the ability to create smart policy initiatives and determine the best way to allocate funds.

Technology-Assisted Review vs Manual Review

While using TAR to combat a pandemic is unique, this type of vast data repository presents similar problems faced by law firms when there are mountains of corporate data to sort and eDiscovery obligations pressing for time. One study estimates that when TAR review is matched against a pure manual review, the manual reviewers can miss up to 75% of relevant documents. TAR helps people incur a fifty-fold savings over a traditional exhaustive review. In either case, getting through this data in the traditional way is an extremely time consuming and labor intensive process.

Grossman and Cormack are influential leaders in the overlap between data science, information retrieval, and legal discovery. We salute their efforts now in using these powerful tools in pursuit of treatment for COVID-19.