Although we are still in the early days of applying AI to law, the question is not “Will AI work?” but rather how to make it work. Whether it is email, e-signatures, or due diligence, the increased efficiencies of new technology inevitably become standard expectations. With AI, we are currently in an exciting period where law firms can proactively develop strategies that work for them and their clients. This is a challenging task that demands understanding the inherent strengths and weaknesses of AI. However, the firms that figure out how to embrace and use this new technology will likely be the ones that come out ahead.

Introduction

“Artificial” intelligence (AI), defined in contrast to “natural” human intelligence, refers to the ability of a computer to do tasks requiring such high levels of discernment and specialized knowledge that they are normally considered exclusive to the realm of human cognition. Applying AI to legal work might not be as hyped as self-driving cars or computers becoming sentient,¹ but it does portend several exciting developments. One area of particular promise is using AI to turn contracts into computable data.

The application of AI to contracts is still in its early stages, though we have already seen it evolve in a myriad of ways. Merely reading documents and sifting out provisions was just the starting point. That technology has been applied to contract negotiations, due diligence, and analytics of all contracts within a company. These types of products provide new tools and new forms of legal information with which lawyers advise their clients. More recently, products are evolving in different directions. Some companies aim to broaden the application of AI from merely reading contracts to the entire contract lifecycle – drafting, negotiation, integration with other corporate departments, etc.² Still, others are deepening their specialization in the underlying AI engines, which can then be embedded into a variety legal workflows.³

Such a flurry of activity can make it difficult to assess how AI can fit into the day-to-day work of a practicing lawyer. This article looks at the actual practical considerations involved in using AI for due diligence. First, we take a closer look at how this technology is developed. With a better understanding of its inherent strengths and weaknesses, we then look closely at what the potential downsides might be. The final section of this article focuses on how to maximize the strengths of AI in a due diligence context.

Law firms are still in the early stages of finding the right balance between efficiency and attention to detail. They must also figure out how to adopt products that, in the short term, appear to eat into their revenue streams. However, those who proactively face these challenges will likely come out the furthest ahead.

What we talk about when we talk about AI

The study of how a computer can (“artificially”) develop a human-level capability (or “intelligence”) is known as machine learning (ML). This process, while still connected to computer science, is quite different from traditional programming methods.

Traditional programming occurs as you might expect: send the computer a program (i.e., a set of steps) and data inputs, then hopefully receive your desired output. The bulk of the work goes into continually checking the output and then modifying the program until it behaves as desired. In other words, the computer’s ability comes from a human programming it directly.

Machine learning occurs in a different fashion: send the computer data and desired outputs, then have it figure out a program to get there. The bulk of the work goes into creating dataset and defining its outputs – for example, lines of text (data) that all relate to governing law (output). In other words, the computer’s ability does not come from a human directly programming it. It instead “learns” through examples.

Providing these examples gives the computer a dataset from which it develops pattern recognition and statistical models. Each type of provision will require its own dataset on which to identify patterns and calculate a probability score. With the right dataset, the computer model can look across the patterns it has identified and determine that a certain text is probably, say, a change of control provision. Where a human intuits “this is a change of control provision,” the computer calculates a numerical value, such as a 91% probability that this pattern of text is a change of control provision. Though the answer from the computer appears as a binary yes/no, it is based on a sliding scale with a set cutoff number. For example, if the cutoff number is 70%, then the results will exclude any text where the probability of it being the specific provision is 69% or less.

Although these algorithms can be tweaked to allow for a certain amount of fuzziness, there are limitations to what they can accommodate, especially encountering a novel formation of a clause. Much like humans, the computer doesn’t necessarily need to see specific words to accurately classify a provision. That said, a computer is more constrained by the data it has seen than a human might be.

The risks of AI

A program’s ability to infer only based on past examples is a source of risk. While vendors can (and should) only release new models after ensuring they perform at a high level of accuracy on the widest possible variety of contracts, they will never be fully accurate.⁴ As provisions undergo negotiation, they can end up differing widely from the standard language that might have been used to train the AI product, increasing the chances that such language will be wrongly excluded. Maura Grossman, a leading expert in technology assisted review for eDiscovery, explains that “purveyors and pundits have grossly exaggerated the accuracy with which recall and precision can be estimated using sampling… At best […] they give a coarse estimate of recall with a margin of error several times larger than implied, which may identify a major failure, but cannot guarantee success.”⁵ Similarly, Gartner Consulting predicts that “through 2022, 85 percent of AI projects will deliver erroneous outcomes.”⁶

Reports or case studies from AI companies might be tailored to provisions or documents that have low variation and/or are widely available (e.g., NDAs). Thus, a glowing result might not be indicative of success in a more complex scenario. Also, AI capabilities can be particularly limited when the state of the law is in a period of flux. Because the program is learning only from past examples of a provision, it is susceptible to missing newer examples of a provision - examples that might reflect a trend of growing importance.⁷

In the highly confidential world of M&A, getting enough data can be an acute challenge. Kira, one of the leading companies in this space, notes that “finding sufficient amounts of legal documents to train machine learning models can be particularly difficult… Our efforts are often hampered by a lack of documents exhibiting the desired topic, so we are left to explore other topics or spend effort trying to find example documents should the estimated benefit be sufficient.”⁸

Further compounding this challenge is the fact that, after collecting the contracts, someone must comb through all of them in order to create enough examples that fit the label “assignment clause.”⁹ Often, this job requires experienced lawyers. Procuring sufficient hard-to-find documents and hiring lawyers to provide reliable datasets account in large part for what makes these tools expensive, and why companies and firms alike are very protective of this data.¹⁰

Due diligence represents a particularly challenging use case for computers. Unlike technical fields, which typically start with more computer-friendly data, computers must first translate the language of contracts into calculable data.¹¹ Unlike eDiscovery products, which typically reduce a set of documents based on a relevancy scale, due diligence products behave in a more binary fashion – a provision is either included or excluded. Furthermore, eDiscovery review typically seeks to match evidence to pre-established issues. Due diligence, by contrast, is a means of uncovering the issues themselves. When the potential downside of missing a key issue could equate to millions of dollars or lead to litigation, it is understandable why lawyers may be cautious to adopt this new technology.

The opportunities of AI

Despite the risks discussed above, AI can provide tremendous advantages for law firms. The same study that expected 85% of AI projects to deliver erroneous outcomes also found that roughly half of CIOs have implemented or plan on implementing AI. To not act presents an even greater risk of being left behind. Humans naturally raise their expectations once they are accustomed to a more capable or reliable product. Thus, the automation of due diligence via machine learning will lead to increased expectations from clients. After all, human review is not 100% accurate, either.

Making direct comparisons between AI products and lawyers can miss the point. The biggest opportunities for using technology in due diligence come from developing processes that complement the strengths of both lawyers and AI. Lawyers rely on a due diligence report to uncover material issues in a deal. For human lawyers to simply abdicate that responsibility would be negligent. Lawyers excel in the intricate, complex, and somewhat intuitive work of reviewing material contracts to parse unique, heavily negotiated terms. However, AI can excel in scanning what might be thousands of additional contracts which are less material. Thus, instead of expecting to replace lawyers, firms can gain more from AI by gaining a nuanced understanding of who reviews what.

Another area where computers excel over human lawyers is in the minutiae of managing documents and reviewers. In fact, some experts have determined that most of the products we associate with contract analysis are “primarily workflow tools that incorporate contracts AI.”¹² Any product in this space does indeed provide features such as dashboards, advanced searches, and linking reports directly to the original text in the document. Manually jumping between the assignment provisions across several documents introduces significant tedium that saps focus from the actual legal analysis at hand. Although workflow features have nothing to do with AI, they can still vastly improve a lawyer’s ability to deliver legal services.

It’s possible that AI companies might be able to offer products that better complement the strengths of human lawyers by actually focusing less on precise identification of provisions. They could shift resources from duplicating lawyers’ deep analyses to address other aspects of due diligence. For example, typical products already offer some high-level information such as a contract’s language or type (employment offer, NDA, etc.); these algorithms could be extended to identifying topics of law that require input from subject matter experts. Additionally, AI that could automatically connect an original agreement with amending agreements, or group together all agreements related to an MSA would be particularly useful in a due diligence platform.

As due diligence tools and processes become more sophisticated, law firms will need to adjust their processes and resources. As in eDiscovery, we can expect to see a growing specialization in the technical aspects of these programs: transferring and reorganizing documents, managing server space, and training reviewers. It will soon be (or is already) untenable to expect lawyers to do all this additional work, or to expect clients to pay for it at associate rates. Thus, as technology-enabled due diligence evolves, firms will likely need to experiment with new business models in order to capitalize on the opportunity to deliver more comprehensive legal services at a competitive price.

Conclusion

While there is a lot of speculation on what the future might look like, there are a few points we can firmly establish. First, the question is not “will AI work?” but how to make it work. Whether it is Outlook, Word, or something more cutting edge, legal work is deeply intertwined with technology. Recognizing that we are often already working in organizations of mixed machine and human intelligence encourages greater focus on the right uses for emerging AI innovations.  Clients expect their service providers, including law firms, to provide the most efficient and effective services.  This means that law firms that are not willing or able to grow and develop with the times to offer appropriate AI technology could find clients seeking those that do.

Second, the right balance between the complementary strengths of AI and lawyers is still undefined. Even in a field like eDiscovery, with its massive amounts of data and research confirming that “technology-assisted review can (and does) yield more accurate results than exhaustive manual review, with much lower effort,”¹³ courts are still reluctant to order its usage.¹⁴ Part of the caution around technology is that, in order to achieve its full benefits, lawyers must rethink their processes at a much deeper level than Zoom or Excel requires. While some law professors argue that “the true benefits of artificially intelligent tools in the legal profession may be realized only once lawyers completely rethink the provision of legal services,”¹⁵ the “complete overhaul” they recommend understandably sounds a bit reckless to lawyers whose fiduciary duty behooves them to be prudent.  Law firms and their clients will need to find a balance in the use of AI that ensures that work is done efficiently while at the same time being done with a level of care and attention to detail that important items are not missed or overlooked.  This balance will take some work, but those too afraid to find such balance are likely to be left behind.

Lastly, getting AI to work in legal services is as much of a business challenge as it is a legal process challenge. COVID-19 confirmed that the vast majority of lawyers can rapidly become proficient in new technology when they are incentivized to do so. However, without the right processes and pricing to capture some of the value that AI brings to legal services, it’s hard to make a compelling business case for it. Lawyers are generally very busy and like any other business, are often reluctant to introduce products that may take away from their own revenue streams.  Although there are significant differences between due diligence and eDiscovery, continually improving technology makes the threat of disruption increasingly probable. eDiscovery companies used essentially the same junior lawyers and technology that law firms had access to and turned it into highly profitable businesses. They effectively “took” this revenue from law firms.¹⁶ Law firms still have the upper hand with due diligence. A 2020 survey found that 89.7% of law departments “would appreciate law firms approaching us more frequently with new legal service delivery models.”¹⁷ So for those who can figure it out, there is a receptive audience.  While it may seem counterintuitive to a law firm to adopt a product that appears to eat into its revenue stream, the firms that figure out how to embrace and use AI most effectively are the ones that will likely come out the furthest ahead.

Key takeaways

• We are still in the very early stages of applying AI to due diligence. The right balance between the complementary strengths of AI and lawyers, and between efficiency and attention to detail, is still undefined.
• To work well, AI needs lots of labelled data. Acquiring and labelling contracts can be especially difficult in the confidential, specialized world of M&A. This difficulty explains some of the expensiveness of these products and also some of the risks involved in AI.
• The biggest (and safest) opportunities for using AI come from developing processes that balance the complementary strengths of both lawyers and AI.
• Getting AI to fit into legal services is as much of a business challenge as it is a legal process challenge. While it may seem counterintuitive to adopt a product that appears to eat into its revenue stream, the firms that figure out how to embrace and use AI most effectively will likely be the ones that come out ahead.

Since our inception, Dentons has always been a firm on the cutting edge of disruption and innovation in the legal profession. A firm that will always be where the profession is headed, not where it has been. As the applications of AI to legal services continues to grow, we would welcome the opportunity to discuss any points touched on in this article, and how any of them might help your business.

[1] https://www.makeuseof.com/is-google-ai-lamda-sentient/

[2] https://www.artificiallawyer.com/2021/02/24/how-did-clm-become-the-hottest-segment-in-legal-tech/

[3] E.g. Zuva (splitting off from Kira), Google’s Contract DocAI, Microsoft’s OpenAI

[4] Especially when it comes to language, the difference between 100% and 90% accuracy can be striking. Consider this example, which shows that 80% comprehension of a language leaves huge comprehension gaps: https://www.sinosplice.com/life/archives/2016/08/25/what-80-comprehension-feels-like

[5] https://moritzlaw.osu.edu/sites/default/files/2022- 01/THE%20EDISCOVERY%20MEDICINE%20SHOW.
pdf

[6] https://www.gartner.com/en/newsroom/press-releases/2018-02-13-gartner-says-nearly-half-of-cios-are-planning-to-deploy-artificial-intelligence

[7] This limitation of making decisions based only on past results and not on an envisioned (possibly better) future is why using AI in the criminal justice system warrants rigorous scrutiny.

[8] Granted, this quote was published in 2018, but the core problem of finding and labelling contracts still remains. https://kirasystems.com/files/science/fp252.pdf

[9] It is possible to build a dataset out of made-up examples, known as “synthetic data”. However, this comes with additional risks and challenges.

[10] Contrast this with a more discrete task, such as reading handwritten numbers, for which plenty of large, pre-labelled datasets already exist.

[11] This is known as “natural language processing” (NLP). For scanned contracts, there is often an additional preliminary phase of turning an image into text, known as “optical character recognition” (OCR).

[12] https://www.artificiallawyer.com/2021/12/15/a-guide-to-evaluating-contract-analysis-ai-solutions/

[13] https://scholarship.richmond.edu/cgi/viewcontent.cgi?article=1344&context=jolt

[14] https://www.jdsupra.com/legalnews/judges-make-the-case-for-tar-8429510/

[15] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3066816

[16] For more on how this happens, cf. The Innovator’s Solution

[17] https://f.hubspotusercontent30.net/hubfs/470182/13th_Annual_Law_Department_Operations_
Survey_2020.pdf