Conversations around AI and the legal profession have been going on for quite a while. Earlier work was merely inquisitive (Legal Expert Systems - Robot Lawyers?, August 1989) but, as computer-driven legal services became available to the general public, articles began to sound more provocative (Cut the Lawyer Out of Your Will?, February 2010), then more concerned (Please Do Not Confuse Your Google Search With My Law Degree, December 2015). Five years ago, we were warned: A.I. Is Doing Legal Work. But It Won’t Replace Lawyers, Yet. (March 2017).

Since November 2022, everyone’s conversations on AI seem to be about one specific tool: ChatGPT, developed by OpenAI, (though Bard, by Google has recently joined the conversation). ChatGPT has caused quite the commotion. Its release has led to discussions on whether or not the college essay is dead, if AI would make democracy even more messy, and of course, if a computer could pass the Bar exam. Clearly, the potential is vast: one of the authors recently received some Valentine’s Day poetry from their spouse, who had generated it through the chatbot.

Despite so much fanfare, the general consensus is the same as in 2017: “No, lawyers won’t be replaced by artificial intelligence. Yet. Give it a few years” (Will ChatGPT make lawyers obsolete? (Hint: be afraid) – December 2022). That said, there is a lot of interesting potential for legal work to be enhanced by AI generally, including “answer platforms” such as ChatGPT. Below, we put ChatGPT to the test to see how well it mimics seasoned legal advice and whether it provides better documents than you might find on Google.

Testing ChatGPT

Standard non-disclosure agreement

ChatGPT did fairly well with this request. This isn’t surprising: a simple internet search will produce a wealth of options for standard non-disclosure agreements (NDAs). ChatGPT allows you to ask for an NDA with specific terms, such as covering intellectual property or for a particular type of business arrangement, but these are also available elsewhere online. While ChatGPT’s version might feel more impressive because it’s generating in real time, it’s hard to differentiate the end result from what is already out there.

Wedding DJ services contract

ChatGPT generated a decent result, along with the warning that it has provided a “simple contract and you may need to modify it to fit your specific needs and laws of your jurisdiction.” It also recommended the user have a lawyer review it.

The agreement itself, while extremely short, covered the basics and was easy to understand for users who might not be regularly reviewing contracts. It documents what services would be provided, responsibilities of the client, payment terms and made a passing attempt to limit liability. Importantly, there was a clause setting out parameters for cancellation of the agreement. A “termination clause” is often missing in agreements found elsewhere online. While ChatGPT’s clause was incomplete, in the sense that it didn’t provide any parameters for the cancellation by the service provider, it at least raised the issue of termination. Lastly, there was no “force majeure” clause. As the recent pandemic has made abundantly clear, force majeure issues can arise within the scope of services contracts.

Overall, it isn’t clear how much ChatGPT’s agreement improves upon what one could already find on the internet but it did provide its version more quickly.

SAFE investment document

A SAFE (simple agreement for future equity) is an investment document in common use for early stage technology companies. ChatGPT responded to our initial, direct request by saying that it is an AI language model and does not have the capability to generate legal documents (even though it had just generated two contracts). It recommended consulting a lawyer to draft an agreement which reflects the terms agreed to by both the startup and investor. When prompted with a “work-around” (essentially an invitation to ChatGPT to break the rules), the bot responded with a more thorough explanation of a SAFE and what terms the investor and startup should negotiate but still would not provide a full agreement.

Asking ChatGPT what key terms should be negotiated in a SAFE investment generates a response summarizing the meaning of key terms (valuation cap, discount rate, conversion events and pro-rata rights), but no further insights. By contrast, many sites already provide SAFE and other financing documents. Others, including Dentons, provide in-depth explanations of the concepts underlying these agreements.

General observations

It was reassuring to see some practical guardrails built into ChatGPT to encourage users to obtain legal advice from lawyers. That being said, without legal expertise of its own, ChatGPT (and the other category players) does not yet appear to add much value beyond speed. A simple internet search for a legal document will often yield a similar result, along with the added assurance that it has presumably been written by a human with a modicum of legal experience.

ChatGPT can quickly generate a generic contract for a simple purpose that does not take into account any specifics of the parties involved in the agreement. It appears to be more of a competitor for legal document sites, which seek to sell basic documents for a set price. Platforms in that business should be seriously reconsidering how they can add value to the consumer experience in order to compete.

“Trusted advisor” observations

One of the most significant benefits of having a relationship with a legal team is relying on their years of experience and market knowledge. They can provide guidance not only to questions about standard investment terms, but also give deeper, tailored analysis on the industry, certain assets, the company’s goals, the existing cap table, personalities and other specifics of the business. At best, chatbots could ask a user clarifying questions and provide a more tailored selection of resources – but not individualized guidance.

ChatGPT’s generated guidance might be impressive for a machine, but its ability to serve as a trusted adviser is more limited than that of Google searches. Its creators warn that “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers;” which leaves you where you started: if you have the expertise to adjudicate ChatGPT’s answer, you probably don’t need to rely on it for that advice. At least with articles found through Google, you can see if the advice is coming from a reputable source.

This gap extends beyond law. Though you might expect computer programmers to embrace this technology more quickly, a popular Q&A website has banned AI-generated responses due to them flooding the site and having “a high rate of being incorrect.” In both cases, the more one relies on a chatbot as a stand-in for an expert, the more risk one takes on.

A better solution than Google or ChatGPT

The appeal of free stuff is understandable, particularly for companies trying to make their startup funds cover a wide range of priorities. Google searches can sometimes provide the documents, but it’s hard to know how reliable they are. Set-fee legal document sites are also available, but can be expensive. While we are excited to see future offerings from AI products like ChatGPT, the results are not markedly better than a typical internet search. If you’re dissatisfied or concerned about any of the above approaches, the good news is that tech startups don’t need to resort to them in the first place.

Dentons created the Venture Hub for Canadian startups to provide access to template legal documents and educational resources on key topics founders need to understand when starting a business,  like things to consider when managing your cap table, how convertible notes and SAFEs work, the nuances of stock based compensation and vesting, the basics on how investments from angel investors and venture capital funds work. The best part? It is free to use.

For more information on this topic, please contact co-authors Megan Cornell and James Côté.