Building AI Revise, LegalOn's Generative AI Contract Editor
Growing up, my grandfather often quipped, “Hurry up, don’t rush.” His not-so-subtle nudge whenever he wanted his spirited grandchildren to get in the car or finish getting ready for bed.
As a 7-year-old, all I heard was a directive to stop dillydallying. I know now, though, that his real intention was more nuanced.
Sure, he was pushing us to move fast, to stop wasting time (hurry up). But he also didn’t want us to injure ourselves or forget something in the process (don’t rush).
Today, as I watch the market race to develop or adopt solutions powered by the latest GenAI technologies, I’m reminded of his expression.
Legal tech companies are making haste to offer their customers the “next best thing.” And for good reason! In the contract review space alone, GenAI can (and will) play a role in:
- Drafting contracts, clauses, and provisions
- Generating non-contract content
- Auto-populating terms
- Summarizing contracts, obligations, and risks
- Creating contract analytics and reports
- And so much more!
But, as any lawyer or legal professional will tell you, there’s a difference between doing something and doing something right. Especially when you’re talking about contracts.
___
This blogpost explores LegalOn’s recent journey to build our newest GenAI solution: AI Revise.
Keep reading to hear our own learnings, as we embraced the “hurry up, don’t rush” mindset to bring you one of the most valuable GenAI-powered contract review tools on the market.
AI Revise, LegalOn’s New GenAI Tool, Is Now Available to Users
Earlier this year, LegalOn unveiled AI Revise – the first GenAI contract editor enhanced by expert legal knowledge. Over the last two months, we demoed AI Revise at CLOC Global Institute in Las Vegas and collected feedback from beta testers.
Our team is encouraged by the overwhelmingly positive reception from users. So we’re thrilled to announce that AI Revise is now available for LegalOn users!
So, then, what exactly is AI Revise?
AI Revise allows users to directly insert LegalOn suggestions into their contracts with just one click, mitigating material contract risk in seconds. Users tell us they love that AI Revise:
- Consistently produces professional-grade edits
- Is deeply informed by expert legal guidance written by lawyers
- Works right off the shelf – no playbook or costly implementation required
- Harmonizes with the specific contract, party position, and context at hand
- Leaves the contract reviewer in the driver seat
Risky Pitfalls of GenAI Tools in Legal Tech
Admittedly, it’s easier now for us to look at AI Revise with rose-tinted glasses.
But was it easy to get there? Definitely not.
Early in the discovery process, our R&D team realized that a lot could go wrong. Without question, GenAI has pitfalls. Those pitfalls are particularly worrisome for Legal and Legal-adjacent functions, which regularly deal with high-stakes issues. For example, LegalOn is a global leader in the pre-signature contract review space, where mistakes expose clients to material legal, financial, and operational risks.
We at LegalOn are passionate about helping our community to cut through the noise and stay informed on the rapidly evolving topic of GenAI. So I’ve compiled this (nonexhaustive) list of key pitfalls to avoid:
Despite the ironic name, AI “hallucinations” – and the dangers they pose – are quite real.
In layman's terms, an AI hallucination is when a large language model (LLM) generates false information.
In a recent research paper, OpenAI warned, “Counterintuitively, hallucinations can become more dangerous as models become more truthful, as users build trust in the model when it provides truthful information in areas where they have some familiarity.”
On June 22, a judge imposed sanctions on two NY lawyers who included six fictitious case citations generated by ChatGPT in a legal brief. Those lawyers contend they made a “good faith mistake” in falsely believing that ChatGPT was a “super search engine.” The judge, however, found the lawyers committed “acts of conscious avoidance and false and misleading statements to the court.”
For contract reviewers, AI hallucinations could threaten the strength, integrity, or legitimacy of a contract. When identified by a contract reviewer, hallucinations are frustrating inaccuracies that require time to rectify. When overlooked, hallucinations inject contracts with unforeseen risks that could jeopardize a client’s interest and a legal professional’s reputation.
Without proper protections in place, GenAI poses data privacy, cybersecurity, and compliance risks.
Contracts are inherently jam-packed with sensitive data. Naturally then, legal tech companies focused on contracts are handling massive amounts of highly sensitive data on a daily basis. These companies have an obligation to safely and securely manage and protect data.
But upholding that obligation becomes more difficult with GenAI.
LayerX – a browser security company – published a report entitled Revealing GenAI's True Data Exposure Risk. After examining ~10K employees, the report found that 6% have sent sensitive data to ChatGPT and other GenAI apps (with 4% doing so on a weekly basis). Types of sensitive data include trade secrets, source code, internal business data, personal identifiable information, and customer data.
Moreover, legal professionals worry that sharing data via GenAI tools will undercut their competitive advantage or compromise their own responsibilities to clients.
- One lawyer said, “I’ve done this for 25+ years. I know more about this contract type than any of my peers. How can I be sure that my contract data won’t be used to train a model that could help my competitors to be better at reviewing this niche contract type?”
- Another said, “I’ve made certain [data protection] commitments to my own clients. I need to know that your GenAI tool protects my clients’ data as well as I do.”
Generic prompts generate low-quality output; overly specific prompts aren’t scalable.
If your LinkedIn feed looks like mine, it’s flooded with articles with titles like “5 Hacks to Prompt Engineering” or “You’re Doing Prompting Wrong.”
The truth is that generating a mediocre output from an LLM isn’t too hard. But mediocre outputs aren’t sufficient for legal professionals performing contract review. Legal professionals will (and should) embrace GenAI tools that consistently produce top-notch content that is (ideally) indistinguishable from language written by an experienced and knowledgeable attorney.
And this is where many contract-focused legal tech players are struggling to deliver today.
Why? One likely culprit is that it’s really tough to strike the right balance between genericness and specificity in prompts.
- Generic prompts are low-cost and easily reproducible. However, the LLM output quality will generally be mehhh, and the output definitely won’t be tailored to the individual contract at hand. ChatGPT can give you a generic limitation of liability clauses – but 9 times out of 10, it won’t satisfy the context-specific need of a contract reviewer.
- Hyper-specific prompts are more likely to get you what you want, but they’re not scalable. The first draft of a contract may have dozens of material risks. Individuals and companies alike don’t have the resources or time to intricately engineer a unique prompt to mitigate each risk. At that point, it’s faster to do manually.
LegalOn’s Approach to Avoiding GenAI Pitfalls
The AI Revise development process involved seemingly endless trial-and-error experimentation. Along the way, we learned a lot about how to successfully sidestep these pitfalls.
I can share some best practices:
- Test extensively with lawyer-led validations: Conducting thousands and thousands of tests before releasing a GenAI tool to users is table stakes. But when you’re dealing with legal content generation, your validators must be qualified legal professionals who are intimately familiar with contract writing, legal nuances, and legalese.
- Walk LLMs on a tight leash; don’t let LLMs walk you: LegalOn has spent 6+ years developing large databases of rich legal content authored by in-house lawyers. Because AI Revise is built on top of our proprietary AI Review technology, we can weave relevant portions of that legal content into our prompts. This allows us to coach the LLM to consistently produce professional-grade outputs. Moreover, we’ve fine-tuned our system settings through large-scale experimentation to control for LLM variability and creativity.
- Protect your users’ property: LegalOn’s AI Revise uses Microsoft’s Azure OpenAI Service. Neither Microsoft nor OpenAI use any contract data to train, retrain, or improve their models. LegalOn also secured additional data privacy protections from Microsoft, whereby no data is stored by Microsoft for any period of time.
Hurry Up, Don’t Rush
For the foreseeable future, many legaltech players will continue to sprint to leverage the rapid advances in GenAI. And that’s exciting!
However, while speed-to-market is always tempting, we – the companies that develop technologies for law firms, in-house legal teams, LegalOps, ALSPs, and more – carry a collective responsibility to avoid these pitfalls for our customers.
Don’t rush, if you will.
___
Curious to see AI Revise in action? Sign up for a demo today to connect with our team and experience the best AI contract review software available today.
