Human review of agreements hasn’t improved these past two decades, Spellbook said
A new analysis by Spellbook Labs found that – of 60,000 pages from more than 3,000 public company agreements that 520 organizations filed with the Securities and Exchange Commission (SEC) over two decades – 60 percent of contracts contained drafting issues.
While many errors were minor, Spellbook described 2.5 percent as high-risk, such that they altered the agreement’s meaning in a significantly negative way for at least one party.
Spellbook found these high-risk errors mostly in provisions determining who would receive payment, who would be liable, and what would occur in a dispute.
“In legal work, AI hallucinations also get the headlines,” Spellbook stated. “An AI-generated legal brief citing non-existent case studies. A judge who used AI to create a restraining order and misquoted state law. These are real issues that deserve attention.”
However, Spellbook put a spotlight on the errors that humans made when manually drafting and reviewing contracts, considering the limited time and the voluminous and complicated documents involved.
“None of this is a matter of carelessness by lawyers so much as a consequence of the length, complexity, and disjointed nature of how these types of SEC-filed agreements are prepared and negotiated,” Spellbook said.
Spellbook found that the data supported that utilizing AI could save time, improve accuracy, and increase output.
Spellbook explained that organizations could use AI tools to analyze contracts to find mistakes, identify risks, and recommend changes based on the specific agreement’s context and current market data.
Key findings
Spellbook shared some insights it gleaned from the research.
- Some of the most valuable companies around the globe had contracts with the most drafting errors
- In terms of organizations’ average number of issues per contract, the lowest was 0.18, and the highest was three
- Larger companies averaged 0.85 errors per contract, compared with 1.31 for everyone else
- Age was a notable predictor of contract quality, as older and more established organizations generally had agreements with fewer drafting issues
- Younger companies tended to have more errors, and their mistakes were disproportionately more serious
- Among industries, media, telecommunications, and travel had the worst showings in terms of high-risk issue rate
- Healthcare and pharma had the lowest high-risk issue rate
- Note purchase agreements had a 15 percent high-risk rate, while stock and asset agreements had a 12 percent rate, compared with the average three percent rate for all contract types
“Better review will not come from more organization, more care, or more lawyers,” Spellbook said. “It will come from pairing legal judgment with tools that check every page with a consistency no human team can sustain.”
