For lawyers at Sullivan & Cromwell, reviewing a draft agreement includes weighing its terms against relevant precedents, using approaches based on the Firm’s experience and being mindful of a client’s priorities. The firm’s Agreement Analyzer brings those inputs into a single review workflow, producing analysis and proposed changes that lawyers can inspect and refine.
“I think AI will . . . allow lawyers to focus on the areas where their judgment is most important . . . so they can concentrate on the questions that are most critical for their clients.”
—John Savva, Partner, Sullivan & Cromwell
Bringing firm expertise into agreement review
Though AI was initially used most for repetitive, administrative tasks like drafting email responses or checking calendars, Savva finds that recent improvements in AI models’ analytical capabilities have changed its usage. “Now, we’re seeing many more opportunities for use in challenging legal tasks such as deep analysis of complex agreements,” he says.
Agreement Analyzer helps lawyers compare corresponding provisions across precedents and track how those provisions interact.
Sullivan & Cromwell partner John Savva developed Agreement Analyzer as a single-user proof of concept built with Codex. He then collaborated with OpenAI on improving the quality of outputs, enhancing efficiency through parallel processing and transforming the application into a tool that could be used by multiple lawyers at the same time.
Agreement Analyzer integrates Sullivan & Cromwell’s workflows, curated firm precedents, negotiating playbooks and ChatGPT Enterprise. After inputting a draft agreement into the tool, a lawyer can add their own selected precedents and a playbook with firm and client positions, as well as information on market practice. The tool performs multiple review cycles from different perspectives, synthesizes the outputs, helps identify potential issues and returns a memorandum and proposed tracked changes for a lawyer to review, adapt or reject.
“It’s not about replacing lawyers. It’s about providing a tool for lawyers to do deeper analyses and find more authority and precedents to provide an even higher quality product for their clients.”
—John Savva, Partner, Sullivan & Cromwell
Making sources and uncertainty visible
Savva emphasizes that what makes an AI tool’s output trustworthy is full transparency. “If an AI tool makes a claim about market practice, you need to be able to see the source of that information,” he says.
Savva describes an application in which agents develop conclusions and other agents challenge them in an adversarial process. In a potential use case, specialized agents review the agreement in parallel for such things as market deviations and risk allocation, interactions between clauses, and ambiguities that could impact litigation. The results distinguish findings supported after challenge by other agents, dismissed findings, and unverified issues retained with qualifications. Lawyers can inspect the issues, their supporting explanations, and the relevant precedents, then decide which findings warrant changes in the final markup.
What’s next
Sullivan & Cromwell is continuing to explore tools shaped around specific legal tasks. For Savva, stronger analytical capabilities create an opportunity to support more of the research and writing process.
Savva points to advances in reasoning capabilities with GPT‑6 Astra: “Astra’s really moved things forward in terms of the quality of the reasoning that’s available, and that has been the bottleneck in the past,” he says.
