Project Overview
A US small business owner was sending every vendor contract to an external attorney for full review because neither they nor their internal team had the knowledge to identify which clauses were problematic and which were standard. The result was substantial legal costs on routine vendor agreements and turnaround times long enough to slow down vendor relationships. We fine-tuned a foundation model on standard commercial contract language and the attorney’s own historical review notes to automatically flag common risk clauses before the contract reached the attorney, reducing the volume of billable review time required on each agreement.
The Challenge
The core problem was a knowledge gap, not a volume problem. The business owner did not lack time to review contracts: they lacked the legal literacy to distinguish a standard liability clause from a problematic one. Every contract felt equally opaque, so every contract went to the attorney in full. The attorney’s time was billable, the review cycle added days to every vendor negotiation, and the internal team remained dependent on outside counsel for work that did not require the attorney’s full expertise.
- Full attorney review was being applied to every vendor contract regardless of complexity or risk level
- Legal costs were significant relative to the size and revenue of the business
- Vendor agreement turnaround was slower than competitors who had internal legal capability
- Contracts arrived in varying formats and structures, making any manual triage process difficult to standardize
- The internal team had no exposure to risk clause language, so they could not develop contract literacy through experience
Our Solution
We fine-tuned a foundation model on two primary data sources: a curated library of standard commercial contract language covering the business’s most common vendor agreement types, and the attorney’s past review notes and flagged clause annotations from previous engagements. The resulting model could identify common risk clauses, explain why each clause warranted attention, and surface a prioritized review summary for the attorney before the full contract was sent for billable review.
- Training Data Curation: We worked with the attorney to compile annotated examples of flagged clauses from past reviews, covering liability limitations, indemnification language, termination conditions, and payment terms, categorized by risk level and clause type.
- Risk Flagging Layer: The model was fine-tuned to identify clause-level risk patterns and output a structured summary of flagged sections with plain-language explanations of the concern each clause raised.
- Uncertainty Signaling: The model was calibrated to flag its own uncertainty clearly rather than silently pass over clauses it could not confidently assess. Silent misses were the highest-risk failure mode for this use case, and we prioritized conservative flagging behavior over recall precision.
- Attorney Review Queue: Flagged contracts were passed to the attorney with the model’s summary prepended, enabling the attorney to focus review time on the identified sections rather than reading the full document from scratch.
Technical Approach
The model was fine-tuned on a structured instruction dataset built from the attorney’s annotated clause library, with each example consisting of a contract excerpt, the clause type, the risk annotation, and a plain-language explanation. Contracts were processed through a document parsing pipeline that handled variable formats including PDF, Word, and email-attached plain text. Clause segmentation was handled programmatically before model inference, with each identified clause segment evaluated independently and results aggregated into the final summary output.
Results & Impact
- Attorney review time and cost on vendor contracts reduced significantly, as the attorney could focus on flagged sections rather than conducting a full document read on each agreement
- Turnaround time on vendor agreements shortened, improving the business’s speed in finalizing supplier relationships
- The internal team became more contract-literate through regular exposure to the model’s flagged clause explanations, reducing their dependence on outside counsel for routine pattern recognition
- Flagged clause accuracy was high across the contract types covered by the training data, with the model reliably surfacing the clause categories the attorney had identified as most commonly problematic
- The monthly volume of contracts reviewed increased as the process became faster and less costly per agreement
Lessons Learned
The most important design decision in this project was the uncertainty flagging behavior. In a legal context, a model that silently misses a problematic clause creates liability. We chose a conservative flagging posture: the model surfaced uncertain cases for human review rather than making a low-confidence negative judgment. This meant the attorney occasionally reviewed a section that turned out to be standard, but it preserved the guarantee that nothing genuinely risky was passed through without human eyes on it. That conservative posture was the right tradeoff for this client and this use case, even if it meant the model’s precision was not maximized. Contract format variability was the other significant challenge: the parsing pipeline required more iteration than anticipated to handle the range of document structures the business received from different vendors.