AI for document review in legal teams: first pass, privilege and the defensibility record

A disclosure exercise starts with a collection nobody has read. Custodian mailboxes holding years of threads and attachments, a shared drive with eleven near identical versions of the same board pack, a chat export where the substance sits in a two word reply, scanned bundles from a filing cabinet, a handset extraction where the exchange you need runs across a group thread and a voice note. De-duplication and threading remove a slice. What remains has to be read by somebody who can say, under a signed statement, what it is and why it was given or withheld.

That is the work. Not the legal analysis at the end, but the reading that produces it: the part that gets outsourced, eats the budget on a large matter, and stays quiet about its errors until they are catastrophic. AI belongs in that layer, with a record of what it did.

What actually lands in the review platform

Be honest about the corpus, because the corpus is what breaks systems that demo beautifully.

Email archives arrive as families: a parent message with attachments, where the parent is anodyne and the attached spreadsheet is the whole case. Coding has to respect family relationships, because a responsive parent with a non responsive attachment is a production error waiting to be raised.

Chat exports arrive as continuous streams that have to be cut into reviewable units. A day of a channel is not a document. The unit you choose, a fixed window, a conversation break, a participant change, decides what reviewer and model both see. Get it wrong and every later decision is made on fragments.

Scanned bundles arrive as images. Optical character recognition on a clean laser printed page is solved. On a fax of a fax, a stamped exhibit, or the handwritten margin note that says the thing you are looking for, it is not. Those need human eyes, not a model that will confidently read noise.

Mobile threads arrive with reactions carrying meaning, gaps where messages expired, and attachments referenced but uncollected. Foreign language documents arrive inside mixed language threads, where a German email quotes an English clause and replies about a third language attachment.

An agent that works here handles all of that, or says what it cannot.

First pass review: relevance and responsiveness coding

This is the highest volume decision in the process and the one most amenable to automation. The agent reads the document, its family and its thread history, then writes back into the review platform a responsiveness call against the issues list, a confidence score, and a short reason citing the passages it relied on.

The reason matters more than the call. A decision with no reasoning is a number a senior reviewer has to verify independently, which saves nothing. A decision that says this is responsive to issue three because the sender confirms the pricing change took effect in March, with the sentence quoted, is checked in seconds. The reviewer is no longer reviewing the document, they are reviewing the argument about it.

Issue coding follows the same shape. Most matters run an issues list that gets revised as the picture changes, and each revision means a human re-review. An agent can re-run the affected population against the new definitions overnight and produce a delta report of what changed and why, work that rarely gets done properly under time pressure.

Hot document identification

Separate from responsiveness is what matters. Teams want the documents that change the case early, not at the end of a linear review. An agent reading for narrative rather than classification can surface candidates: the message where somebody voices concern about a decision, the version where a paragraph was removed, the exchange where a person is asked to delete something. These are recommendations for a lawyer to read, never findings.

Privilege is the hardest call in the set

Everything else here can be described with rules. Privilege cannot, and this is where automated review earns trust or a very bad afternoon.

The difficulty is structural. Privilege attaches to a relationship and a purpose, not to a document type. An email from a lawyer about restaurant bookings is not privileged. An email between two business people forwarding legal advice may be. The same advice, forwarded outside the circle, may have lost protection. In house lawyers give commercial and legal advice in one paragraph, and the privileged portion may be a single sentence. Litigation privilege turns on dominant purpose and on when proceedings were reasonably contemplated, a factual question about a date nobody wrote down.

What an agent can do reliably is the evidence gathering underneath the call. It can hold a resolved list of legal personnel across name variants and address changes, and flag an unrecognised participant on a thread that was otherwise internal legal. It can spot a privileged chain later forwarded to a third party and mark the potential waiver with the message that caused it. It can tell a lawyer who is the author from a lawyer merely on copy, a signal keyword lists handle badly. It can draft the log entry: parties, date, the nature of the advice described without disclosing it, the basis claimed.

What it cannot do is make the call unsupervised. Run privilege at a deliberately low automation threshold, with borderline documents escalated, not coded. The failure mode is asymmetric: an over inclusive claim gets challenged and costs you time, while an inadvertent production of privileged material is a different category of problem.

Redaction before production

Redaction is where automation is both most useful and most dangerous, because a redaction failure is not a coding error a second pass catches. It is material that has left your control.

The useful part: finding personal data across a production set is pattern work at scale. Names, addresses, identifiers, bank details, health references, data about people who are not parties at all. An agent can propose redactions with coordinates, apply the same treatment to every copy of a document appearing in multiple families, and produce a log of what was redacted and on what ground, which is the artefact a regulator or opposing party will ask about.

The dangerous parts, which any serious implementation addresses explicitly: personal data in the text layer of a scan that the burn in misses; spreadsheet cells that are hidden, filtered or pulled into a formula shown elsewhere; metadata carrying the author name after the visible text was cleaned; tracked changes and comments in native files; redactions applied as a removable overlay. Verify every production on the delivered file, extracting the text and confirming the redacted content is absent. That check is cheap and it is the one that prevents the incident.

Diligence, second requests and investigations

The same machinery serves work that is not litigation. In transactional diligence, a data room is a document review with a different question. The reviewer wants change of control provisions, exclusivity, assignment restrictions, unusual indemnities, the terms on which key people are employed, and anything contradicting the disclosure schedule. An agent can read the room in full rather than the sample the timetable allows, index what exists against what was promised, and flag where a referenced schedule or amendment is absent. Missing documents are often the finding.

In regulatory production and second requests, the specification is prescriptive and the deadline immovable. Volume is the whole problem, and an automated first pass with human review concentrated on the responsive and borderline strata is how these land on time.

In investigations, the set is smaller but the reading is deeper. What is wanted is a chronology with citations: who knew what, when, and from which document. Building it as the review proceeds rather than afterwards is one of the clearer wins.

The quality control layer that makes it defensible

None of this is usable without a measurement layer, and teams skip it.

Run a control set. Have senior reviewers code a statistically drawn sample independently, before they see the model output, and compare. Track the overturn rate, the proportion of automated calls a human changes on review, broken down by issue, custodian and document type, because an aggregate figure hides the category where the system is failing badly.

Sample the null set, the population coded not responsive and therefore never read. That sample is the evidence for the claim that nothing important was missed, and it is what a court or a regulator will ask about.

Keep the record. For every document: which model version coded it, when, against which version of the issues list, with what confidence and stated reasoning, and who reviewed it. A team that can produce that months later is in a different position from one that can only say a tool was used.

Validate before relying, not after. Measure agreement on an initial population, adjust thresholds and definitions, and only then move review capacity. That period costs more, not less.

Where AI fails here

Honest limitations, from systems in production.

Privilege remains a judgement call. Anything near a dominant purpose question, a mixed commercial and legal communication, or a possible waiver has to be escalated. Treating it as solved is the most common way these projects go wrong.

Poor quality scans produce confident nonsense. Handwriting, marginalia, stamps, degraded faxes and photographs of screens defeat text extraction. The correct behaviour is to route them out with the reason stated, and a system that does not is silently dropping documents.

Sarcasm, code words and deliberate obfuscation are invisible. People who intend to conceal do not use the vocabulary you are searching for. They use a nickname, an internal shorthand, or a reference to a conversation held elsewhere. Reviewers who know the matter catch these. Models mostly do not.

Thin context defeats everything. A one line reply saying approved, sent from a phone with no quoted text, cannot be coded by reading it alone.

Foreign and mixed language documents degrade unevenly. Quality varies by language pair and by subject matter, and legal terminology with no direct equivalent across systems is a recurring source of error. Native speaker review remains necessary.

And the process itself has to be disclosed and sometimes negotiated. Opposing parties and regulators have views on automated review protocols. Design yours expecting to describe it in a witness statement, because that is how it will be examined.

Where to start

Start with the first pass on a live matter, running alongside human review rather than replacing it, so you have an agreement rate you trust before anything depends on it. Keep privilege conservative, with escalation as the default, and instrument the quality control from day one, because a defensibility record cannot be retrofitted.