A California appellate court published an opinion in 2025 for the express purpose of warning the bar, after a Los Angeles-area attorney was fined $10,000 over an appellate brief riddled with quotations that no court had ever written. The court did not treat it as an embarrassing slip. It treated it as a filing that wasted judicial resources and misled the tribunal.
That is the shape of the problem. Generative AI produces text that reads exactly like competent legal writing, including citations formatted flawlessly to cases that do not exist. The defect is invisible at the level of style and only appears when someone pulls the reporter. Courts have now decided that not pulling the reporter is itself the misconduct.
For anyone drafting a filing, the practical questions are narrow: which tasks can a model do safely, what has to be verified by a human before it goes out, and what disclosure the specific judge requires. Free tools now exist that will draft a motion in seconds. A litigant using a free AI drafting service is still the person who signs the document, and the signature is what carries the sanctions exposure. For anything with real stakes, particularly federal court representation in California, the model is a research assistant with no accountability, not a substitute for counsel.
How Fabricated Citations Actually Happen
Large language models predict plausible next tokens. They do not consult a database of decisions unless they are explicitly connected to one, and even retrieval-connected tools can misattribute a holding to the wrong case or quote language that appears nowhere in the opinion. The failure mode is not random noise; it is fluent, confident, correctly formatted invention.
The pattern in sanctioned filings is consistent. A real case name gets paired with a fabricated reporter citation. A real citation gets attached to a proposition the case does not support. A quotation is synthesized from the general sense of a line of authority rather than lifted from any single opinion. Parentheticals describing holdings are the most frequently invented element, because they are the part a checker is least likely to verify against the full text.
Federal sanctions in this area began in 2023, when a New York district judge fined lawyers who submitted a brief containing several non-existent decisions and then, when challenged, produced fabricated excerpts of those decisions rather than withdrawing them. The court emphasized that using the tool was not the violation. Failing to check the output, and then doubling down when opposing counsel flagged it, was.
The Rules That Already Apply
There is no need for a new AI statute to sanction a bad filing, and that is the point most commentary misses. Existing authority reaches the conduct directly.
| Authority | What it requires | Exposure |
|---|---|---|
| Federal Rule of Civil Procedure 11 | That legal contentions be warranted by existing law after an inquiry reasonable under the circumstances | Monetary sanctions, fee shifting, striking the filing |
| 28 U.S.C. section 1927 | That counsel not multiply proceedings unreasonably and vexatiously | Personal liability for excess costs and fees |
| Court inherent authority | Candor toward the tribunal | Sanctions, referral to disciplinary authorities |
| State rules of professional conduct | Competence, candor, confidentiality, supervision of nonlawyer assistance | Bar discipline independent of court sanctions |
| Judge standing orders | Disclosure or certification of generative AI use in filings | Striking of filings, contempt for noncompliance |
The American Bar Association addressed generative AI in a formal ethics opinion in 2024, tying it to established duties rather than inventing new ones: competence requires a reasonable understanding of the tool limits, confidentiality restricts what client information can be entered into a system that may retain or train on it, communication may require telling the client that AI is being used, and fees may not bill a client for hours the tool actually saved.
Disclosure Requirements Vary Judge by Judge
Beginning in 2023, individual federal judges started issuing standing orders on generative AI, and the requirements are not uniform. Some require a certification that either no generative AI was used or that every AI-generated passage was verified by a human. Some require disclosure of the specific tool. Some are silent, which does not mean permissive. In California, the Judicial Council moved in 2025 to require courts to put generative AI use policies in place, pushing the question from individual preference toward institutional rule.
The operational consequence is unglamorous: before filing anything in an unfamiliar court, read that judge standing orders and the local rules. This is a five minute task that has become a malpractice trap for people who skip it.
Where AI Is Genuinely Useful in Litigation
The sanctions stories obscure the fact that machine assistance is already ordinary in places where verification is cheap or the output is not asserted as fact to a court.
- Document review and privilege screening in discovery, where technology assisted review has been judicially accepted for over a decade and is measured against a validated recall standard
- Deposition and hearing transcript summarization, where the source text is right there to check
- Chronology building from medical records or financial statements
- First drafts of routine correspondence, discovery requests, and scheduling stipulations
- Translating a dense contract clause into plain language for a client conversation
- Adversarial rehearsal, where the model argues the other side position so counsel can find the weak points
What these have in common is that a human can verify the output against a source in less time than producing it from scratch would take. That ratio is the actual test of whether an AI use is safe.
Where the Ratio Breaks Down
Legal research on an unsettled question, where verification means reading everything anyway. Fact assertions about the record. Anything involving client confidences typed into a consumer tool with no confidentiality agreement. And risk prediction about individual people, which is a different problem entirely.
Algorithms That Decide Things About People
Drafting tools are the visible controversy. The consequential one is machine scoring used in bail, sentencing, and parole. The Wisconsin Supreme Court addressed this in 2016 in State v. Loomis, holding that a proprietary risk assessment score could be considered at sentencing but not be the determinative factor, and that its use required a written warning to the sentencing court about its limitations, including that the underlying methodology was a trade secret the defendant could not examine.
That compromise has held uneasily. A defendant cannot cross-examine a model. Validation studies for these instruments are usually done on populations different from the one being scored, and the training data reflects historical enforcement patterns rather than underlying conduct. Meanwhile facial recognition matches have contributed to wrongful arrests in several documented cases, typically where an investigator treated a candidate match as an identification rather than as a lead.
Evidence Is the Next Fight
Authentication under the evidence rules assumes a human can testify that a recording is what it purports to be. Synthetic audio and video break that assumption in two directions. Fabricated evidence can be offered as genuine, and genuine evidence can be attacked as fabricated, a dynamic sometimes called the liar dividend, in which the mere existence of the technology gives every party a ready objection.
Federal rulemakers have been working on machine-generated evidence, considering whether output produced by an algorithm without a human witness should face a reliability screening similar to the one applied to expert testimony. Until something is settled, the practical burden falls on counsel to preserve original files with metadata intact, document chain of custody carefully, and retain a forensic examiner early when authenticity is genuinely contested.
Frequently Asked Questions
Is it against the rules for a lawyer to use ChatGPT?
No, not as a general matter. Ethics authorities have treated generative AI as a tool subject to existing duties rather than a prohibited practice. What is sanctionable is filing content the lawyer did not verify, entering confidential client information into a system without appropriate protections, billing for time the tool saved, or violating a specific judge disclosure order. Competent use is permitted; unverified use is not.
What happens if a self-represented litigant files an AI-generated brief with fake cases?
Courts generally hold self-represented parties to the same procedural rules as lawyers, though they often show more leniency on the sanction. Realistic outcomes include the filing being struck, the argument being disregarded, an order to show cause, monetary sanctions, and a serious credibility loss that affects how the rest of the case is received. The safest approach is to verify every citation yourself before filing.
How do I check whether a citation is real?
Look it up in a primary source rather than asking the model to confirm it, because a model will often confirm its own fabrication. Government court websites publish opinions directly, and free repositories carry most published decisions. Read the actual opinion for the proposition you are citing, not just the case name, since a real case attached to an invented holding is the more common and more dangerous error.
Can AI replace a lawyer for a small case?
For genuinely simple matters such as understanding a form or drafting a demand letter, a model can be a real help. It cannot assess whether a claim is viable, spot a jurisdictional defect, judge settlement value, evaluate a judge tendencies, or take responsibility when it is wrong. The gap widens sharply once there is an opposing lawyer, a deadline that bars the claim, or a criminal exposure.
Do courts use AI themselves?
Increasingly, yes, for administrative functions such as translation, transcription, scheduling, and document triage. Several court systems have adopted policies allowing staff use for support tasks while restricting anything that touches judicial decision making. The consistent principle in these policies is that a human judicial officer remains accountable for the decision and that AI output is never the basis of a ruling on its own.
What to Do Before You File
Treat every citation produced by a model as unverified until you have opened the opinion and read the passage you are relying on. That one habit prevents essentially all of the sanctions cases reported so far. Then check the standing orders of the specific judge for a disclosure or certification requirement, and keep client confidences out of any tool whose terms permit retention or training on your inputs.
This article is general information, not legal advice. Rules on artificial intelligence in court filings differ by jurisdiction and by judge, and you should consult a licensed attorney about your specific situation.
