When AI Gets the Law Wrong

People representing themselves in court have increasingly turned to artificial intelligence for legal help, but the technology is creating new problems for judges who must determine whether the information submitted is accurate. Courts have issued more decisions about AI-generated errors involving self-represented litigants than those involving lawyers. Judges and legal-aid leaders say the technology can help people navigate the courts, but courts still sanction people who file fabricated citations, even by mistake. As of Oct. 19, researcher Damien Charlotin’s database listed 2,202 decisions addressing AI-generated hallucinations worldwide, including 1,501 in the United States.

Of the 2,202 decisions in Charlotin’s database, 1,267 involved self-represented litigants and 872 involved lawyers. The database includes only decisions in which a court addressed the use of AI, “whether established or merely alleged.” Charlotin’s site describes it as “a work in progress,” not a count of every AI error litigants file. The figures illustrate how the growing use of AI is presenting courts with a challenge: allowing people to benefit from the technology while ensuring that the legal arguments they submit are reliable.

For many Americans, the attraction of AI begins with the difficulty of obtaining affordable legal assistance. Low-income Americans “do not get any or enough legal help for 92% of their substantial civil legal problems,” the Legal Services Corporation (LSC) found in its 2022 Justice Gap study. The study also found that organizations funded by LSC, the largest national funder of civil legal aid, “must turn away 1 out of every 2 (49%) requests they receive due to limited resources.” LSC President Ronald S. Flagg pointed to AI as part of the answer in written testimony to the Senate Judiciary Committee in July 2024, discussing how carefully developed AI tools could provide basic legal information, answer common questions and help people navigate the legal system. In a report from its 2024-2025 Technology Summit, LSC later recommended supporting “high-quality self-help tools that leverage emerging technologies to improve access to justice for self-represented litigants.”

Research suggests that the availability of AI is coinciding with changes in how people approach the courts without legal representation. The self-represented share of federal civil cases rose “from a long-term steady-state average of 11% to 16.8% in FY2025” as generative AI tools spread, according to Anand Shah of the Massachusetts Institute of Technology and Joshua Levy of the University of Southern California, who posted a working paper to the research site SSRN in May. In a random sample of 1,600 complaints, the share that tested positive for AI-generated text rose from “essentially zero in the pre-AI period to more than 18% in 2026.” Because the figure reflects the results of a detection method, it indicates likely, rather than confirmed, AI use. The findings suggest that AI is becoming part of the process for people who might otherwise struggle to prepare legal documents on their own.

Some courts and legal researchers are exploring how to make that assistance more useful. In England, many judges find AI-assisted submissions from people without lawyers “more helpful, easier to digest,” Lady Chief Justice Sue Carr of England and Wales told the House of Lords Constitution Committee on June 10, according to the official transcript. “AI, we are seeing, can really help litigants in person have access to justice because they can get support in drafting their submissions,” she said. She added that judges are trained to recognize fabricated cases and other AI-generated errors.

Stanford Law School announced in August that it is working with the Superior Court of Los Angeles County to test tools for tenants without lawyers in eviction cases, including an AI-enabled online dispute resolution platform. In 2023, lawyers represented nearly 92% of landlords in the county’s eviction cases but only about 14% of tenants, according to the announcement. “Court users must be at the center of judicial system efforts to improve processes and develop effective legal-help resources,” said Margaret Hagan, executive director of Stanford’s Legal Design Lab. Presiding Judge Sergio C. Tapia II said the partnership “will help us serve the public more effectively.”

Yet the same technology that may help people prepare legal documents can also produce information that does not exist. General-purpose AI models answered direct, verifiable questions about federal court cases incorrectly “between 58% (ChatGPT 4) and 88% (Llama 2) of the time,” researchers Matthew Dahl, Varun Magesh, Mirac Suzgun and Daniel E. Ho found in a 2024 study in the Journal of Legal Analysis. The models also tended to accept false premises built into users’ questions, and the authors noted that users may not realize their questions contain errors, which they said is “especially true for unsophisticated pro se litigants.” The authors concluded that “the risks are highest for those who would benefit from LLMs most,” whom they identified as “under-resourced or pro se litigants.”

Those risks have already led to sanctions against people representing themselves. Submitting hallucinated case citations is “unjustifiable,” whether “intentional or unintentional,” the Arizona Court of Appeals ruled July 15 when it sanctioned Jeffrey Kotchka, who represented himself in a dispute over his mother’s estate. Six of the eight citations in his brief were deficient, and two of the cases he cited did not exist, according to the opinion. Kotchka told the court he did “independent internet research” using AI-assisted tools and admitted he took no steps to verify the citations. Judge Brian Y. Furuya wrote the opinion for the three-judge panel, which ordered Kotchka to pay the share of his sister’s attorney fees that the faulty citations caused.

The 10th U.S. Circuit Court of Appeals took a similar approach in December 2025 after Rashonna Moore, who represented herself in an employment discrimination lawsuit against the city of Del City, Oklahoma, cited “eleven cases we could not locate on Westlaw.” When the court asked her to explain, she cited another case that the panel called “yet another hallucination.” The judges affirmed the lower court's judgment and, as an alternative sanction, dismissed her appeal, concluding that “Moore’s misuse of GenAI is sanctionable.” According to the opinion, Moore neither admitted nor denied using AI, and her claims failed for a separate reason: she did not file her discrimination charge on time.

Lawyers are not exempt from the same responsibility to verify AI-generated material. AI hallucinations in court filings are “a problem that is getting worse,” Chief Judge Jennifer Walker Elrod wrote for the 5th Circuit in Fletcher v. Experian Information Solutions, the February ruling that Gonzalez cited. The court applied the same verification standard to a lawyer, fining attorney Heather Hersh $2,500 after it found fabricated quotations in her brief. Hersh told the court she had used AI to “help organize and structure [her] arguments and to break up overly long paragraphs.” The court rejected that explanation as not credible and concluded that she had used generative AI to draft much of the brief without checking it.

Beyond the consequences for individual litigants and lawyers, courts are also dealing with the additional work created by AI-assisted filings. Shah and Levy also found that self-represented cases generated 158% more docket entries per court in their first 180 days in 2025 than the pre-AI average, adding that “pro se cases are not terminating faster” and that “these cases are placing a larger burden on federal district courts.” Oregon Court of Appeals Chief Judge Erin C. Lagesen described the same strain in an April 29 Oregon Judicial Department release. She said filings with fabricated legal authority “are being submitted by lawyers and self-represented litigants alike” and “create a significant drain on court resources that, at present, appears to be growing.”

AI products “may seem like an appealing shortcut to legal research and presenting legal arguments,” Oregon Supreme Court Chief Justice Meagan A. Flynn said in a separate June 5 Oregon Judicial Department release, which announced the court’s first AI-related orders. “But when the court has to spend considerable time and effort addressing fabricated legal arguments, it comes at the expense of other cases,” she said. In a June 25 standing order, U.S. District Judge Ernest Gonzalez of the Western District of Texas likewise wrote that while “the Court recognizes that AI can be a beneficial tool,” a lawyer using it “must ensure that the legal propositions and authority generated are trustworthy,” quoting a February ruling by the 5th U.S. Circuit Court of Appeals. The order requires any party that uses generative AI on a filing in his court, including those “appearing pro se,” to disclose that use and certify that they independently verified the document’s legal and factual assertions.

People facing a civil legal problem without an attorney can find help through LSC’s directory of local legal-aid organizations, the list of free and affordable legal help on USA.gov or their local court’s self-help resources. People who use AI to prepare court documents are responsible for what they file, as the rulings against Kotchka and Moore show. Chief Justice John Roberts described both sides of that balance in his 2023 Year-End Report on the Federal Judiciary. “For those who cannot afford a lawyer, AI can help,” Roberts wrote, but “any use of AI requires caution and humility.”

Alexander Fernandez

Life News Today