What was hallucinated

Fabricated: Case Law | Amended Motion cited 'In re Berry Good, LLC, No. 4:20-bk-12345 (Bankr. E.D. Ark. May 2020)'; court found no such case or supporting order and determined the citation was AI-generated.

Details

Counsel's Amended Motion cited a non-existent case (In re Berry Good, LLC) generated by AI. Court found the citation fabricated, counsel admitted AI use and negligence, implemented safeguards, and court declined sanctions while transmitting matter to disciplinary office. The court's reflections are worth quoting in full."When examined, paragraph 15 is the result of a flawed AI search that suggested an incorrect generalization falsely supported by a specific but non-existent case. Under any scenario, a suspect argument buttressed with a case from whole cloth violates the expressed and tacit norms governing the practice of law as it intersects with the courts. Simply, you cannot make stuff up to convince a court to do something.In simpler times, that would end the inquiry with draconian consequences. The times, however, are no longer simple; thus, the inquiry does not end here.AI interposes additional considerations that defy easy categorization or scrutiny when examining the traditional norms of practice and advocacy. Threshold, two fields of inquiry present in any AI debate. The first is whether exposition artificially generated by a computer is acceptable in what has contextually been a forum for human critical analysis, thought, and advocacy. That is not the instance here. Rather, it is the second; that is, when AI sacrifices accuracy to satisfy the consumer and that inaccuracy is advocated before the court.[...]The only acceptable conclusion is that citing a made-up case is a false representation to the court that violates the expressed norms of practice, is actionable, and should bear consequences. The answer, however, to the second question —whether an artificial component alters the calculus—is more difficult.Here, our traditional norms are challenged. In context, the traditional advocacy process begins with black letter law in the form of statutes, codes, and regulations. Then, there are judicial opinions and orders interpreting black letter law in the context of justiciable issues based on discrete facts. Lawyers cite cases for their binding or persuasive authority and have done so ever since someone started committing judicial decisions to stone, parchment, or paper. The ability to find and cite cases has evolved over time from laborious research in courthouses, to compendiums, indices, digests, reporters, advance sheets, key numbers, and now the internet.The internet changed this dynamic in two ways. First, it streamlined, refined, and improved the research process. Users could select fields, topics, and key numbers to find and locate cases. Eventually, more specific tools, such as key words or phrases, enhanced the process. Second, and more pertinent to the present issue, AI is creating a process where instead of just finding authority, the computer also does the supplicant's thinking and analysis for them. The first innovation continues to evolve; the second is new, likewise continues to evolve, but constitutes an entirely new dimension, rather than a continuum, in legal practice. Finding has evolved into finding and thinking.Travel back to research, the first changed dynamic. From stone to the internet, how lawyers found and used case law took many forms. Some would carefully read every case they found; some would read every page of select cases they thought pertinent; others might read the digest entry and then read only that section of the case; others might read only the digest entry; others barely read anything at all and cited anything that looked like it might support their position. Each was a personal decision by the lawyer for which he or she is accountable under the traditional norms of practice both officially—through codes of conduct and Rule 9011—and unofficially per the norms and expectations of courts.AI compresses this historical dynamic. The research engine is now the search and thinking engine. AI finds case law and tells the subscriber that it supports their position. All the personalities described above—and known oh-so-well to us all—have become one.Except, however, AI is flawed. In its infancy—albeit one that may be in college soon—it is immature and does not always provide accurate information or correct analysis; it may even make things up. A tool that potentially supplants your advocacy by doing the finding and thinking for you is enticing. 3 But in the end, it is only a tool. Despite our near total and unfortunate faith in the internet, it does not relieve the attorney of his or her responsibility to make sure that the information and analysis are correct.Reliance on AI can, however, mitigate intent. Here, Counsel is responsible for their negligence and misplaced reliance on AI, but there is no indication—and this court does not believe —that they purposely misled the court. They did not decide to make up a case. Misplaced reliance or negligence does not always equal intentional misrepresentation."

Sanction teardown · E.D. Arkansas, USA · 2025-09-03

In re Whitehall Pharmacy LLC

What happened

In E.D. Arkansas, USA, a filing relied on an unnamed/unconfirmed AI tool to help draft legal argument. The court identified the following problems with the citations in that filing:

Fabricated (Case Law)

Amended Motion cited 'In re Berry Good, LLC, No. 4:20-bk-12345 (Bankr. E.D. Ark. May 2020)'; court found no such case or supporting order and determined the citation was AI-generated.

Which AI tool

an unnamed/unconfirmed AI tool. Note: Charlotin's public database records tool attribution only where a court order, brief, or reporting on the matter states it explicitly; "unidentified" or "implied" means the record indicates AI use but does not name a specific product — we do not guess.

Outcome

OSC withdrawn and dismissed; no sanctions imposed; matter transmitted to Arkansas Office of Professional Conduct

Additional detail

Counsel's Amended Motion cited a non-existent case (In re Berry Good, LLC) generated by AI. Court found the citation fabricated, counsel admitted AI use and negligence, implemented safeguards, and court declined sanctions while transmitting matter to disciplinary office. The court's reflections are worth quoting in full."When examined, paragraph 15 is the result of a flawed AI search that suggested an incorrect generalization falsely supported by a specific but non-existent case. Under any scenario, a suspect argument buttressed with a case from whole cloth violates the expressed and tacit norms governing the practice of law as it intersects with the courts. Simply, you cannot make stuff up to convince a court to do something.In simpler times, that would end the inquiry with draconian consequences. The times, however, are no longer simple; thus, the inquiry does not end here.AI interposes additional considerations that defy easy categorization or scrutiny when examining the traditional norms of practice and advocacy. Threshold, two fields of inquiry present in any AI debate. The first is whether exposition artificially generated by a computer is acceptable in what has contextually been a forum for human critical analysis, thought, and advocacy. That is not the instance here. Rather, it is the second; that is, when AI sacrifices accuracy to satisfy the consumer and that inaccuracy is advocated before the court.[...]The only acceptable conclusion is that citing a made-up case is a false representation to the court that violates the expressed norms of practice, is actionable, and should bear consequences. The answer, however, to the second question —whether an artificial component alters the calculus—is more difficult.Here, our traditional norms are challenged. In context, the traditional advocacy process begins with black letter law in the form of statutes, codes, and regulations. Then, there are judicial opinions and orders interpreting black letter law in the context of justiciable issues based on discrete facts. Lawyers cite cases for their binding or persuasive authority and have done so ever since someone started committing judicial decisions to stone, parchment, or paper. The ability to find and cite cases has evolved over time from laborious research in courthouses, to compendiums, indices, digests, reporters, advance sheets, key numbers, and now the internet.The internet changed this dynamic in two ways. First, it streamlined, refined, and improved the research process. Users could select fields, topics, and key numbers to find and locate cases. Eventually, more specific tools, such as key words or phrases, enhanced the process. Second, and more pertinent to the present issue, AI is creating a process where instead of just finding authority, the computer also does the supplicant's thinking and analysis for them. The first innovation continues to evolve; the second is new, likewise continues to evolve, but constitutes an entirely new dimension, rather than a continuum, in legal practice. Finding has evolved into finding and thinking.Travel back to research, the first changed dynamic. From stone to the internet, how lawyers found and used case law took many forms. Some would carefully read every case they found; some would read every page of select cases they thought pertinent; others might read the digest entry and then read only that section of the case; others might read only the digest entry; others barely read anything at all and cited anything that looked like it might support their position. Each was a personal decision by the lawyer for which he or she is accountable under the traditional norms of practice both officially—through codes of conduct and Rule 9011—and unofficially per the norms and expectations of courts.AI compresses this historical dynamic. The research engine is now the search and thinking engine. AI finds case law and tells the subscriber that it supports their position. All the personalities described above—and known oh-so-well to us all—have become one.Except, however, AI is flawed. In its infancy—albeit one that may be in college soon—it is immature and does not always provide accurate information or correct analysis; it may even make things up. A tool that potentially supplants your advocacy by doing the finding and thinking for you is enticing. 3 But in the end, it is only a tool. Despite our near total and unfortunate faith in the internet, it does not relieve the attorney of his or her responsibility to make sure that the information and analysis are correct.Reliance on AI can, however, mitigate intent. Here, Counsel is responsible for their negligence and misplaced reliance on AI, but there is no indication—and this court does not believe —that they purposely misled the court. They did not decide to make up a case. Misplaced reliance or negligence does not always equal intentional misrepresentation."

How Citation Safe would have caught this

Citation Safe runs three deterministic layers before a brief is filed: (1) does the citation exist against CourtListener's database of published opinions, (2) if quoted, does that exact language appear in the source, (3) does the cited case actually support the proposition it is cited for. Fabricated case citations fail Layer 1. Fabricated or misattributed quotations fail Layer 2 even when the underlying case is real. Misrepresented holdings — a real case cited for a proposition it does not support — are the target of Layer 3. None of these checks involve asking another language model whether the citation looks right; they are lookups and text-matches against the actual source, which is why a hallucinated citation has to survive a direct lookup against the authoritative source — not another model's opinion — to earn a VERIFIED stamp; our measured false-verify rate is published live at /quality.

Check a brief before you file it → · See our live false-verify rate

Source: https://www.damiencharlotin.com/documents/788/In_re_Whitehall_Pharmacy_LLC.pdf, via Damien Charlotin's public AI Hallucination Cases Database (CC0).