Ever since Artificial Intelligence has become a staple in professional research, there is a new term that is entering the judicial lexicon: hallucination. This occurs when a Large Language Model (LLM) generates a response that is confident, authoritative, and perfectly formatted… but entirely false.
For the unsuspecting researcher, these often manifest as phantom cases, meaning legal citations that look real, sound plausible, but do not exist in any law book.
The high cost of “fake” cases
One example of this occurred in the case of Mata v. Avianca, where a legal team used AI to draft a brief. The LLM hallucinated several non-existent judicial decisions, complete with fake internal citations and quotes. The attorneys, relying on the AI, submitted the brief to a federal court. This had consequences such as a rebuke from the bench and thousands of dollars in sanctions, as technology can be used only as a tool and not to substitute an attorney’s professional judgment.
How courts are reacting
Many judges across the country are no longer taking “AI error” as a valid excuse, and they are implementing several measures to combat this:
- Certificates of compliance: A judge may require a signed certificate stating that any brief filed using AI has been verified by a human being for accuracy.
- Heightened Rule 11 sanctions: Under federal law, attorneys have a duty to perform a reasonable inquiry. Submitting a hallucinated case may be treated as a failure of that duty, leading to heavy fines.
- Disciplinary Boards: Some judges have reported attorneys to the disciplinary board for review.
- Standing orders on generative AI: A court may prohibit the use of AI-generated research unless it is explicitly disclosed and double-checked against traditional legal databases.
While AI offers help with creative work and data processing, it can misconstrue facts or even make up entire laws and bills. But an experienced law firm can use their legal expertise to discern fact from fiction.