Expert Accused of Inserting Fake ChatGPT Citations into Document | Generative ai course free | Generative ai in banking pdf | Generative ai certification | Turtles AI

A disinformation expert has come under fire for using ChatGPT to draft a legal statement, inadvertently including inaccurate details. The incident highlights the risks associated with using AI in legal settings and communications.

Key Points:

• A Stanford professor used ChatGPT to simplify citations in a legal document.
• The AI ​​generated inaccurate references, compromising the reliability of the text.
• The error was called inadvertent, and the main content was defended as accurate.
• The case highlights the dangers of unwitting use of AI tools in sensitive areas.

The growing use of AI in document preparation and organization has sparked a new debate after a disinformation expert admitted to using ChatGPT to compile a sworn statement related to a court case. The case involves Jeff Hancock, a professor at Stanford University who supported a Minnesota law to ban the use of deep fake technologies to influence elections. Ironically, in an effort to combat AI, Hancock has come under fire for including false AI-generated details in his legal document.

Hancock reportedly used GPT-4o to streamline the process of organizing the bibliographic citations included in his statement. However, the AI ​​added nonexistent references, creating an accuracy issue that undermined the document’s reliability. The expert said the tool was used only to compile citations, leaving out other parts of the draft. Hancock also stressed that the error was unintentional, attributing it to his lack of knowledge of so-called AI “hallucinations,” a known phenomenon where language models produce fabricated or inaccurate information.

In a subsequent statement, Hancock expressed regret for the confusion, saying that his intention was not to mislead the court or the lawyers involved. He reiterated that all the substantive points in the affidavit were valid, supported by academic research in the field of misinformation and the effects of AI technology on society. He also explained that he had used a combination of Google Scholar and GPT-4o to create the reference list and had not adequately checked the accuracy of the citations.

The case, while declared to be an unintentional error, highlights the risks inherent in using AI tools, especially in areas where accuracy and credibility are of paramount importance. Incidents like this show how even high-profile experts can make significant mistakes if the capabilities and limitations of the technology are not fully understood. While Hancock said the paper’s core points remain valid, the inclusion of misquotations has raised questions about the reliability of content generated in part by AI, especially when it impacts legal or policy decisions.

In addition, the case is a concrete example of how AI is revolutionizing communication and academic work, but it also highlights the need for greater vigilance in the use of such tools.