AI-Fabricated Citations In Over 2,800 Biomedical Journal Articles
The use of artificial intelligence (AI) in scientific research has come under scrutiny recently, with a research team uncovering a significant number of fabricated references in published journal articles. According to a correspondence to The Lancet, over a three-year period, 4,046 references in 2,810 scientific papers were found to be completely fabricated, likely due to AI-generated content.
To investigate this issue, a team of researchers from Columbia University and the University of Eastern Finland developed an automated reference verification system using AI technology. By comparing citations in papers to bibliographic records, they were able to identify discrepancies and flag potentially fabricated references. Using a large language model known as Claude 3.5 Haiku, the team sorted through the flagged references to differentiate between honest errors and fabricated content.
The results of their investigation were alarming. In 2023, approximately one in 2828 papers had at least one fabricated reference. By 2025, this number had increased to one in 458, and in the first seven weeks of 2026, the ratio had risen even further to one in 277 papers. This represents a significant increase in fabricated citations over a short period of time.
One particularly egregious example was a paper published in 2025 in an oncology journal, where 60% of the references cited were found to be fabricated. The research team also identified authors who consistently included fabricated references in their papers, suggesting the presence of “paper mills” that churn out fraudulent research.
The rise in fabricated citations has been linked to the increased use of generative AI platforms, such as ChatGPT, Claude, and Perplexity, which rely on large language models. These models are susceptible to hallucinations, as they may inaccurately associate information found in vast amounts of data without critical verification.
As the use of AI in scientific research continues to grow, the problem of fabricated citations is likely to worsen. The proliferation of for-profit scientific journals has led to a decline in quality control, as researchers are pressured to publish in these journals to advance their careers. Developing AI tools to detect inaccuracies, especially AI-generated content, may be one way to combat this issue.
In conclusion, the scientific publishing industry is facing a reckoning as researchers increasingly rely on AI to produce research papers. As funding for scientific research dwindles and publication fees soar, researchers may be tempted to cut corners and use AI-generated content without proper verification. This trend underscores the need for rigorous quality control measures in scientific publishing to maintain the integrity of research.
