Tomorrowish · Bibliography Verifier

How to check whether a citation is real

A reference can look complete and still point to the wrong work, or to no work at all. This page covers where to look a reference up, what each source misses, and the error patterns studies have found.

What a DOI is and how to resolve it

A DOI is "a unique number made up of a prefix and a suffix separated by a forward slash" (DOI Foundation). To resolve one, add https://doi.org/ in front of it, as in https://doi.org/10.1000/182, and the system redirects you to the object (DOI Foundation).

A working DOI does not prove the reference is right. In one APA Style test, a DOI given by ChatGPT "belongs to a different article" (APA Style, 2023). Always compare the title, authors, year and journal on the landing page with the reference.

Where to look, and how big each source is

• Crossref registers DOIs for scholarly content. Its metadata "is deposited directly by our members, who publish the content" (Crossref REST API docs). Its status page listed 189,485,984 total records, updated 24 October 2026 (Crossref status).
• DataCite registers DOIs for research outputs such as data, preprints, images and samples (DataCite). Its homepage counter showed more than 139.8 million DOIs registered to date (DataCite).
• PubMed "contains more than 40 million citations and abstracts of biomedical literature" (page last updated 11 March 2025) (PubMed About). It draws on MEDLINE, PubMed Central and NCBI Bookshelf (PubMed About).
• arXiv hosts "more than three million scholarly articles in eight subject areas" (arXiv About). "Material is not peer-reviewed by arXiv" (arXiv About).
• OpenAlex says it has "over 320 million works" (no date shown) (OpenAlex docs). It is built mainly from Microsoft Academic Graph and Crossref, plus DataCite, PubMed, HAL and repositories (OpenAlex docs).
• Open Library, an Internet Archive project, aims for "One web page for every book ever published" (Open Library About). On 24 October 2026 it listed 41,806,546 works and 56,898,953 editions (Open Library stats).
• Library of Congress recorded more than 178.2 million items in fiscal year 2023, including more than 25.77 million cataloged books (Library of Congress). Its catalogue is at catalog.loc.gov.

What these sources miss

• Not every publication has a DOI. A 2023 study found "a considerable amount of publications from developing countries" lack DOIs (Turki et al., 2023).
• Crossref only holds what its members deposit (Crossref REST API docs). A book, report or thesis from a publisher that is not a member will not appear there.
• PubMed is limited to biomedical literature, and arXiv to physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics (PubMed About; arXiv About).
• Theses and books are better checked in a library catalogue. WorldCat lists 30 million theses and dissertations and 405 million books (WorldCat).
• Not finding a work in one database is not proof it does not exist. One study searched Google, Scopus, PubMed and WorldCat, then the journal's own volume and issue, before calling a reference fabricated (Linardon et al., 2025).

Error patterns documented in studies

• Walters and Wilder (Scientific Reports, 7 September 2023). In 636 citations from 84 ChatGPT-written reviews, 55% of GPT-3.5 and 18% of GPT-4 citations were fabricated (Walters & Wilder). Of real works cited, 43% (GPT-3.5) and 24% (GPT-4) had substantive errors (Walters & Wilder). Most fabricated citations "include the names of real journals, publishers, and organizations" (Walters & Wilder).
• Bhattacharyya et al. (Cureus, 19 May 2023). Of 115 ChatGPT references in medical texts, 47% were fabricated, 46% were authentic but inaccurate, and 7% were authentic and accurate (Bhattacharyya et al.). An incorrect PubMed ID was the most common error (Bhattacharyya et al.).
• Real title, wrong authors, year or journal. McGowan et al. found 12 of 35 ChatGPT citations had "titles with incorrect author lists, journals, or publication years" (McGowan et al., 2023).
• Real author, invented paper. A taxonomy of 100 hallucinated citations in NeurIPS 2025 papers lists "real authors attributed to papers they did not write" (Ansari, 2026, preprint).
• DOI resolves to an unrelated paper. When GPT-4o gave a DOI for a fabricated citation, 64% (21/33) "were valid but linked to irrelevant and unrelated journal articles" (Linardon et al., JMIR Mental Health, 12 November 2025).
• Rates vary by tool. Chelli et al. found hallucination rates of 39.6% (GPT-3.5), 28.6% (GPT-4) and 91.4% (Bard) (Chelli et al., JMIR, 22 May 2024).

Retractions

Crossref announced on 12 September 2023 that it had acquired the Retraction Watch database and made it public (Crossref blog). Since 29 January 2025, Retraction Watch retractions are in the Crossref REST API (Crossref blog). Retraction Watch updates the data every working day (Crossref docs). For MEDLINE journals, the ICMJE says to search PubMed for "Retracted publication [pt]" (ICMJE, January 2026).

Manual checklist

1. Search the exact title in quotes. If nothing comes up, search a few distinctive words, since a swapped synonym "reduces the value of the title as a search string" (Walters & Wilder).
2. Resolve the DOI at https://doi.org/ and compare title, every author, year, journal, volume and pages.
3. If there is no DOI, or it does not match, open the journal's own website and find the cited volume and issue.
4. For books and theses, search WorldCat, the Library of Congress catalogue or Open Library. Check the edition and year.
5. For older works, check the original publication year. Citing the online posting date instead is a common error (Walters & Wilder).
6. For preprints, check for a published version. The ICMJE says to cite the published article "whenever appropriate" (ICMJE).
7. Check retractions in the Crossref REST API (filter update-type:retraction) or with PubMed's "Retracted publication [pt]".

Sources

All accessed 2026-10-24. Each entry gives the date shown on the source, if any.

• doi: DOI Foundation, "What is a DOI?"; no date shown. https://www.doi.org/the-identifier/what-is-a-doi/
• apa-chatgpt: McAdoo, APA Style blog; created 7 April 2023, last updated 15 September 2025; live page blocked automated access, text checked in an Internet Archive copy of 25 September 2026. https://apastyle.apa.org/blog/how-to-cite-chatgpt
• cr-api: Crossref; last updated 8 April 2020. https://www.crossref.org/documentation/retrieve-metadata/rest-api/
• cr-stats: Crossref status page; updated 24 October 2026. https://www.crossref.org/06members/53status.html
• dc: DataCite homepage counter; computed from registrations up to the previous day. https://datacite.org/
• pm: NLM; last updated 11 March 2025. https://pubmed.ncbi.nlm.nih.gov/about/
• ax: arXiv; no date shown. https://info.arxiv.org/about
• oa: OpenAlex, "Works overview"; no date shown. https://help.openalex.org/data/works/
• ol-about: Open Library; last edited 15 April 2024. https://openlibrary.org/about
• ol-stats: Open Library; live figures, 24 October 2026. https://openlibrary.org/stats
• loc: Library of Congress; fiscal year 2023 figures. https://www.loc.gov/about/general-information/
• loc-cat: https://catalog.loc.gov/
• wc: WorldCat homepage figures; no date shown. https://search.worldcat.org/
• turki: Turki et al., Frontiers in Research Metrics and Analytics, 17 August 2023. https://doi.org/10.3389/frma.2023.1207980
• ww: Walters & Wilder, Scientific Reports 13, 14045, 7 September 2023; full text read at https://pmc.ncbi.nlm.nih.gov/articles/PMC10484980/. https://doi.org/10.1038/s41598-023-41032-5
• bhatt: Bhattacharyya et al., Cureus, 19 May 2023; https://pmc.ncbi.nlm.nih.gov/articles/PMC10277170/. https://doi.org/10.7759/cureus.39238
• mcgowan: McGowan et al., Psychiatry Research 326, 2023. https://doi.org/10.1016/j.psychres.2023.115334
• ansari: Ansari, arXiv preprint, not peer reviewed, submitted 5 February 2026. https://arxiv.org/abs/2602.05930
• linardon: Linardon et al., JMIR Mental Health, 12 November 2025; https://pmc.ncbi.nlm.nih.gov/articles/PMC12658395/. https://doi.org/10.2196/80371
• chelli: Chelli et al., Journal of Medical Internet Research 26, e53164, 22 May 2024. https://doi.org/10.2196/53164
• cr-rw: Crossref; 12 September 2023. https://www.crossref.org/blog/news-crossref-and-retraction-watch/
• cr-rw-api: Crossref; 29 January 2025. https://www.crossref.org/blog/retraction-watch-retractions-now-in-the-crossref-api/
• cr-rw-docs: Crossref; last updated 19 January 2025. https://www.crossref.org/documentation/retrieve-metadata/retraction-watch/
• icmje: ICMJE Recommendations; updated January 2026. https://www.icmje.org/icmje-recommendations.pdf