• An Evaluation of Current Trend...

Treffer: An Evaluation of Current Trends in AI-Generated Text in Otolaryngology Publications.

Title:
An Evaluation of Current Trends in AI-Generated Text in Otolaryngology Publications.
Authors:
Kutler RB; Department of Otolaryngology - Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA., Setzen SA; Department of Otolaryngology - Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA., Tsai S; Department of Otolaryngology - Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA., Rameau A; Department of Otolaryngology - Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA.
Source:
The Laryngoscope [Laryngoscope] 2025 Oct; Vol. 135 (10), pp. 3579-3587. Date of Electronic Publication: 2025 Apr 25.
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: Wiley-Blackwell Country of Publication: United States NLM ID: 8607378 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1531-4995 (Electronic) Linking ISSN: 0023852X NLM ISO Abbreviation: Laryngoscope Subsets: MEDLINE
Imprint Name(s):
Publication: <2009- >: Philadelphia, PA : Wiley-Blackwell
Original Publication: St. Louis, Mo. : [s.n., 1896-
MeSH Terms:
Otolaryngology*/trends , Periodicals as Topic*/trends , Natural Language Processing*, Humans ; Language
References:
Eur Arch Otorhinolaryngol. 2024 Nov;281(11):6141-6146. (PMID: 39014250)
JAMA Otolaryngol Head Neck Surg. 2024 Jun 1;150(6):457-458. (PMID: 38635259)
Cureus. 2023 Nov 18;15(11):e49019. (PMID: 38111405)
JAMA. 2023 Sep 5;330(9):866-869. (PMID: 37548965)
Am J Obstet Gynecol. 2024 Aug;231(2):276.e1-276.e10. (PMID: 38710267)
Laryngoscope Investig Otolaryngol. 2023 Dec 09;9(1):e1193. (PMID: 38362184)
J Am Acad Orthop Surg. 2023 Dec 1;31(23):1173-1179. (PMID: 37671415)
Cureus. 2023 Feb 19;15(2):e35179. (PMID: 36811129)
Patterns (N Y). 2023 Jul 10;4(7):100779. (PMID: 37521038)
J Med Internet Res. 2023 May 31;25:e46924. (PMID: 37256685)
EBioMedicine. 2023 Apr;90:104512. (PMID: 36924620)
JMIR Med Educ. 2024 Jan 16;10:e49970. (PMID: 38227351)
Malays Fam Physician. 2023 Sep 25;18:58. (PMID: 37814667)
Cureus. 2023 May 19;15(5):e39238. (PMID: 37337480)
NPJ Digit Med. 2023 Apr 26;6(1):75. (PMID: 37100871)
Nature. 2023 Jan;613(7944):423. (PMID: 36635510)
Cureus. 2024 May 8;16(5):e59857. (PMID: 38854312)
Eur Arch Otorhinolaryngol. 2023 Sep;280(9):4271-4278. (PMID: 37285018)
Lancet Digit Health. 2023 Jun;5(6):e333-e335. (PMID: 37120418)
Commun Med (Lond). 2023 Oct 10;3(1):141. (PMID: 37816837)
J Korean Med Sci. 2023 Sep 25;38(38):e319. (PMID: 37750374)
Future Healthc J. 2021 Jul;8(2):e188-e194. (PMID: 34286183)
Princ Pract Clin Res. 2023 Jul 10;9(1):1-4. (PMID: 37693832)
Grant Information:
K76 AG079040 United States AG NIA NIH HHS; OT2 OD032720 United States OD NIH HHS; OT2 OD032720 United States CD ODCDC CDC HHS; OT2 OD032720 United States CD ODCDC CDC HHS; K76 AG079040 United States AG NIA NIH HHS
Contributed Indexing:
Keywords: artificial intelligence; large language models; otolaryngology
Entry Date(s):
Date Created: 20250425 Date Completed: 20250927 Latest Revision: 20250929
Update Code:
20250929
PubMed Central ID:
PMC12353330
DOI:
10.1002/lary.32202
PMID:
40277459
Database:
MEDLINE

Weitere Informationen

Objectives: Since the release of ChatGPT-4 in March 2023, large language models (LLMs) application in biomedical manuscript production has been widespread. GPT-modified text detectors, such as GPTzero, lack sensitivity and reliability and do not quantify the amount of AI-generated text. However, recent work has identified certain adjectives more frequently used by LLMs that can help identify and quantify LLM-modified text. The aim of this study is to utilize these adjectives to identify LLM-generated text in otolaryngology publications.
Study Design: Meta-research.
Methods: Twenty-five otolaryngology journals were studied between November 2022 and July 2024, encompassing 8751 published works. Articles from countries where ChatGPT-4 is not available were removed, yielding 7702 articles for study inclusion. These publications were analyzed using a Python script to determine the frequency of the top 100 adjectives disproportionately generated by ChatGPT-4.
Results: A significant increase in the frequency of adjectives associated with GPT use was observed from November 2023 to July 2024 across all journals (p < 0.001), with a significant difference before and after the release of ChatGPT in March 2023. Journals with higher impact factors had significantly lower usage of GPT-associated adjectives than those with lower impact factors (p < 0.001). There was no significant difference in GPT-associated adjective use by first authors with a doctoral degree versus those without. Publications by authors from English-speaking countries demonstrated a significantly more frequent use of LLM-associated adjectives (p < 0.001).
Conclusions: This study suggests that ChatGPT use in otolaryngology manuscript production has significantly increased since the release of ChatGPT-4. Future research should be aimed at further characterizing the landscape of AI-generated text in otolaryngology and developing tools that encourage authors' transparency regarding the use of LLMs.
Level of Evidence: NA.
(© 2025 The American Laryngological, Rhinological and Otological Society, Inc.)