This study develops an AI-detection model for news articles by distinguishing texts from a well-known English-language newspaper from those generated by NewsGPT, a platform that uses generative AI to produce news content from existing sources. Given the risks of AI-generated journalism, including fabricated news and the amplification of misinformation, the task is framed as an authorship attribution problem within a machine learning framework. We constructed a balanced dataset of approximately 14,000 articles: half drawn from the newspaper before the launch of ChatGPT, to minimise the likelihood of LLM-assisted writing, and half generated by NewsGPT. We employed a radial kernel Support Vector Machine classifier using several feature sets, including the most frequent words and coordinates obtained through dimensionality reduction methods such as Correspondence Analysis and Multidimensional Scaling. We also exploited a Large Language Model through contextual embeddings and fine-tuning. Across all feature sets, the models identified article origin with accuracy close to or above 99%. To enhance interpretability, we examined the most important variables using SHAP values and assessed prediction confidence through conformal p-values. Finally, we applied the models to 2025 articles from the same newspaper, identifying a subset with characteristics consistent with AI-assisted writing and further screening those articles using external AI detection services.
Is there any AI-generated news in an online newspaper? An AI detection model for authorship attribution
Sciandra A.
2026
Abstract
This study develops an AI-detection model for news articles by distinguishing texts from a well-known English-language newspaper from those generated by NewsGPT, a platform that uses generative AI to produce news content from existing sources. Given the risks of AI-generated journalism, including fabricated news and the amplification of misinformation, the task is framed as an authorship attribution problem within a machine learning framework. We constructed a balanced dataset of approximately 14,000 articles: half drawn from the newspaper before the launch of ChatGPT, to minimise the likelihood of LLM-assisted writing, and half generated by NewsGPT. We employed a radial kernel Support Vector Machine classifier using several feature sets, including the most frequent words and coordinates obtained through dimensionality reduction methods such as Correspondence Analysis and Multidimensional Scaling. We also exploited a Large Language Model through contextual embeddings and fine-tuning. Across all feature sets, the models identified article origin with accuracy close to or above 99%. To enhance interpretability, we examined the most important variables using SHAP values and assessed prediction confidence through conformal p-values. Finally, we applied the models to 2025 articles from the same newspaper, identifying a subset with characteristics consistent with AI-assisted writing and further screening those articles using external AI detection services.
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
