Integrating Synthetic and Human Data for Complex Psychosocial Categories in Text Classification

The analysis of online discourse surrounding societal crises requires text classification models capable of detecting theoretically grounded psychosocial constructs, which are often difficult to operationalize due to limited annotated data and strong class imbalance. This study addresses this methodological challenge using the Risk Co-De classification system, a taxonomy describing risk perception as a continuum articulated into four macro-categories: Consciousness, Justification, Distance, and Denial. An English-language classifier was developed by combining human-annotated and synthetic data. A large corpus of English tweets related to climate change and COVID-19 was first pre-classified using a multilingual BERT model trained on the original Italian Risk Co-De dataset, and then manually recoded to obtain a curated training set aligned with the theoretical taxonomy. To compensate for the limited size and unbalanced distribution of the annotated corpus, additional tweets were generated through a controlled synthetic data generation pipeline based on large language models, with explicit constraints on category, subtype, topic, and communicative style. Classification analyses compared human-only, synthetic-only, and combined training conditions, and evaluated the effect of incremental augmentation. Results show that synthetic data alone do not fully reproduce the linguistic variability of natural discourse, but their integration with human data reduces class imbalance effects and improves detection of underrepresented categories without degrading overall performance. Distributional analyses further indicate that synthetic texts preserve the general semantic structure of the corpus while showing lower lexical variability.