Systematic reviews are vital for evidence-based decision-making but remain resource-intensive due to the volume of literature requiring expert screening. Technology-Assisted Review (TAR) systems offer a solution by ranking documents for review, yet questions remain about how best to allocate limited human effort across multiple review topics. In this paper, we explore the problem of effort distribution by comparing alternative screening policies under fixed effort constraints. Using real-world data from the CLEF eHealth 2017-2019 TAR tasks, we evaluate both baseline and adaptive policies that account for topic size, screening depth, and residual uncertainty. We introduce effort-aware evaluation metrics to measure trade-offs between review effectiveness and resource use. Our results show that simple, topic-sensitive policies can significantly improve the yield of relevant documents discovered, offering practical insights for scalable and equitable systematic review workflows.
POLAR: Policy Optimization for Literature Analysis under Review Constraints
Di Nunzio G. M.
2025
Abstract
Systematic reviews are vital for evidence-based decision-making but remain resource-intensive due to the volume of literature requiring expert screening. Technology-Assisted Review (TAR) systems offer a solution by ranking documents for review, yet questions remain about how best to allocate limited human effort across multiple review topics. In this paper, we explore the problem of effort distribution by comparing alternative screening policies under fixed effort constraints. Using real-world data from the CLEF eHealth 2017-2019 TAR tasks, we evaluate both baseline and adaptive policies that account for topic size, screening depth, and residual uncertainty. We introduce effort-aware evaluation metrics to measure trade-offs between review effectiveness and resource use. Our results show that simple, topic-sensitive policies can significantly improve the yield of relevant documents discovered, offering practical insights for scalable and equitable systematic review workflows.| File | Dimensione | Formato | |
|---|---|---|---|
|
polar_cikm.pdf
accesso aperto
Tipologia:
Published (Publisher's Version of Record)
Licenza:
Creative commons
Dimensione
1.03 MB
Formato
Adobe PDF
|
1.03 MB | Adobe PDF | Visualizza/Apri |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.




