Abstract
We introduce Guideline-Centered Annotation Methodology (GCAM), a novel data methodology designed to report the annotation guidelines associated with each data instance. GCAM addresses four key limitations of the standard application of the prescriptive annotation methodology by reducing the information loss during annotation, ensuring adherence to guidelines, and enabling the efficient reuse of annotated data across multiple tasks that rely on the same guidelines. We evaluate GCAM with a focus on text classification tasks through (i) a human annotation study and (ii) an experimental evaluation with several machine learning models, guaranteeing a transparent evaluation of the successful application of the prescriptive paradigm and enabling a fine-grained model error analysis.
Federico Ruggeri
Postdoctoral Research Fellow
His research aims to devise Natural Language Processing (NLP) systems that learn to generate, distill, and use knowledge from unstructured text.
Paolo Torroni
Associate Professor
Head of the Language Technologies lab. His main research focus is in artificial intelligence, and in particular natural language processing, multi-agent systems, and computational logics.