CLEF | Language Technologies Lab

CLEF

Fri, 27 Feb 2026 00:00:00 +0000

The CLEF conference includes many multilingual and multi-modal activity proposals. Examples: math question answering, prediction of mental health issues, text simplification of scientific topics, retrieval of arguments, fact checking.

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Mon, 01 Sep 2025 00:00:00 +0000

Resources

Abstract

This paper presents the eighth edition of the CheckThat! lab, part of the 2025 Conference and Labs of the Evaluation Forum (CLEF). As in previous editions of CheckThat!, the lab offers tasks from the core of the verification pipeline, including check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification as well as auxiliary tasks addressing different facets of individual steps of the pipeline: Task 1 is on identification of subjectivity (a follow-up of the CheckThat! 2024 edition), which is related to the check-worthiness task, Task 2 is on claim normalization, Task 3 addresses fact-checking numerical claims, and Task 4 focuses on scientific web discourse processing. These challenging classification and retrieval problems are offered in different mono-, multi- and crosslingual settings covering more than 20 languages. This year, CheckThat! was one of the most popular labs at CLEF-2025 in terms of team registrations: 177 teams registered, almost half of them actually participating (a total of 83 teams) and 54 submitted system description papers.

Citation

Firoj Alam, Julia Maria Struß, Tanmoy Chakraborty, Stefan Dietze, Salim Hafid, Katerina Korre, Arianna Muti, Preslav Nakov, Federico Ruggeri, Sebastian Schellhammer, et al. Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 199–223. Springer, 2025.

@inproceedings{alam-etal-2025-overview,
 title={Overview of the {CLEF}-2025 {C}heck{T}hat! {L}ab: {S}ubjectivity, fact-checking, claim normalization, and retrieval},
 author={Alam, Firoj and Stru{\ss}, Julia Maria and Chakraborty, Tanmoy and Dietze, Stefan and Hafid, Salim and Korre, Katerina and Muti, Arianna and Nakov, Preslav and Ruggeri, Federico and Schellhammer, Sebastian and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={199--223},
 year={2025},
 organization={Springer}
}

Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Sun, 01 Sep 2024 00:00:00 +0000

Resources

Abstract

We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than one-third of them (a total of 46) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tanmoy Chakraborty, Tamer Elsayed, Piotr Przybya, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, et al. Overview of the clef-2024 checkthat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 28–52. Springer, 2024.

@inproceedings{barron-etal-20240-overview,
 title={Overview of the CLEF-2024 CheckThat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness},
 author={Barr{\'o}n-Cede{\~n}o, Alberto and Alam, Firoj and Stru{\ss}, Julia Maria and Nakov, Preslav and Chakraborty, Tanmoy and Elsayed, Tamer and Przyby{\l}a, Piotr and Caselli, Tommaso and Da San Martino, Giovanni and Haouari, Fatima and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={28--52},
 year={2024},
 organization={Springer}
}

A Corpus for Sentence-Level Subjectivity Detection on English News Articles

Wed, 01 May 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Fri, 01 Sep 2023 00:00:00 +0000

Resources

Abstract

We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, verifying whether a claim was fact-checked before, supporting evidence retrieval, and claim verification. In this sixth edition, we zoom into some new problems and for the first time we offer five tasks in seven languages: Arabic, Dutch, English, German, Italian, Spanish, and Turkish. Task 1 asks to determine whether an item —text or text plus image— is check-worthy. Task 2 aims to predict whether a sentence from a news article is subjective or not. Task 3 asks to assess the political bias of the news at the article and at the media outlet level. Task 4 focuses on the factuality of reporting of news media. Finally, Task 5 looks at identifying authorities in Twitter that could help verify a given target claim. For a second year, CheckThat! was the most popular lab at CLEF-2023 in terms of team registrations: 127 teams. About one-third of them (a total of 37) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S. Cheema, Fatima Haouari, Maram Hasanain, Mücahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, and Wajdi Zaghouani. Overview of the CLEF-2023 checkthat! lab on checkworthiness, subjectivity, political bias, factuality, and authority of news articles and their source. In Avi Arampatzis, Evangelos Kanoulas, Theodora Tsikrika, Stefanos Vrochidis, Anastasia Giachanou, Dan Li, Mohammad Aliannejadi, Michalis Vlachos, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 14th International Conference of the CLEF Association, CLEF 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings, volume 14163 of Lecture Notes in Computer Science, pages 251–275. Springer, 2023.

@inproceedings{cedeno-etal-2023-clef-overview,
 author = {Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and
 Firoj Alam and
 Andrea Galassi and
 Giovanni Da San Martino and
 Preslav Nakov and
 Tamer Elsayed and
 Dilshod Azizov and
 Tommaso Caselli and
 Gullal S. Cheema and
 Fatima Haouari and
 Maram Hasanain and
 M{\"{u}}cahid Kutlu and
 Chengkai Li and
 Federico Ruggeri and
 Julia Maria Stru{\ss} and
 Wajdi Zaghouani},
 editor = {Avi Arampatzis and
 Evangelos Kanoulas and
 Theodora Tsikrika and
 Stefanos Vrochidis and
 Anastasia Giachanou and
 Dan Li and
 Mohammad Aliannejadi and
 Michalis Vlachos and
 Guglielmo Faggioli and
 Nicola Ferro},
 title = {Overview of the {CLEF-2023} CheckThat! Lab on Checkworthiness, Subjectivity,
 Political Bias, Factuality, and Authority of News Articles and Their
 Source},
 booktitle = {Experimental {IR} Meets Multilinguality, Multimodality, and Interaction
 - 14th International Conference of the {CLEF} Association, {CLEF}
 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings},
 series = {Lecture Notes in Computer Science},
 volume = {14163},
 pages = {251--275},
 publisher = {Springer},
 year = {2023},
 url = {https://doi.org/10.1007/978-3-031-42448-9\_20},
 doi = {10.1007/978-3-031-42448-9\_20}
}