Natural Language Processing | Language Technologies Lab

EquAl: Equitable Algorithms, Promoting Fairness and Countering Algorithmic Discrimination Through Norms and Technologies - Final Conference

Fri, 23 Jan 2026 00:00:00 +0000

Project info

The EquAl project addresses algorithmic evaluations, decisions, and predictions, to promote fairness and counter discrimination affecting individuals and groups. The research project fundedis by the EU Commission under the NextGenerationEU program and the Italian Ministry of Education, University and Research. (PRIN 2022. Ref. prot. n.: 2022KFLF3E-001 - CUP J53D23005560001).

Useful Links

PRIMA: PRivacy Infringements Machine-Advice - Final Conference

Mon, 12 Jan 2026 00:00:00 +0000

Project info

PRIMA (PRivacy Infringements Machine-Advice) studies the law and practice of privacy policies, develops methods and techniques for their automated analysis, and implements a prototype to assess their lawfulness. It deploys legal analytics—a mix of data science, artificial intelligence, machine learning, natural language processing and statistics—to detect and assess privacy policies’ infringements.

Useful Links

ALMA-AI | Workshop LLM: a debate on technical experiences

Thu, 20 Nov 2025 00:00:00 +0000

Workshop Abstract

La diffusione dei Large Language Model in ogni ambito delle attività umane ha reso necessario valutare lo stato dell’arte, i limiti, gli ostacoli e le barriere che questa potente tecnologia offre. I rapidi aggiornamenti di queste tecnologie impongono di allargare gli orizzonti degli obiettivi di ricerca e delle metodologie. Le capacità di reasoning degli Agentic AI indicano nuove sfide. Tuttavia, sempre più emerge la necessità di valutare in quali ambiti queste tecnologie possano produrre un reale valore aggiunto e quando invece, anche considerando l’uso delle risorse necessarie, sia meglio adottare approcci più tradizionali. Emerge quindi il tema di come valutare i modelli messi a confronto fra loro (benchmarking). Gli aspetti tecnici si intersecano con quelli etici, giuridici, di sostenibilità, di efficacia, nonché di spiegabilità dei passaggi di reasoning. Il workshop intende investigare questi temi, con particolare riguardo alle applicazioni nell’ambito del diritto dove il linguaggio è un pilastro costitutivo della disciplina. Il dibattito è utile a definire anche le aspettative future delle quali le istituzioni, come i Parlamenti e le pubbliche amministrazioni, potranno avvantaggiarsi senza tuttavia ignorare rischi e false illusioni.

Federico Ruggeri’s speech

The talk addresses three main aspects of LLMs and reasoning capabilities. First, we discuss what kind of reasoning type LLMs are tested for. The short answer is that in the majority of cases, it is unclear which reason type(s) is (are) considered. Second, we discuss to what extent do LLMs perform reasoning. Some view LLMs as stochastic parrots, while others believe they acquire true reasoning capabilities. Third, we show how reasoning and argumentation are tightly connected and discuss how argumentation is being progressively used as a way to assess reasoning capabilities in LLMs.

Useful Links

The 13th Workshop on Argument Mining and Reasoning Co-located with ACL 2026

Wed, 01 Oct 2025 00:00:00 +0000

Introduction

The 2026 edition of the ArgMining workshop therefore places a special focus on understanding and evaluating arguments in both human and machine reasoning. With this topic, we broaden the workshop’s focus to include reasoning, a long-standing area of research in AI that has recently gained renewed interest within the *ACL community, driven by the latest generation of LLMs. Reasoning is tightly connected to argumentation as it represents, analyzes and evaluates the process of reaching conclusions on the basis of available information. If we consider argumentation as a paradigm to capture reasoning, then machines (particularly LLMs) can be evaluated based on their ability to address argument mining tasks.

The workshop will be co-located with ACL 2026 and held in San Diego, United States in a hybrid format.

Useful Links

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Mon, 01 Sep 2025 00:00:00 +0000

Resources

Abstract

This paper presents the eighth edition of the CheckThat! lab, part of the 2025 Conference and Labs of the Evaluation Forum (CLEF). As in previous editions of CheckThat!, the lab offers tasks from the core of the verification pipeline, including check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification as well as auxiliary tasks addressing different facets of individual steps of the pipeline: Task 1 is on identification of subjectivity (a follow-up of the CheckThat! 2024 edition), which is related to the check-worthiness task, Task 2 is on claim normalization, Task 3 addresses fact-checking numerical claims, and Task 4 focuses on scientific web discourse processing. These challenging classification and retrieval problems are offered in different mono-, multi- and crosslingual settings covering more than 20 languages. This year, CheckThat! was one of the most popular labs at CLEF-2025 in terms of team registrations: 177 teams registered, almost half of them actually participating (a total of 83 teams) and 54 submitted system description papers.

Citation

Firoj Alam, Julia Maria Struß, Tanmoy Chakraborty, Stefan Dietze, Salim Hafid, Katerina Korre, Arianna Muti, Preslav Nakov, Federico Ruggeri, Sebastian Schellhammer, et al. Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 199–223. Springer, 2025.

@inproceedings{alam-etal-2025-overview,
 title={Overview of the {CLEF}-2025 {C}heck{T}hat! {L}ab: {S}ubjectivity, fact-checking, claim normalization, and retrieval},
 author={Alam, Firoj and Stru{\ss}, Julia Maria and Chakraborty, Tanmoy and Dietze, Stefan and Hafid, Salim and Korre, Katerina and Muti, Arianna and Nakov, Preslav and Ruggeri, Federico and Schellhammer, Sebastian and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={199--223},
 year={2025},
 organization={Springer}
}

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Tue, 01 Jul 2025 00:00:00 +0000

Multimodal Argumentative Fallacy Detection and Classification on Political Debates Shared Task.

Co-located with The 12th Workshop on Argument Mining in Vienna, Austria.

Overview

This shared task focuses on detecting and classifying fallacies in political debates by integrating text and audio data. Participants will tackle two sub-tasks:

Argumentative Fallacy Detection
Argumentative Fallacy Classification

We offer three input settings:

Text-only: Analyze textual arguments.
Audio-only: Explore paralinguistic features.
Text + Audio: Combine both for a multimodal perspective.

Join us to advance multimodal argument mining and uncover new insights into human reasoning! 💬

Tasks

Task A

Input: a sentence, in the form of text or audio or both, extracted from a political debate.
Task: to determine whether the input contains an argumentative fallacy.

Task B

Input: a sentence, in the form of text or audio or both, extracted from a political debate, containing a fallacy.
Task: to determine the type of fallacy contained in the input, according to the classification introduced by Goffredo et al. (2022). We only refer to macro categories.

For each sub-task, participants can leverage the debate context of a given input: all its previous sentences and corresponding aligned audio samples. For instance, consider the text-only input mode. Given a sentence from a political debate at index i, participants can use sentences with indexes from 0 to i - 1, where 0 denotes the first sentence in the debate.

Data

We use MM-USED-fallacy and release a version of the dataset specifically designed for argumentative fallacy detection. This dataset includes 1,278 sentences from Haddadan et al.’s (2019) dataset on US presidential elections. Each sentence is labeled with one of six argumentative fallacy categories, as introduced by Goffredo et al. (2022).

Inspired by observations from Goffredo et al. (2022) on the benefits of leveraging multiple argument mining tasks for fallacy detection and classification, we also provide additional datasets to encourage multi-task learning. A summary is provided in the table below:

Dataset	Description	Size
MM-USED-fallacy	A multimodal extension of USElecDeb60to20 dataset, covering US presidential debates (1960-2020). Inlcludes labels for argumentative fallacy detection and argumentative fallacy classification.	1,278 samples (updated version)
MM-USED	A multimodal extension of the USElecDeb60to16 dataset, covering US presidential debates (1960–2016). Includes labels for argumentative sentence detection and component classification.	23,505 sentences (updated version)
UKDebates	386 sentences and audio samples from the 2015 UK Prime Ministerial elections. Sentences are labeled for argumentative sentence detection: containing or not containing a claim.	386 sentences
M-Arg	A multimodal dataset for argumentative relation classification from the 2020 US Presidential elections. Sentences are labeled as attacking, supporting, or unrelated to another sentence.	4,104 pairs

All datasets will be available through MAMKit.

Since many multimodal datasets cannot release audio samples due to copyright restrictions, MAMKit provides an interface to dynamically build datasets and promote reproducible research.

Datasets are formatted as torch.Dataset objects, containing input values (text, audio, or both) and corresponding task-specific labels. More details about data formats and dataset building are available in MAMKit’s documentation. ## Retrieving the Data through MAMKit

To retrieve the datasets through MAMKit, you can use the following code interface:

from mamkit.data.datasets import MMUSEDFallacy, MMUSED, UKDebates, MArg, InputMode

import logging
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')

 # MM-USED-fallacy dataset
 mm_used_fallacy_loader = MMUSEDFallacy(
 task_name='afc', # Choose between 'afc' or 'afd' 
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # MM-USED dataset
 mm_used_loader = MMUSED(
 task_name='asd',#Choose between 'asd' or 'acc' 
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # UKDebates dataset
 uk_debates_loader = UKDebates(
 task_name='asd',
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # M-Arg dataset
 m_arg_loader = MArg(
 task_name='arc',
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

Each loader is initialized with the appropriate task name (afc for argumentative fallacy classification, asd for argumentative sentence detection, and ‘arc’ for argumentative relation classification), input mode (InputMode.TEXT_ONLY, InputMode.AUDIO_ONLY, or InputMode.TEXT_AUDIO), and the base data path.

Ensure that you have MAMKit installed and properly configured in your environment to use these loaders.

For more details, refer to the MAMKit GitHub repository and website .

Test Set Access 🔍

The test set for mm-argfallacy-2025 is now available! To use it, please:

Create a fresh environment
Clone the repository and install the requirements:

git clone git@github.com:nlp-unibo/mamkit.git
cd mamkit
pip install -r requirements.txt
pip install --editable .

Access MAMKit in your Python code:

import mamkit

Then, retrieve the data using the following code:

For Fallacy Classification (`afc`):

from mamkit.data.datasets import MMUSEDFallacy, InputMode
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')
 loader = MMUSEDFallacy(
 task_name='afc',
 input_mode=InputMode.TEXT_ONLY, # or TEXT_AUDIO or AUDIO_ONLY
 base_data_path=base_data_path
 )
 split_info = loader.get_splits('mm-argfallacy-2025')

For Fallacy Detection (`afd`):

from mamkit.data.datasets import MMUSEDFallacy, InputMode
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')
 loader = MMUSEDFallacy(
 task_name='afd',
 input_mode=InputMode.TEXT_ONLY, # or TEXT_AUDIO or AUDIO_ONLY
 base_data_path=base_data_path
 )
 split_info = loader.get_splits('mm-argfallacy-2025')

References

MM-USED-fallacy: Mancini et al. (2024). The version provided through MAMKit includes updated samples, with refinements in the alignment process. This results in a different number of samples compared to the original dataset.
MM-USED: Mancini et al. (2022). The version provided through MAMKit includes updated samples, with refinements in the alignment process. This results in a different number of samples compared to the original dataset.
UK-Debates: Lippi and Torroni (2016).
M-Arg: Mestre et al. (2021).

Note: By “updated version,” we mean that the datasets have undergone a refinement in the alignment process, which has resulted in adjustments to the number of samples included compared to the original versions published in the referenced papers.

Evaluation

For argumentative fallacy detection, we will compute the binary F1-score on predicted sentence-level labels.
For argumentative fallacy classification, we will compute the macro F1-score on predicted sentence-level labels.
Metrics will be computed on the hidden test set to determine the best system for each sub-task and input mode.

Evaluation will be performed via the CodaLab platform.
On CodaLab, participants will find the leaderboard, along with the results of the provided baselines.
Submission guidelines can be found under the Evaluation section of the CodaLab competition page.

🚨 Important: In the evaluation website, you will also find a link to a mandatory participation survey.
Filling out this survey is required in order to participate in the task.
We also provide the survey link here for convenience: https://tinyurl.com/limesurvey-argfallacy

Baseline Results on Test Set

Argumentative Fallacy Classification (AFC) – Macro F1-score

Input Modality	Model	F1-Score
Text-only	BiLSTM w/ GloVe	47.21
Text-only	RoBERTa	39.25
Audio-only	BiLSTM w/ MFCCs	15.82
Audio-only	WavLM	6.43
Text + Audio	BiLSTM (GloVe + MFCCs)	21.91
Text + Audio	MM-RoBERTa + WavLM	38.16

Argumentative Fallacy Detection (AFD) – Binary F1-score

Input Modality	Model	F1-Score
Text-only	BiLSTM w/ GloVe	24.62
Text-only	RoBERTa	27.70
Audio-only	BiLSTM w/ MFCCs	0.00
Audio-only	WavLM	0.00
Text + Audio	BiLSTM (GloVe + MFCCs)	23.37
Text + Audio	MM-RoBERTa + WavLM	28.48

Submission

All evaluated submissions are required to commit to submitting a system description paper. You can choose between two options:

Non-Archival Paper:
A 2-page paper describing your system, with unlimited pages for appendices and bibliography. These papers will not be published in the workshop proceedings, but your system will be mentioned in the Overview Paper of the shared task, upon acceptance.
Archival Paper:
A 4-page paper describing your system, also with unlimited pages for appendices and bibliography. These papers will be published in the official ACL workshop proceedings and must be presented at the workshop (poster or oral session).
⚠️ In accordance with ACL policy, at least one team member must register for the workshop in order to present an archival paper if aaccepted to be published at the ACL proceedings.

All papers must use the official ACL style templates, available in both LaTeX and Word. We strongly recommend using the official Overleaf template for convenience.

We have sent an email to each team with all the details regarding the system description paper submission for MM-ArgFallacy2025. Please check your inbox (and spam folder just in case).

🗓️ Submissions open: May 1st, 2025 (the day after the end of the evaluation period)
🗓️ Submissions close: May 15th, 2025
📢 Notification of acceptance: May 20th, 2025
📝 Camera-ready deadline: May 25th, 2025

Important notes:

All accepted archival papers will be presented during the workshop’s poster session and require at least one registered author.
Non-archival papers do not require registration and are not presented at the workshop, but their systems will be acknowledged in the Overview Paper.

We look forward to receiving your submissions!

🏆 Leaderboard – Shared Task Results

`AFC Task – Argumentative Fallacy Classification`

📝 Text-only

Rank	Team	F1-Macro
1	Team NUST	0.4856
2	Baseline BiLSTM	0.4721
3	alessiopittiglio	0.4444
4	Baseline RoBERTa	0.3925
5	Team CASS	0.1432

🔊 Audio-only

Rank	Team	F1-Macro
1	alessiopittiglio	0.3559
2	Team NUST	0.1588
3	Baseline BiLSTM + MFCCs	0.1582
4	Team CASS	0.0864
5	Baseline WavLM	0.0643

🔁 Text-Audio

Rank	Team	F1-Macro
1	Team NUST	0.4611
2	alessiopittiglio	0.4403
3	Baseline RoBERTa + WavLM	0.3816
4	Baseline BiLSTM + MFCCs	0.2191
5	Team CASS	0.1432

`AFD Task – Argumentative Fallacy Detection`

📝 Text-only

Rank	Team	F1-Macro
1	Baseline RoBERTa	0.2770
2	Ambali_Yashovardhan	0.2534
3	Baseline BiLSTM	0.2462
4	Team EvaAdriana	0.2195

🔊 Audio-only

Rank	Team	F1-Macro
1	Ambali_Yashovardhan	0.2095
2	Team EvaAdriana	0.1690
3	Baseline BiLSTM + MFCCs	0.0000
4	Baseline WavLM	0.0000

🔁 Text-Audio

Rank	Team	F1-Macro
1	Baseline RoBERTa + WavLM	0.2848
2	Baseline BiLSTM + MFCCs	0.2337
3	Ambali_Yashovardhan	0.2244
4	Team EvaAdriana	0.1931

Key Dates (Anywhere on Earth)

Release of Training Data: February 25th
Release of Test Set: ~~March 24th~~ → April 7th
Evaluation Start: ~~April 14th~~ → April 21st
Evaluation End: ~~April 25th~~ → April 30th
Paper Submissions Open: May 1st
Paper Submission Close: May 15th
Notification of acceptance: May 20th
Camera-ready Due: May 25th
Workshop: July 31st

Task Organizers

	Eleonora Mancini Language Technologies Lab, University of Bologna, Italy		Federico Ruggeri Language Technologies Lab, University of Bologna, Italy
	Serena Villata Inria-I3S WIMMICS Laboratoire I3S, CNRS, Sophia Antipolis, France		Paolo Torroni Language Technologies Lab, University of Bologna, Italy

Contacts

Join the MM-ArgFallacy2025 Slack Channel!

Cite

Eleonora Mancini, Federico Ruggeri, Serena Villata, and Paolo Torroni. 2025. Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates. In Proceedings of the 12th Argument mining Workshop, pages 358–368, Vienna, Austria. Association for Computational Linguistics.

@inproceedings{mancini-etal-2025-overview,
 title = "Overview of {MM}-{A}rg{F}allacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates",
 author = "Mancini, Eleonora and
 Ruggeri, Federico and
 Villata, Serena and
 Torroni, Paolo",
 editor = "Chistova, Elena and
 Cimiano, Philipp and
 Haddadan, Shohreh and
 Lapesa, Gabriella and
 Ruiz-Dolz, Ramon",
 booktitle = "Proceedings of the 12th Argument mining Workshop",
 month = jul,
 year = "2025",
 address = "Vienna, Austria",
 publisher = "Association for Computational Linguistics",
 url = "https://aclanthology.org/2025.argmining-1.35/",
 doi = "10.18653/v1/2025.argmining-1.35",
 pages = "358--368",
 ISBN = "979-8-89176-258-9",
 abstract = "We present an overview of the MM-ArgFallacy2025 shared task on Multimodal Argumentative Fallacy Detection and Classification in Political Debates, co-located with the 12th Workshop on Argument Mining at ACL 2025. The task focuses on identifying and classifying argumentative fallacies across three input modes: text-only, audio-only, and multimodal (text+audio), offering both binary detection (AFD) and multi-class classification (AFC) subtasks. The dataset comprises 18,925 instances for AFD and 3,388 instances for AFC, from the MM-USED-Fallacy corpus on U.S. presidential debates, annotated for six fallacy types: Ad Hominem, Appeal to Authority, Appeal to Emotion, False Cause, Slippery Slope, and Slogan. A total of 5 teams participated: 3 on classification and 2 on detection. Participants employed transformer-based models, particularly RoBERTa variants, with strategies including prompt-guided data augmentation, context integration, specialised loss functions, and various fusion techniques. Audio processing ranged from MFCC features to state-of-the-art speech models. Results demonstrated textual modality dominance, with best text-only performance reaching 0.4856 F1-score for classification and 0.34 for detection. Audio-only approaches underperformed relative to text but showed improvements over previous work, while multimodal fusion showed limited improvements. This task establishes important baselines for multimodal fallacy analysis in political discourse, contributing to computational argumentation and misinformation detection capabilities."
}

Credits

This shared task is partially supported by the project European Commission's NextGeneration EU programme, PNRR -- M4C2 -- Investimento 1.3, Partenariato Esteso, PE00000013 - FAIR - Future Artificial Intelligence Research'' -- Spoke 8 Pervasive AI’’.

Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Sun, 01 Sep 2024 00:00:00 +0000

Resources

Abstract

We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than one-third of them (a total of 46) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tanmoy Chakraborty, Tamer Elsayed, Piotr Przybya, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, et al. Overview of the clef-2024 checkthat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 28–52. Springer, 2024.

@inproceedings{barron-etal-20240-overview,
 title={Overview of the CLEF-2024 CheckThat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness},
 author={Barr{\'o}n-Cede{\~n}o, Alberto and Alam, Firoj and Stru{\ss}, Julia Maria and Nakov, Preslav and Chakraborty, Tanmoy and Elsayed, Tamer and Przyby{\l}a, Piotr and Caselli, Tommaso and Da San Martino, Giovanni and Haouari, Fatima and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={28--52},
 year={2024},
 organization={Springer}
}

Disruptive situation detection on public transport through speech emotion recognition

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Fri, 01 Sep 2023 00:00:00 +0000

Resources

Abstract

We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, verifying whether a claim was fact-checked before, supporting evidence retrieval, and claim verification. In this sixth edition, we zoom into some new problems and for the first time we offer five tasks in seven languages: Arabic, Dutch, English, German, Italian, Spanish, and Turkish. Task 1 asks to determine whether an item —text or text plus image— is check-worthy. Task 2 aims to predict whether a sentence from a news article is subjective or not. Task 3 asks to assess the political bias of the news at the article and at the media outlet level. Task 4 focuses on the factuality of reporting of news media. Finally, Task 5 looks at identifying authorities in Twitter that could help verify a given target claim. For a second year, CheckThat! was the most popular lab at CLEF-2023 in terms of team registrations: 127 teams. About one-third of them (a total of 37) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S. Cheema, Fatima Haouari, Maram Hasanain, Mücahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, and Wajdi Zaghouani. Overview of the CLEF-2023 checkthat! lab on checkworthiness, subjectivity, political bias, factuality, and authority of news articles and their source. In Avi Arampatzis, Evangelos Kanoulas, Theodora Tsikrika, Stefanos Vrochidis, Anastasia Giachanou, Dan Li, Mohammad Aliannejadi, Michalis Vlachos, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 14th International Conference of the CLEF Association, CLEF 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings, volume 14163 of Lecture Notes in Computer Science, pages 251–275. Springer, 2023.

@inproceedings{cedeno-etal-2023-clef-overview,
 author = {Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and
 Firoj Alam and
 Andrea Galassi and
 Giovanni Da San Martino and
 Preslav Nakov and
 Tamer Elsayed and
 Dilshod Azizov and
 Tommaso Caselli and
 Gullal S. Cheema and
 Fatima Haouari and
 Maram Hasanain and
 M{\"{u}}cahid Kutlu and
 Chengkai Li and
 Federico Ruggeri and
 Julia Maria Stru{\ss} and
 Wajdi Zaghouani},
 editor = {Avi Arampatzis and
 Evangelos Kanoulas and
 Theodora Tsikrika and
 Stefanos Vrochidis and
 Anastasia Giachanou and
 Dan Li and
 Mohammad Aliannejadi and
 Michalis Vlachos and
 Guglielmo Faggioli and
 Nicola Ferro},
 title = {Overview of the {CLEF-2023} CheckThat! Lab on Checkworthiness, Subjectivity,
 Political Bias, Factuality, and Authority of News Articles and Their
 Source},
 booktitle = {Experimental {IR} Meets Multilinguality, Multimodality, and Interaction
 - 14th International Conference of the {CLEF} Association, {CLEF}
 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings},
 series = {Lecture Notes in Computer Science},
 volume = {14163},
 pages = {251--275},
 publisher = {Springer},
 year = {2023},
 url = {https://doi.org/10.1007/978-3-031-42448-9\_20},
 doi = {10.1007/978-3-031-42448-9\_20}
}

Natural Language Processing | Language Technologies Lab

EquAl: Equitable Algorithms, Promoting Fairness and Countering Algorithmic Discrimination Through Norms and Technologies - Final Conference

Project info

Useful Links

PRIMA: PRivacy Infringements Machine-Advice - Final Conference

Project info

Useful Links

ALMA-AI | Workshop LLM: a debate on technical experiences

Workshop Abstract

Federico Ruggeri’s speech

Useful Links

The 13th Workshop on Argument Mining and Reasoning Co-located with ACL 2026

Introduction

Useful Links

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Resources

Abstract

Citation

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Overview

Tasks

Data

Test Set Access 🔍

For Fallacy Classification (afc):

For Fallacy Detection (afd):

References

Evaluation

Baseline Results on Test Set

Argumentative Fallacy Classification (AFC) – Macro F1-score

Argumentative Fallacy Detection (AFD) – Binary F1-score

Submission

🏆 Leaderboard – Shared Task Results

AFC Task – Argumentative Fallacy Classification

📝 Text-only

🔊 Audio-only

🔁 Text-Audio

AFD Task – Argumentative Fallacy Detection

📝 Text-only

🔊 Audio-only

🔁 Text-Audio

Key Dates (Anywhere on Earth)

Task Organizers

Eleonora Mancini

Federico Ruggeri

Serena Villata

Paolo Torroni

Contacts

Cite

Credits

Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Resources

Abstract

Citation

Disruptive situation detection on public transport through speech emotion recognition

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Resources

Abstract

Citation

For Fallacy Classification (`afc`):

For Fallacy Detection (`afd`):

`AFC Task – Argumentative Fallacy Classification`

`AFD Task – Argumentative Fallacy Detection`