Language Technologies Lab

Argument Mining

Mon, 02 Mar 2026 00:00:00 +0000

Argument Mining on Legal Datasets

Mon, 02 Mar 2026 00:00:00 +0000

Description:
Argumentation in legal documents is typically well-structured and follows specific domain rules. We are interested in applying both the most recent NLP techniques and also hybrid techniques that can leverage the domain knowledge.

Contact: Andrea Galassi, Marco Lippi

Hate Speech Detection with Argumentative Reasoning

Mon, 02 Mar 2026 00:00:00 +0000

Description:
Hate speech often lies on implicit content and subtle reasoning nuances. Our idea is to apply argumentative reasoning to hate speech to make implicit content explicit in order to define more interpretable and user-friendly hate speech detection systems.

Contact: Federico Ruggeri, Arianna Muti

References:

Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts
Arianna Muti, Federico Ruggeri, Khalid Al-Khatib, Alberto Barrón-Cedeño, Tommaso Caselli
Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 21091–21107, 2024
DOI | PDF

PejorativITy: Disambiguating Pejorative Epithets to Improve Misogyny Detection in Italian Tweets
Arianna Muti, Federico Ruggeri, Cagri Toraman, Lorenzo Musetti, Samuel Algherini, Silvia Ronchi, Gianmarco Saretto, Caterina Zapparoli, Alberto Barrón-Cedeño.
In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 12700–12711, Torino, Italia. ELRA and ICCL.
PDF | Anthology

Interpretability

Mon, 02 Mar 2026 00:00:00 +0000

Knowledge Extraction from Rationalization

Mon, 02 Mar 2026 00:00:00 +0000

Description:
Rationalization is a type of example-specific explanation. However, samples belonging to the same class might share similar rationales. The idea is to define ways to go from a local explanation (i.e., rationalization) to a global explanation (i.e., knowledge base) by aggregating and summarizing extracted rationales. This can be done with LLMs (e.g., prompting techniques) or other solutions.

Contact: Federico Ruggeri

References:

A Game Theoretic Approach to Class-wise Selective Rationalization
Shiyu Chang, Yang Zhang, Mo Yu, Tommi S. Jaakkola. 33rd Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, 2019.
PDF

Legal Analytics

Mon, 02 Mar 2026 00:00:00 +0000

Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology

Mon, 02 Mar 2026 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Mixture of Experts for Rationalization

Mon, 02 Mar 2026 00:00:00 +0000

Description:
Mixture of Experts (MoE) is a technique whereby several models are trained on the same data, each specializing in a certain subset. MoE have been shown to be successful in a variety of applications and their original formulation dates back early 2000s. The idea is to understand whether we can develop a MoE model for selective rationalization to address interlocking.

Contact: Federico Ruggeri

References:

A Survey on Mixture of Experts in Large Language Models
W. Cai, J. Jiang, F. Wang, J. Tang, S. Kim and J. Huang.
In IEEE Transactions on Knowledge and Data Engineering, vol. 37, no. 7, pp. 3896-3915, July 2025.
DOI

Multi-cultural Abusive and Hate Speech Detection

Mon, 02 Mar 2026 00:00:00 +0000

Description:
What is attributable as abusive or hate speech depends on the given socio-cultural context. The same text might be reputed offensive by a certain culture, allowed by another, and, in the most extreme case, legally prosecutable by a third one. Our aim is to evaluate how machine learning model are affected by different definitions of abusive and hate speech to promote awareness in developing accurate abusive speech detection systems.

Contact: Federico Ruggeri, Katerina Korre, Arianna Muti

References:

Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains.
Katerina Korre, Arianna Muti, Federico Ruggeri, and Alberto Barrón-Cedeño. 2025.
In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3184–3198, Albuquerque, New Mexico. Association for Computational Linguistics.
DOI | PDF

Multimodal Argument Mining

Mon, 02 Mar 2026 00:00:00 +0000

Description:
Make use of speech information (e.g. prosody) to enhance the set of features that can be used to detect arguments. Speech can either be represented by means of ad-hoc feature extraction methods (e.g. MFCC) or via end-to-end architectures. Few existing corpora both offer argument annotation layers and speech data regarding a given text document.

Contact: Eleonora Mancini, Federico Ruggeri

References:

MAMKit: A Comprehensive Multimodal Argument Mining Toolkit.
Eleonora Mancini, Federico Ruggeri, Stefano Colamonaco, Andrea Zecca, Samuele Marro, and Paolo Torroni. 2024.
In Proceedings of the 11th Workshop on Argument Mining (ArgMining 2024), pages 69–82, Bangkok, Thailand. Association for Computational Linguistics.
DOI | PDF

Multimodal Fallacy Classification in Political Debates
Eleonora Mancini, Federico Ruggeri, Paolo Torroni
18th Conference of the European Chapter of the Association for Computational Linguistics (EACL), pp. 170–178, 2024
DOI | PDF

Multimodal Argument Mining: A Case Study in Political Debates
Eleonora Mancini, Federico Ruggeri, Andrea Galassi, and Paolo Torroni.
In Proceedings of the 9th Workshop on Argument Mining, pages 158–170, Online and in Gyeongju, Republic of Korea. International Conference on Computational Linguistics, 2022.
PDF | Anthology

Paper accepted at TACL!

Mon, 02 Mar 2026 00:00:00 +0000

Description

Our paper ‘Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology’ has been accepted to Transactions of the Association for Computational Linguistics (TACL)!

Proceedings and the updated paper will be available soon!

ArXiv

Rationalization via LLMs

Mon, 02 Mar 2026 00:00:00 +0000

Description:
LLMs are ubiquitous in NLP. Our aim is to evaluate LLM capabilities in performing selective rationalization via prompting. How do they compare w.r.t. traditional SPP models?

Contact: Federico Ruggeri

References:

Towards Faithful Explanations: Boosting Rationalization with Shortcuts Discovery
Linan Yue, Qi Liu, Yichao Du, Li Wang, Weibo Gao, Yanqing An.
The Twelfth International Conference on Learning Representations, 2024.
PDF

Learning Robust Rationales for Model Explainability: A Guidance-Based Approach
S Hu, K Yu.
Proceedings of the AAAI Conference on Artificial Intelligence, 2024.
DOI | PDF

Structured Rationalization via Tree kernel methods

Mon, 02 Mar 2026 00:00:00 +0000

Description:
There are several techniques for transforming text into abstract structured content (AMR graphs, Parse trees, etc…). We are interested in applying rationalization in these contexts by also enforcing some structural constraints depending on the given scenario of application. The constraints describe which type of allowed structured the rationalization system can extract. In the case of tree kernels, these structures are different types of trees.

Contact: Federico Ruggeri

References:

Tree-constrained Graph Neural Networks for Argument Mining
Federico Ruggeri, Marco Lippi, Paolo Torroni
September 2021
PDF

Text Classification with Guidelines Only

Mon, 02 Mar 2026 00:00:00 +0000

Description:
The standard approach for training a machine learning model on a task is to provide an annotated dataset $(\mathcal{X}, \mathcal{Y})$. The dataset is built by providing unlabeled data $\mathcal{X}$ to a group of annotators previously trained on a set of annotation guidelines $\mathcal{G}$. Annotators label data $\mathcal{X}$ via a given class set $\mathcal{C}$. The main issue of this approach is that annotators define the mapping from data $\mathcal{X}$ to the class set $\mathcal{C}$ via the guidelines $\mathcal{G}$, while machine learning models are trained to learn the same mapping without guidelines $\mathcal{G}$. Consequently, these models can learn any kind of mapping from $\mathcal{X}$ to $\mathcal{C}$ that better fits given data. Our idea is to directly provide guidelines $\mathcal{G}$ to models without any access to class labels during training.

Contact: Federico Ruggeri

References:

Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology
Federico Ruggeri, Eleonora Misino, Arianna Muti, Katerina Korre, Paolo Torroni, Alberto Barrón-Cedeño
September 2024
PDF

Tour

Mon, 02 Mar 2026 00:00:00 +0000

Transformers and LLMs for the detection and classification of unfair clauses

Mon, 02 Mar 2026 00:00:00 +0000

Description:
For several years, we have been working on tools for the automatic detection of unfair clauses in Terms of Services and Privacy Policies documents in the English language (see CLAUDETTE and PRIMA Projects page). We have already conducted several studies on this topic, and we are interested in applying new effective methods and techniques. Right now, we are focused on LLMs, but we are also interested in alternative techniques.

Contact: Andrea Galassi, Marco Lippi

Unstructured Knowledge Integration

Mon, 02 Mar 2026 00:00:00 +0000

AMELR

Fri, 27 Feb 2026 00:00:00 +0000

The AMELR workshop focuses on Legal Argument Mining (LAM) - using NLP to automatically detect legal arguments. Recent developments in NLP and LAM have provided legal scholars with a powerful tool for studying reasoning patterns, interpretative theories, and biases across jurisdictions and legal systems. The workshop gathers experts in computer science, AI & Law, legal theory, and empirical legal studies to address key challenges of LAM: creating training datasets, developing reliable models, establishing reproducibility standards, and integrating LAM into legal research. The workshop aims to strengthen the emerging field of LAM and its role in empirical legal studies by sharing latest implementations, addressing core challenges, and establishing best practices.

ArgMining

Fri, 27 Feb 2026 00:00:00 +0000

Argument mining (also known as “argumentation mining”) is a well-established research area in computational linguistics that focuses on the automatic identification of argumentative structures, such as premises, conclusions, and inference schemes. Since its beginnings, the focus has been on the development of large-scale argumentation dataset and tasks like argument quality assessment, argument persuasiveness, and the synthesis of argumentative texts, spanning various domains, such as legal, social, medical, political, and scientific settings.

Argument Mining

Fri, 27 Feb 2026 00:00:00 +0000

Argumentative Fallacies

The detection and classification of argumentative fallacies. Fallacy constitute an important aspect of argumentation because they require substantial reasoning capabilities to be spotted, thus, representing a valuable challenge for machine learning models.

Multimodality

We have worked in evaluating the combination of audio and text modalities. Does audio modality provide any benefit in addressing argument mining tasks?

Dialogues

We have developed a benchmark for argumentative dialogues on scientific papers. Nonetheless, nowadays, the focus is more on developing commercial tools rather than on developing benchmarks.

Interpretability

The development of interpretable models for argumentation. A current focus is on extracting local explanations from texts to acquire insights about data. For instance, which kind of patterns are associated with certain argumentative components?

Reasoning in LLMs

LLMs and reasoning are hot topics that are currently vastly being investigated. Though, there are still few attempts that aim to use argumentation as a resource for assessing reasoning in LLMs.

ASAIL

Fri, 27 Feb 2026 00:00:00 +0000

The ASAIL workshop series and interest group serves as a platform for researchers and practitioners working on natural language processing of legal text. Its goals include (i) Organising regular peer-reviewed workshop events for presentation and discussion of research and practical implementations around legal NLP; (ii) Facilitating communication and collaboration among academic researchers as well as practitioners from industry, government, and the public sector, and other interested individuals and organisations; (iii) Providing an entry point into the research field and community.

CLEF

Fri, 27 Feb 2026 00:00:00 +0000

The CLEF conference includes many multilingual and multi-modal activity proposals. Examples: math question answering, prediction of mental health issues, text simplification of scientific topics, retrieval of arguments, fact checking.

CLIC-it

Fri, 27 Feb 2026 00:00:00 +0000

The spirit of the conference is inclusive. Recognizing the multifaceted nature of language phenomena and the need for interdisciplinary expertise, CLiC-it aims to bring together researchers from different fields including Computational Linguistics and Natural Language Processing, Linguistics, Cognitive Science, Machine Learning, Computer Science, Knowledge Representation, Information Retrieval, and Digital Humanities. CLiC-it welcomes contributions focusing on all languages, with a particular emphasis on Italian.

EVALITA

Fri, 27 Feb 2026 00:00:00 +0000

Suite of tasks in Italian language. Examples: hate speech detection, sentiment analysis, identification of memes, “la ghigliottina” game.

Legal Analytics

Fri, 27 Feb 2026 00:00:00 +0000

Argumentation and Argument Schema

We have defined legal arguments and developed benchmarks for training machine learning in automatically identifying them. Legal arguments are relevant to several legal tasks (e.g., judgement prediction) as they encode important decisions and opinions that are tailored to a given problem. Moreover, legal experts are interested in assessing if there are some recurring patterns concerning legal arguments. For instance, do similar documents convey similar legal arguments? Can we compare documents based on their arguments?

Judgement Prediction

The automatic prediction of the judge’s decision. We have developed benchmarks and assessed standard and transformer-based models on this task.

Interpretability

We have explored memory-augmented neural networks and decision trees to define more interpretable models, encouraging user trustworthiness based on explanations in addition to model efficiency.

Unfair Clause Detection

The automatic identification and classification of unfair clauses. We have developed benchmarks on this topic.

Summarization

Can machine learning models summarize legal documents based on certain guidelines? Are the developed summaries useful to legal experts?

Cross-linguality

We have developed some methods to project labels from similar documents written in different languages.

Information Retrieval

The automatic retrieval of legal documents and knowledge based on a similarity metric. This also link to the argumentation topic if we use arguments (either quantitatively or qualitatively) to compare and rank retrieved documents.

LUHME

Fri, 27 Feb 2026 00:00:00 +0000

The “Language Understanding in the Human-Machine Era” (LUHME) workshop aims to reignite, retrieve, resume, and refocus the enduring debate about the role of understanding in natural language use and related applications. Specifically, it seeks to elucidate the nature of language understanding and ascertain whether it is indispensable for computational natural language tasks such as automated translation and natural language generation. Furthermore, it aims to provide insight into the role played by language professionals (e.g., linguists, professional translators, interpreters, language educators) in computational natural language understanding. It will, therefore, convene researchers interested in the intersection of language understanding and the effective use of language technologies in human-machine interaction.

SemEval

Fri, 27 Feb 2026 00:00:00 +0000

SemEval is a series of international natural language processing (NLP) research workshops whose mission is to advance the current state of the art in semantic analysis and to help create high-quality annotated datasets in a range of increasingly challenging problems in natural language semantics. Each year’s workshop features a collection of shared tasks in which computational semantic analysis systems designed by different teams are presented and compared.

Speech Processing

Fri, 27 Feb 2026 00:00:00 +0000

Argumentation

We have investigated argumentation task using audio features. We have also developed a toolkit to foster research on this topic.

Clinical domain

We have defined a taxonomy of best practices for curating and maintaining clinical datasets for audio modality. We are currently evaluating interpretable speech techniques on clinical data (e.g., depression).

Interpretability

The development of interpretable audio features to address downstream task. An example is audio tokens, a discretization process to better analyze audio inputs.

Multimodality

We have explored text and audio modalities to assess their overall and individual contribution. We have mainly target argumentation tasks for now.

EquAl: Equitable Algorithms, Promoting Fairness and Countering Algorithmic Discrimination Through Norms and Technologies - Final Conference

Fri, 23 Jan 2026 00:00:00 +0000

Project info

Useful Links

PRIMA: PRivacy Infringements Machine-Advice - Final Conference

Mon, 12 Jan 2026 00:00:00 +0000

Project info

PRIMA (PRivacy Infringements Machine-Advice) studies the law and practice of privacy policies, develops methods and techniques for their automated analysis, and implements a prototype to assess their lawfulness. It deploys legal analytics—a mix of data science, artificial intelligence, machine learning, natural language processing and statistics—to detect and assess privacy policies’ infringements.

Useful Links

Leveraging Whisper Embeddings for Audio-based Lyrics Matching

Thu, 01 Jan 2026 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

ALMA-AI | Workshop LLM: a debate on technical experiences

Thu, 20 Nov 2025 00:00:00 +0000

Workshop Abstract

La diffusione dei Large Language Model in ogni ambito delle attività umane ha reso necessario valutare lo stato dell’arte, i limiti, gli ostacoli e le barriere che questa potente tecnologia offre. I rapidi aggiornamenti di queste tecnologie impongono di allargare gli orizzonti degli obiettivi di ricerca e delle metodologie. Le capacità di reasoning degli Agentic AI indicano nuove sfide. Tuttavia, sempre più emerge la necessità di valutare in quali ambiti queste tecnologie possano produrre un reale valore aggiunto e quando invece, anche considerando l’uso delle risorse necessarie, sia meglio adottare approcci più tradizionali. Emerge quindi il tema di come valutare i modelli messi a confronto fra loro (benchmarking). Gli aspetti tecnici si intersecano con quelli etici, giuridici, di sostenibilità, di efficacia, nonché di spiegabilità dei passaggi di reasoning. Il workshop intende investigare questi temi, con particolare riguardo alle applicazioni nell’ambito del diritto dove il linguaggio è un pilastro costitutivo della disciplina. Il dibattito è utile a definire anche le aspettative future delle quali le istituzioni, come i Parlamenti e le pubbliche amministrazioni, potranno avvantaggiarsi senza tuttavia ignorare rischi e false illusioni.

Federico Ruggeri’s speech

The talk addresses three main aspects of LLMs and reasoning capabilities. First, we discuss what kind of reasoning type LLMs are tested for. The short answer is that in the majority of cases, it is unclear which reason type(s) is (are) considered. Second, we discuss to what extent do LLMs perform reasoning. Some view LLMs as stochastic parrots, while others believe they acquire true reasoning capabilities. Third, we show how reasoning and argumentation are tightly connected and discuss how argumentation is being progressively used as a way to assess reasoning capabilities in LLMs.

Useful Links

Do Large Language Models understand how to be judges?

Wed, 01 Oct 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

The 13th Workshop on Argument Mining and Reasoning Co-located with ACL 2026

Wed, 01 Oct 2025 00:00:00 +0000

Introduction

The 2026 edition of the ArgMining workshop therefore places a special focus on understanding and evaluating arguments in both human and machine reasoning. With this topic, we broaden the workshop’s focus to include reasoning, a long-standing area of research in AI that has recently gained renewed interest within the *ACL community, driven by the latest generation of LLMs. Reasoning is tightly connected to argumentation as it represents, analyzes and evaluates the process of reaching conclusions on the basis of available information. If we consider argumentation as a paradigm to capture reasoning, then machines (particularly LLMs) can be evaluated based on their ability to address argument mining tasks.

The workshop will be co-located with ACL 2026 and held in San Diego, United States in a hybrid format.

Useful Links

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Mon, 01 Sep 2025 00:00:00 +0000

Resources

Abstract

This paper presents the eighth edition of the CheckThat! lab, part of the 2025 Conference and Labs of the Evaluation Forum (CLEF). As in previous editions of CheckThat!, the lab offers tasks from the core of the verification pipeline, including check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification as well as auxiliary tasks addressing different facets of individual steps of the pipeline: Task 1 is on identification of subjectivity (a follow-up of the CheckThat! 2024 edition), which is related to the check-worthiness task, Task 2 is on claim normalization, Task 3 addresses fact-checking numerical claims, and Task 4 focuses on scientific web discourse processing. These challenging classification and retrieval problems are offered in different mono-, multi- and crosslingual settings covering more than 20 languages. This year, CheckThat! was one of the most popular labs at CLEF-2025 in terms of team registrations: 177 teams registered, almost half of them actually participating (a total of 83 teams) and 54 submitted system description papers.

Citation

Firoj Alam, Julia Maria Struß, Tanmoy Chakraborty, Stefan Dietze, Salim Hafid, Katerina Korre, Arianna Muti, Preslav Nakov, Federico Ruggeri, Sebastian Schellhammer, et al. Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 199–223. Springer, 2025.

@inproceedings{alam-etal-2025-overview,
 title={Overview of the {CLEF}-2025 {C}heck{T}hat! {L}ab: {S}ubjectivity, fact-checking, claim normalization, and retrieval},
 author={Alam, Firoj and Stru{\ss}, Julia Maria and Chakraborty, Tanmoy and Dietze, Stefan and Hafid, Salim and Korre, Katerina and Muti, Arianna and Nakov, Preslav and Ruggeri, Federico and Schellhammer, Sebastian and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={199--223},
 year={2025},
 organization={Springer}
}

Sustainable Italian LLM Evaluation: Community Perspectives and Methodological Guidelines

Mon, 01 Sep 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification

Tue, 01 Jul 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Interlocking-free Selective Rationalization Through Genetic-based Learning

Tue, 01 Jul 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Tue, 01 Jul 2025 00:00:00 +0000

Multimodal Argumentative Fallacy Detection and Classification on Political Debates Shared Task.

Co-located with The 12th Workshop on Argument Mining in Vienna, Austria.

Overview

This shared task focuses on detecting and classifying fallacies in political debates by integrating text and audio data. Participants will tackle two sub-tasks:

Argumentative Fallacy Detection
Argumentative Fallacy Classification

We offer three input settings:

Text-only: Analyze textual arguments.
Audio-only: Explore paralinguistic features.
Text + Audio: Combine both for a multimodal perspective.

Join us to advance multimodal argument mining and uncover new insights into human reasoning! 💬

Tasks

Task A

Input: a sentence, in the form of text or audio or both, extracted from a political debate.
Task: to determine whether the input contains an argumentative fallacy.

Task B

Input: a sentence, in the form of text or audio or both, extracted from a political debate, containing a fallacy.
Task: to determine the type of fallacy contained in the input, according to the classification introduced by Goffredo et al. (2022). We only refer to macro categories.

For each sub-task, participants can leverage the debate context of a given input: all its previous sentences and corresponding aligned audio samples. For instance, consider the text-only input mode. Given a sentence from a political debate at index i, participants can use sentences with indexes from 0 to i - 1, where 0 denotes the first sentence in the debate.

Data

We use MM-USED-fallacy and release a version of the dataset specifically designed for argumentative fallacy detection. This dataset includes 1,278 sentences from Haddadan et al.’s (2019) dataset on US presidential elections. Each sentence is labeled with one of six argumentative fallacy categories, as introduced by Goffredo et al. (2022).

Inspired by observations from Goffredo et al. (2022) on the benefits of leveraging multiple argument mining tasks for fallacy detection and classification, we also provide additional datasets to encourage multi-task learning. A summary is provided in the table below:

Dataset	Description	Size
MM-USED-fallacy	A multimodal extension of USElecDeb60to20 dataset, covering US presidential debates (1960-2020). Inlcludes labels for argumentative fallacy detection and argumentative fallacy classification.	1,278 samples (updated version)
MM-USED	A multimodal extension of the USElecDeb60to16 dataset, covering US presidential debates (1960–2016). Includes labels for argumentative sentence detection and component classification.	23,505 sentences (updated version)
UKDebates	386 sentences and audio samples from the 2015 UK Prime Ministerial elections. Sentences are labeled for argumentative sentence detection: containing or not containing a claim.	386 sentences
M-Arg	A multimodal dataset for argumentative relation classification from the 2020 US Presidential elections. Sentences are labeled as attacking, supporting, or unrelated to another sentence.	4,104 pairs

All datasets will be available through MAMKit.

Since many multimodal datasets cannot release audio samples due to copyright restrictions, MAMKit provides an interface to dynamically build datasets and promote reproducible research.

Datasets are formatted as torch.Dataset objects, containing input values (text, audio, or both) and corresponding task-specific labels. More details about data formats and dataset building are available in MAMKit’s documentation. ## Retrieving the Data through MAMKit

To retrieve the datasets through MAMKit, you can use the following code interface:

from mamkit.data.datasets import MMUSEDFallacy, MMUSED, UKDebates, MArg, InputMode

import logging
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')

 # MM-USED-fallacy dataset
 mm_used_fallacy_loader = MMUSEDFallacy(
 task_name='afc', # Choose between 'afc' or 'afd' 
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # MM-USED dataset
 mm_used_loader = MMUSED(
 task_name='asd',#Choose between 'asd' or 'acc' 
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # UKDebates dataset
 uk_debates_loader = UKDebates(
 task_name='asd',
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

 # M-Arg dataset
 m_arg_loader = MArg(
 task_name='arc',
 input_mode=InputMode.TEXT_AUDIO, # Choose between TEXT_ONLY, AUDIO_ONLY, or TEXT_AUDIO
 base_data_path=base_data_path
 )

Each loader is initialized with the appropriate task name (afc for argumentative fallacy classification, asd for argumentative sentence detection, and ‘arc’ for argumentative relation classification), input mode (InputMode.TEXT_ONLY, InputMode.AUDIO_ONLY, or InputMode.TEXT_AUDIO), and the base data path.

Ensure that you have MAMKit installed and properly configured in your environment to use these loaders.

For more details, refer to the MAMKit GitHub repository and website .

Test Set Access 🔍

The test set for mm-argfallacy-2025 is now available! To use it, please:

Create a fresh environment
Clone the repository and install the requirements:

git clone git@github.com:nlp-unibo/mamkit.git
cd mamkit
pip install -r requirements.txt
pip install --editable .

Access MAMKit in your Python code:

import mamkit

Then, retrieve the data using the following code:

For Fallacy Classification (`afc`):

from mamkit.data.datasets import MMUSEDFallacy, InputMode
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')
 loader = MMUSEDFallacy(
 task_name='afc',
 input_mode=InputMode.TEXT_ONLY, # or TEXT_AUDIO or AUDIO_ONLY
 base_data_path=base_data_path
 )
 split_info = loader.get_splits('mm-argfallacy-2025')

For Fallacy Detection (`afd`):

from mamkit.data.datasets import MMUSEDFallacy, InputMode
from pathlib import Path

def loading_data_example():
 base_data_path = Path(__file__).parent.parent.resolve().joinpath('data')
 loader = MMUSEDFallacy(
 task_name='afd',
 input_mode=InputMode.TEXT_ONLY, # or TEXT_AUDIO or AUDIO_ONLY
 base_data_path=base_data_path
 )
 split_info = loader.get_splits('mm-argfallacy-2025')

References

MM-USED-fallacy: Mancini et al. (2024). The version provided through MAMKit includes updated samples, with refinements in the alignment process. This results in a different number of samples compared to the original dataset.
MM-USED: Mancini et al. (2022). The version provided through MAMKit includes updated samples, with refinements in the alignment process. This results in a different number of samples compared to the original dataset.
UK-Debates: Lippi and Torroni (2016).
M-Arg: Mestre et al. (2021).

Note: By “updated version,” we mean that the datasets have undergone a refinement in the alignment process, which has resulted in adjustments to the number of samples included compared to the original versions published in the referenced papers.

Evaluation

For argumentative fallacy detection, we will compute the binary F1-score on predicted sentence-level labels.
For argumentative fallacy classification, we will compute the macro F1-score on predicted sentence-level labels.
Metrics will be computed on the hidden test set to determine the best system for each sub-task and input mode.

Evaluation will be performed via the CodaLab platform.
On CodaLab, participants will find the leaderboard, along with the results of the provided baselines.
Submission guidelines can be found under the Evaluation section of the CodaLab competition page.

🚨 Important: In the evaluation website, you will also find a link to a mandatory participation survey.
Filling out this survey is required in order to participate in the task.
We also provide the survey link here for convenience: https://tinyurl.com/limesurvey-argfallacy

Baseline Results on Test Set

Argumentative Fallacy Classification (AFC) – Macro F1-score

Input Modality	Model	F1-Score
Text-only	BiLSTM w/ GloVe	47.21
Text-only	RoBERTa	39.25
Audio-only	BiLSTM w/ MFCCs	15.82
Audio-only	WavLM	6.43
Text + Audio	BiLSTM (GloVe + MFCCs)	21.91
Text + Audio	MM-RoBERTa + WavLM	38.16

Argumentative Fallacy Detection (AFD) – Binary F1-score

Input Modality	Model	F1-Score
Text-only	BiLSTM w/ GloVe	24.62
Text-only	RoBERTa	27.70
Audio-only	BiLSTM w/ MFCCs	0.00
Audio-only	WavLM	0.00
Text + Audio	BiLSTM (GloVe + MFCCs)	23.37
Text + Audio	MM-RoBERTa + WavLM	28.48

Submission

All evaluated submissions are required to commit to submitting a system description paper. You can choose between two options:

Non-Archival Paper:
A 2-page paper describing your system, with unlimited pages for appendices and bibliography. These papers will not be published in the workshop proceedings, but your system will be mentioned in the Overview Paper of the shared task, upon acceptance.
Archival Paper:
A 4-page paper describing your system, also with unlimited pages for appendices and bibliography. These papers will be published in the official ACL workshop proceedings and must be presented at the workshop (poster or oral session).
⚠️ In accordance with ACL policy, at least one team member must register for the workshop in order to present an archival paper if aaccepted to be published at the ACL proceedings.

All papers must use the official ACL style templates, available in both LaTeX and Word. We strongly recommend using the official Overleaf template for convenience.

We have sent an email to each team with all the details regarding the system description paper submission for MM-ArgFallacy2025. Please check your inbox (and spam folder just in case).

🗓️ Submissions open: May 1st, 2025 (the day after the end of the evaluation period)
🗓️ Submissions close: May 15th, 2025
📢 Notification of acceptance: May 20th, 2025
📝 Camera-ready deadline: May 25th, 2025

Important notes:

All accepted archival papers will be presented during the workshop’s poster session and require at least one registered author.
Non-archival papers do not require registration and are not presented at the workshop, but their systems will be acknowledged in the Overview Paper.

We look forward to receiving your submissions!

🏆 Leaderboard – Shared Task Results

`AFC Task – Argumentative Fallacy Classification`

📝 Text-only

Rank	Team	F1-Macro
1	Team NUST	0.4856
2	Baseline BiLSTM	0.4721
3	alessiopittiglio	0.4444
4	Baseline RoBERTa	0.3925
5	Team CASS	0.1432

🔊 Audio-only

Rank	Team	F1-Macro
1	alessiopittiglio	0.3559
2	Team NUST	0.1588
3	Baseline BiLSTM + MFCCs	0.1582
4	Team CASS	0.0864
5	Baseline WavLM	0.0643

🔁 Text-Audio

Rank	Team	F1-Macro
1	Team NUST	0.4611
2	alessiopittiglio	0.4403
3	Baseline RoBERTa + WavLM	0.3816
4	Baseline BiLSTM + MFCCs	0.2191
5	Team CASS	0.1432

`AFD Task – Argumentative Fallacy Detection`

📝 Text-only

Rank	Team	F1-Macro
1	Baseline RoBERTa	0.2770
2	Ambali_Yashovardhan	0.2534
3	Baseline BiLSTM	0.2462
4	Team EvaAdriana	0.2195

🔊 Audio-only

Rank	Team	F1-Macro
1	Ambali_Yashovardhan	0.2095
2	Team EvaAdriana	0.1690
3	Baseline BiLSTM + MFCCs	0.0000
4	Baseline WavLM	0.0000

🔁 Text-Audio

Rank	Team	F1-Macro
1	Baseline RoBERTa + WavLM	0.2848
2	Baseline BiLSTM + MFCCs	0.2337
3	Ambali_Yashovardhan	0.2244
4	Team EvaAdriana	0.1931

Key Dates (Anywhere on Earth)

Release of Training Data: February 25th
Release of Test Set: ~~March 24th~~ → April 7th
Evaluation Start: ~~April 14th~~ → April 21st
Evaluation End: ~~April 25th~~ → April 30th
Paper Submissions Open: May 1st
Paper Submission Close: May 15th
Notification of acceptance: May 20th
Camera-ready Due: May 25th
Workshop: July 31st

Task Organizers

	Eleonora Mancini Language Technologies Lab, University of Bologna, Italy		Federico Ruggeri Language Technologies Lab, University of Bologna, Italy
	Serena Villata Inria-I3S WIMMICS Laboratoire I3S, CNRS, Sophia Antipolis, France		Paolo Torroni Language Technologies Lab, University of Bologna, Italy

Contacts

Join the MM-ArgFallacy2025 Slack Channel!

Cite

Eleonora Mancini, Federico Ruggeri, Serena Villata, and Paolo Torroni. 2025. Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates. In Proceedings of the 12th Argument mining Workshop, pages 358–368, Vienna, Austria. Association for Computational Linguistics.

@inproceedings{mancini-etal-2025-overview,
 title = "Overview of {MM}-{A}rg{F}allacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates",
 author = "Mancini, Eleonora and
 Ruggeri, Federico and
 Villata, Serena and
 Torroni, Paolo",
 editor = "Chistova, Elena and
 Cimiano, Philipp and
 Haddadan, Shohreh and
 Lapesa, Gabriella and
 Ruiz-Dolz, Ramon",
 booktitle = "Proceedings of the 12th Argument mining Workshop",
 month = jul,
 year = "2025",
 address = "Vienna, Austria",
 publisher = "Association for Computational Linguistics",
 url = "https://aclanthology.org/2025.argmining-1.35/",
 doi = "10.18653/v1/2025.argmining-1.35",
 pages = "358--368",
 ISBN = "979-8-89176-258-9",
 abstract = "We present an overview of the MM-ArgFallacy2025 shared task on Multimodal Argumentative Fallacy Detection and Classification in Political Debates, co-located with the 12th Workshop on Argument Mining at ACL 2025. The task focuses on identifying and classifying argumentative fallacies across three input modes: text-only, audio-only, and multimodal (text+audio), offering both binary detection (AFD) and multi-class classification (AFC) subtasks. The dataset comprises 18,925 instances for AFD and 3,388 instances for AFC, from the MM-USED-Fallacy corpus on U.S. presidential debates, annotated for six fallacy types: Ad Hominem, Appeal to Authority, Appeal to Emotion, False Cause, Slippery Slope, and Slogan. A total of 5 teams participated: 3 on classification and 2 on detection. Participants employed transformer-based models, particularly RoBERTa variants, with strategies including prompt-guided data augmentation, context integration, specialised loss functions, and various fusion techniques. Audio processing ranged from MFCC features to state-of-the-art speech models. Results demonstrated textual modality dominance, with best text-only performance reaching 0.4856 F1-score for classification and 0.34 for detection. Audio-only approaches underperformed relative to text but showed improvements over previous work, while multimodal fusion showed limited improvements. This task establishes important baselines for multimodal fallacy analysis in political discourse, contributing to computational argumentation and misinformation detection capabilities."
}

Credits

This shared task is partially supported by the project European Commission's NextGeneration EU programme, PNRR -- M4C2 -- Investimento 1.3, Partenariato Esteso, PE00000013 - FAIR - Future Artificial Intelligence Research'' -- Spoke 8 Pervasive AI’’.

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Tue, 01 Jul 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains

Tue, 01 Apr 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Automated Extraction of Judicial Interpretative Formulas in EU Case Law on VAT

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Detecting Vague Clauses in Italian Privacy Policies Using Transformers, LLMs, and Cross-Lingual Techniques

Wed, 01 Jan 2025 00:00:00 +0000

🏆 Awarded the Honorable Mention in the Best Paper Award Consideration

Add the full text or supplementary notes for the publication here using Markdown formatting.

Dynamic Demonstrations Selection for Few-Shot Legal Argument Mining

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Improving the Teaching of Artificial Intelligence Through Project-Based Learning on a Board Game

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Investigating the effectiveness of explainability methods in parkinson’s detection from speech

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Is It Worth Using LLMs for Unfair Clause Detection in Terms of Service?

Wed, 01 Jan 2025 00:00:00 +0000

🏆 Awarded the “Peter Jackson” Award for Best Innovative Application Paper

LMAC-TD: Producing Time Domain Explanations for Audio Classifiers

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Principles Of Law In National and European VAT (POLINE)

Wed, 01 Jan 2025 00:00:00 +0000

POLINE aims at developing an AI-powered pilot tool for the retrieval and analysis of judicial principles of law in the CJEU and national case-law in Value Added Tax (VAT). The tool relies on AI techniques for extracting, clustering and linking judicial principles of law and is embedded in a modular platform, consisting of a Legal Database, Link Visualization and Customised Detection Module. It covers the case-law of the CJEU and the Italian, Swedish and Bulgarian Supreme Courts and will be accessible to judges, other legal practitioners, tax policymakers and taxpayers. The development of the tool will be based on a multidisciplinary approach combining theory and practice of judicial decision-making for the study of the concept of “judicial principle of law” and the analysis of the case-law; legal informatics methods for the creation of an ontology of judicial concepts in VAT and training datasets of annotated judicial principles of law; AI, machine learning, and NLP techniques for the automatic extraction of principles, the detection of textual and semantic similarity, and network analysis. The tool will be trialled in 3 online national testing events and disseminated in 3 national demonstration events and 1 final international conference. The pilot tool provides a robust and trustworthy use case of AI technologies for justice. It will provide non-discriminatory and effective access to justice. Through its collection of principles of law and NLP-powered search engine, the tool will assist judges in accessing legal knowledge reducing their work overload. Moreover, through the Customised Detection Test Module, the tool will allow recipients of VAT measures to identify judicial principles of law applied in a specific case and assess whether VAT law is correctly applied. By developing open-access automated techniques of knowledge extraction, the methods developed can be easily reused and expanded to include other fields of law and other legal systems.

Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

TRI-DEP: A Trimodal Comparative Study for Depression Detection Using Speech, Text, and EEG

Wed, 01 Jan 2025 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

AMELIA - Argument Mining Evaluation on Legal documents in ItAlian: A CALAMITA Challenge

Sun, 01 Dec 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Generation and Evaluation of English Grammar Multiple-Choice Cloze Exercises

Sun, 01 Dec 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts

Fri, 01 Nov 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

A Grice-ful Examination of Offensive Language: Using NLP Methods to Assess the Co-operative Principle

Tue, 01 Oct 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Sun, 01 Sep 2024 00:00:00 +0000

Resources

Abstract

We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF). Previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, identifying previously fact-checked claims, supporting evidence retrieval, and claim verification. In this edition, we introduced some new challenges, offering six tasks in fifteen languages (Arabic, Bulgarian, English, Dutch, French, Georgian, German, Greek, Italian, Polish, Portuguese, Russian, Slovene, Spanish, and code-mixed Hindi-English): Task 1 on estimation of check-worthiness (the only task that has been present in all CheckThat! editions), Task 2 on identification of subjectivity (a follow up of the CheckThat! 2023 edition), Task 3 on identification of the use of persuasion techniques (a follow up of SemEval 2023), Task 4 on detection of hero, villain, and victim from memes (a follow up of CONSTRAINT 2022), Task 5 on rumor verification using evidence from authorities (new task), and Task 6 on robustness of credibility assessment with adversarial examples (new task). These are challenging classification and retrieval problems at the document and at the span level, including multilingual and multimodal settings. This year, CheckThat! was one of the most popular labs at CLEF-2024 in terms of team registrations: 130 teams. More than one-third of them (a total of 46) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tanmoy Chakraborty, Tamer Elsayed, Piotr Przybya, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, et al. Overview of the clef-2024 checkthat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness. In International Conference of the Cross-Language Evaluation Forum for European Languages, pages 28–52. Springer, 2024.

@inproceedings{barron-etal-20240-overview,
 title={Overview of the CLEF-2024 CheckThat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness},
 author={Barr{\'o}n-Cede{\~n}o, Alberto and Alam, Firoj and Stru{\ss}, Julia Maria and Nakov, Preslav and Chakraborty, Tanmoy and Elsayed, Tamer and Przyby{\l}a, Piotr and Caselli, Tommaso and Da San Martino, Giovanni and Haouari, Fatima and others},
 booktitle={International Conference of the Cross-Language Evaluation Forum for European Languages},
 pages={28--52},
 year={2024},
 organization={Springer}
}

MAMKit: A Comprehensive Multimodal Argument Mining Toolkit

Thu, 01 Aug 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

An Artificial Intelligence System for the Autonomous Pagination of Newspapers

Mon, 01 Jul 2024 00:00:00 +0000

Development of an LLM-based System for the Generation of Multiple-choice English Grammar Exercises

Mon, 01 Jul 2024 00:00:00 +0000

Enhancing Document Parsing And Question Answering Through Optimized Table Parsing

Mon, 01 Jul 2024 00:00:00 +0000

A Corpus for Sentence-Level Subjectivity Detection on English News Articles

Wed, 01 May 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

PejorativITy: Disambiguating Pejorative Epithets to Improve Misogyny Detection in Italian Tweets

Wed, 01 May 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Unfair clause detection in terms of service across multiple languages

Mon, 01 Apr 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

ACME Chatbot

Fri, 01 Mar 2024 00:00:00 +0000

Automating Test Case Generation for Automotive Industry using Large Language Models

Fri, 01 Mar 2024 00:00:00 +0000

Cinnamon

Fri, 01 Mar 2024 00:00:00 +0000

CLAUDETTE

Fri, 01 Mar 2024 00:00:00 +0000

Diving into Song Lyrics with Large Language Models: Unveiling Metadata Insights and Fueling Video Lyrics Generation

Fri, 01 Mar 2024 00:00:00 +0000

Example Sentence Suggestion for Learners of Japanese as a Second Language Using Pretrained Language Models

Fri, 01 Mar 2024 00:00:00 +0000

Fine-Tuning Neural Codec Language Models from Feedback with Reinforcement Learning

Fri, 01 Mar 2024 00:00:00 +0000

MAMKit

Fri, 01 Mar 2024 00:00:00 +0000

MARGOT

Fri, 01 Mar 2024 00:00:00 +0000

Multimodal Fallacy Classification in Political Debates

Fri, 01 Mar 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Developing and Comparing Machine Reasoning Models to Humans in NLP Tasks

Thu, 01 Feb 2024 00:00:00 +0000

A Chatbot for Asylum-Seeking Migrants in Europe

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

AI-based Smart Collaborative Manufacturing System (SmartCasm)

Mon, 01 Jan 2024 00:00:00 +0000

The project involves using LLMs to integrate unstructured knowledge into industrial pipelines to speed up production and foster technical advancement.

Combining Transformers with Natural Language Explanations

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Disruptive situation detection on public transport through speech emotion recognition

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Future Artificial Intelligence Research (FAIR)

Mon, 01 Jan 2024 00:00:00 +0000

The objective of the FAIR project is to contribute facing the research questions, methodologies, models, technologies, and ethical and legal rules to build AI systems capable of interacting and collaborating with humans, perceiving and acting in evolving contexts, to be conscious about their limits and capable to adapt to new situations, to be aware of the perimeters of safety and trust, and to be careful with the environmental and social impact that their creation and functioning may cause.

Generative Models: Empowering Business Processes and Enhancing Workflows for Improved Performance (GeMEB)

Mon, 01 Jan 2024 00:00:00 +0000

The project developing ad-hoc LLM-based solutions to speed up existing user assistance systems while guaranteeing privacy.

Overview of the CLEF-2024 CheckThat! Lab Task 2 on Subjectivity in News Articles

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF-2024 CheckThat! lab: check-worthiness, subjectivity, persuasion, roles, authorities, and adversarial robustness

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

PRivacy Infringements Machine-Advice (PRIMA)

Mon, 01 Jan 2024 00:00:00 +0000

Sustainable Development Goals Artificial Intelligence Enhance (ALMA-GAIE)

Mon, 01 Jan 2024 00:00:00 +0000

🏆 The project received the “PA a colori” 2024 award for sustainability in public administrations.

A project funded by the University of Bologna aiming to develop an AI system for the automatic classification of research and educational products of the University of Bologna according to their contribution to the 17 Goals of the United Nations’ 2030 Agenda for Sustainable Development.

The CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

TWOLAR: A TWO-Step LLM-Augmented Distillation Method for Passage Reranking

Mon, 01 Jan 2024 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Addressing Misinformation Challenges in War Scenario: Russo-Ukrainian War

Fri, 01 Dec 2023 00:00:00 +0000

Comprehensive study of clinical entity extraction and classification using Large Language Models

Fri, 01 Dec 2023 00:00:00 +0000

LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain

Fri, 01 Dec 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

A Two-Step LLM-Augmented Distillation Method For Passage Reranking

Sun, 01 Oct 2023 00:00:00 +0000

Argument Mining into Active Learning Systematic Reviews: unlocking the synergy between MARGOT and ASReview

Sun, 01 Oct 2023 00:00:00 +0000

Automatic Terminology Coding for the Biomedical Domain

Sun, 01 Oct 2023 00:00:00 +0000

Design and Implementation of a Neural Machine Translation Engine for Computer-Assisted Translations

Sun, 01 Oct 2023 00:00:00 +0000

Empathic Voice: Enabling Emotional Intelligence in Virtual Assistants

Sun, 01 Oct 2023 00:00:00 +0000

From text to knowledge: Large Language Models-based methods for knowledge extraction

Sun, 01 Oct 2023 00:00:00 +0000

Leveraging Large Language Models for content analysis and generation for podcast transcriptions

Sun, 01 Oct 2023 00:00:00 +0000

Neural-Symbolic Learning: challenges and benchmarks

Sun, 01 Oct 2023 00:00:00 +0000

On the use of Prompting for Fine-Tuning Neural models for Speech Processing

Sun, 01 Oct 2023 00:00:00 +0000

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Fri, 01 Sep 2023 00:00:00 +0000

Resources

Abstract

We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, verifying whether a claim was fact-checked before, supporting evidence retrieval, and claim verification. In this sixth edition, we zoom into some new problems and for the first time we offer five tasks in seven languages: Arabic, Dutch, English, German, Italian, Spanish, and Turkish. Task 1 asks to determine whether an item —text or text plus image— is check-worthy. Task 2 aims to predict whether a sentence from a news article is subjective or not. Task 3 asks to assess the political bias of the news at the article and at the media outlet level. Task 4 focuses on the factuality of reporting of news media. Finally, Task 5 looks at identifying authorities in Twitter that could help verify a given target claim. For a second year, CheckThat! was the most popular lab at CLEF-2023 in terms of team registrations: 127 teams. About one-third of them (a total of 37) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S. Cheema, Fatima Haouari, Maram Hasanain, Mücahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, and Wajdi Zaghouani. Overview of the CLEF-2023 checkthat! lab on checkworthiness, subjectivity, political bias, factuality, and authority of news articles and their source. In Avi Arampatzis, Evangelos Kanoulas, Theodora Tsikrika, Stefanos Vrochidis, Anastasia Giachanou, Dan Li, Mohammad Aliannejadi, Michalis Vlachos, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 14th International Conference of the CLEF Association, CLEF 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings, volume 14163 of Lecture Notes in Computer Science, pages 251–275. Springer, 2023.

@inproceedings{cedeno-etal-2023-clef-overview,
 author = {Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and
 Firoj Alam and
 Andrea Galassi and
 Giovanni Da San Martino and
 Preslav Nakov and
 Tamer Elsayed and
 Dilshod Azizov and
 Tommaso Caselli and
 Gullal S. Cheema and
 Fatima Haouari and
 Maram Hasanain and
 M{\"{u}}cahid Kutlu and
 Chengkai Li and
 Federico Ruggeri and
 Julia Maria Stru{\ss} and
 Wajdi Zaghouani},
 editor = {Avi Arampatzis and
 Evangelos Kanoulas and
 Theodora Tsikrika and
 Stefanos Vrochidis and
 Anastasia Giachanou and
 Dan Li and
 Mohammad Aliannejadi and
 Michalis Vlachos and
 Guglielmo Faggioli and
 Nicola Ferro},
 title = {Overview of the {CLEF-2023} CheckThat! Lab on Checkworthiness, Subjectivity,
 Political Bias, Factuality, and Authority of News Articles and Their
 Source},
 booktitle = {Experimental {IR} Meets Multilinguality, Multimodality, and Interaction
 - 14th International Conference of the {CLEF} Association, {CLEF}
 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings},
 series = {Lecture Notes in Computer Science},
 volume = {14163},
 pages = {251--275},
 publisher = {Springer},
 year = {2023},
 url = {https://doi.org/10.1007/978-3-031-42448-9\_20},
 doi = {10.1007/978-3-031-42448-9\_20}
}

A Dataset of Argumentative Dialogues on Scientific Papers

Sat, 01 Jul 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Design and implementation of a privacy-preserving dialogue system based on argumentation

Sat, 01 Jul 2023 00:00:00 +0000

Emotion Recognition for Human-Centered Conversational Agents

Wed, 01 Mar 2023 00:00:00 +0000

Knowledge graph embedding enhancement using ontological knowledge in the biomedical domain

Wed, 01 Mar 2023 00:00:00 +0000

Prompting techniques for Natural Language Generation in the Medical Domain

Wed, 01 Mar 2023 00:00:00 +0000

Voice conversion with pre-trained representations for audio anonymization

Wed, 01 Feb 2023 00:00:00 +0000

A European AI On Demand Platform and Ecosystem (AI4EU)

Sun, 01 Jan 2023 00:00:00 +0000

The EU-funded AI4EU is working to change Europe’s place in this race, by building the first European AI On-Demand Platform and Ecosystem that will share resources, tools, knowledge, algorithms and more between Member States. It will help to increase innovation and technology transfer, accelerate the growth of start-ups and SMEs, and fulfill the needs of the European AI community. The project will implement eight pilots led by industrial partners to demonstrate the platform’s capabilities.

Argumentation Structure Prediction in CJEU Decisions on Fiscal State Aid

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Argumentation Structure Prediction in CJEU Decisions on Fiscal State Aid

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Inception Models for Fashion Image Captioning: An Extensive Study on Multiple Datasets

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Multi-Task Attentive Residual Networks for Argument Mining

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

On the Definition of Prescriptive Annotation Guidelines for Language-Agnostic Subjectivity Detection

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF-2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Overview of the CLEF-2023 CheckThat! Lab: Task 2 on Subjectivity Detection

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

TeamUnibo at SemEval-2023 Task 6: A transformer based approach to Rhetorical Roles prediction and NER in Legal Texts

Sun, 01 Jan 2023 00:00:00 +0000

The CLEF-2023 CheckThat! Lab: Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority

Sun, 01 Jan 2023 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

A Sentiment and Emotion Annotated Dataset for Bitcoin Price Forecasting Based on Reddit Posts

Thu, 01 Dec 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Combining WordNet and Word Embeddings in Data Augmentation for Legal Texts

Thu, 01 Dec 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Fast Vocabulary Transfer for Language Model Compression

Thu, 01 Dec 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

SynBA: A contextualized Synonim-Based adversarial Attack for text classification

Thu, 01 Dec 2022 00:00:00 +0000

Voice Cloning: Increasing Expressivity of Italian Text-to-Speech with Phonemization

Thu, 01 Dec 2022 00:00:00 +0000

Contact

Mon, 24 Oct 2022 00:00:00 +0000

People

Mon, 24 Oct 2022 00:00:00 +0000

Intermediate Linguistic Task Fine-tuning On Multilingual Models

Fri, 01 Jul 2022 00:00:00 +0000

Royalty-Management Smart Contracts With Graph Neural Network-Based Artist Recommendations

Fri, 01 Jul 2022 00:00:00 +0000

Vocabulary Transfer and Knowledge Distillation for Language Model Compression

Fri, 01 Jul 2022 00:00:00 +0000

A Critical Survey Of Text-to-image Synthesis By Generative Adversarial Networks: Concepts, Methods, And Evaluations

Tue, 01 Mar 2022 00:00:00 +0000

Using semantic entities to improve the distillation of transformers**

Tue, 01 Mar 2022 00:00:00 +0000

Graph Neural Networks for Recommender Systems**

Tue, 01 Feb 2022 00:00:00 +0000

A privacy-preserving dialogue system based on argumentation

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

AMICA: An Argumentative Search Engine for COVID-19 Literature

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Argument mining as rapid screening tool of COVID-19 literature quality: Preliminary evidence

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Detecting and explaining unfairness in consumer contracts through memory networks

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Detecting Arguments in CJEU Decisions on Fiscal State Aid

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Equitable Algorithms, Promoting Fairness and Countering Algorithmic Discrimination Through Norms and Technologies (EquAl)

Sat, 01 Jan 2022 00:00:00 +0000

The EquAl project addresses algorithmic evaluations, decisions, and predictions, to promote fairness and counter discrimination affecting individuals and groups. The research project fundedis by the EU Commission under the NextGenerationEU program and the Italian Ministry of Education, University and Research. (PRIN 2022. Ref. prot. n.: 2022KFLF3E-001 - CUP J53D23005560001) EquAl aims (i) to provide an understanding of the concepts of algorithmic unfairness and discrimination, bridging the notions adopted in social sciences, law, statistics, and artificial intelligence. (ii) To identify the ways in which algorithmic unfairness originates and spreads in different social contexts, affecting individuals and groups, and particularly to identify the cases in which algorithmic unfairness leads to prohibited discrimination. (iii) To analyse the ways in which the law currently addresses algorithmic discrimination and propose appropriate measures to implement or upgrade the existing regulatory framework. (iv) To examine the way in which technologies can promote fairness and support detecting and countering algorithmic unfairness and discrimination, in particular with regard to the assessment of asylum requests. By identifying and remedying algorithmic unfairness and discrimination, EquAl will contribute to preventing and mitigating harms to individuals and groups and favour the law-abiding deployment of AI. EquAl is premised on the fast-growing application of AI techniques for the purposes of prediction, evaluation, and decision making. Algorithmic approaches have the potential to transform many aspects of the economic and social life, delivering cost effective solutions, increasing the equity, efficiency, controllability and precision of decision-making processes. However, they may also lead to new and more subtle, opaque, and resilient forms of unfairness and discrimination. Some discriminatory effects have been already addressed by case-law in Europe and beyond, and some proposals exist to regulate aspects of automated decision-making, but no comprehensive regulatory framework exists yet. EquAl aims to place Italian legal research at the forefront in the domain of algorithmic fairness and non-discrimination, by: (a) delivering new insights on the specific nature, functioning, and evolution of fair and unfair instances of algorithmic decision-making; (b) evaluating existing anti-discrimination technologies and developing new methods to detect instances of unfairness in human and automated decisions and protect vulnerable individuals; (c) providing ethical and legal guidance and (d) supporting public bodies, NGOs and local communities, in particular, in the examination of asylum applications. EquAl’s contribution is crucial to enhance interdisciplinary cross-fertilisation, since currently different criteria and terminologies are used in debating algorithmic fairness and non-discrimination by different research communities (legal scholars, sociologists, computer scientists, statisticians), and to ensure that the corpus of EU and Italian anti-discrimination law, regulations, and case-law can be effectively applied in the algorithmic domain.

Hybrid Offline/Online Optimization for Energy Management via Reinforcement Learning

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Legal Analytics for Italian Law (LAILA)

Sat, 01 Jan 2022 00:00:00 +0000

The project regards the application of Legal Analytics methods to a vast and heterogeneous set of legal information: legislations, contracts, and judgments. The purpose is the application of Artificial Intelligence, Machine Learning, and Natural Language Processing to extract legal knowledge, infer relationships, and produce data-driven forecasts.

Multimodal Argument Mining: A Case Study in Political Debates

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Predicting Outcomes of Italian VAT Decisions

Sat, 01 Jan 2022 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Stairway to AI: Ease the Engagement of Low-Tech users to the AI-on-Demand platform through AI (StairwAI)

Sat, 01 Jan 2022 00:00:00 +0000

The StairwAI project targets low-tech users with the goal of facilitating their engagement on the AI4EU on-demand Platform. This will be achieved through a new service layer enriching the functionalities of the on-demand platform and containing: (1) a multi-lingual interaction layer enabling conversations with the Platform in the user’s own language, (2) a horizontal matchmaking service for the automatic discovery of AI assets (tools, data sets, AI experts, consultants, papers, courses etc.) meeting the user business needs and, (3) a vertical matchmaking service that will dimension and provision hardware resources through a proper hardware provider (HPC, Cloud and Edge infrastructures).

Disruptive Situations Detection on Public Transports through Speech Emotion Recognition

Wed, 01 Dec 2021 00:00:00 +0000

A Neurosymbolic Framework for Markov Logic Networks

Fri, 01 Oct 2021 00:00:00 +0000

Advanced techniques for cross-language annotation projection in legal texts

Thu, 01 Jul 2021 00:00:00 +0000

Small transformers for Bioinformatics tasks

Thu, 01 Jul 2021 00:00:00 +0000

Automatic extraction of scientific articles based on user queries expressed in natural language

Mon, 01 Feb 2021 00:00:00 +0000

An Argumentative Dialogue System for COVID-19 Vaccine Information

Fri, 01 Jan 2021 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Analytics for DEcision of LEgal cases (ADELE)

Fri, 01 Jan 2021 00:00:00 +0000

Project ADELE is premised on the ongoing paradigm shift towards cognitive computing and human-centered AI which is transforming many socio-economic activities, including justice. The project applies legal analytics (LA) – a blend of data science, machine learning, and natural language processing techniques – to judicial decisions. It aims to develop methods to extract knowledge and engage in outcome predictions and there build a pilot tool to support legal research and decision-making processes in the judiciary.

Argument Mining In Covid-19 Articles (AMICA)

Fri, 01 Jan 2021 00:00:00 +0000

The objective of the AMICA project was to exploit the argumentative content present in the scientific literature regarding Covid-19 to improve the retrieval of relevant and reliable articles. The project involved both medical and artificial intelligence experts and aimed to develop an argument mining-based search engine, specifically designed for the analysis of scientific literature related to Covid-19.

Attention in Natural Language Processing

Fri, 01 Jan 2021 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

European Network of Human-Centered Artificial Intelligence (HUMANE-AI-NET)

Fri, 01 Jan 2021 00:00:00 +0000

Facilitating a European brand of trustworthy, ethical AI that enhances Human capabilities and empowers citizens and society to effectively deal with the challenges of an interconnected globalized world. Funded by the European Commission under H2020, ICT-48. The University of Bologna participates with 4 departments: Computer Science and Engineering, Mathematics, Biomedical and Neuromotor Sciences, and Political and Social Sciences. We coordinate the Humane-AI-Net activities for the University of Bologna and work on several projects: the micro-project Ethical Chatbots, the micro-project Promoting Fairness and Diversity in Speech Datasets for Affective Computing, the micro-project A Transparent and Explainable Dialogue System for Immigration Services, the macro-project “Learning with LLMs: Supporting complex reasons, planning & argumentation applied to provide educational guidance”.

Investigating logic tensor networks for neural-symbolic argument mining

Fri, 01 Jan 2021 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

SubjectivITA: An Italian Corpus for Subjectivity Detection in Newspapers

Fri, 01 Jan 2021 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Tree-Constrained Graph Neural Networks For Argument Mining

Fri, 01 Jan 2021 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Cross-lingual Annotation Projection in Legal Texts

Tue, 01 Dec 2020 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Development of an Argumentative Chatbot in Python

Wed, 01 Jul 2020 00:00:00 +0000

Explaining Potentially Unfair Clauses to the Consumer with the CLAUDETTE tool

Wed, 01 Jan 2020 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Neural-Symbolic Argumentation Mining: An Argument in Favor of Deep Learning and Reasoning

Wed, 01 Jan 2020 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

The force awakens: Artificial intelligence for consumer law

Wed, 01 Jan 2020 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Consumer protection requires artificial intelligence

Mon, 01 Apr 2019 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

CLAUDETTE: an automated detector of potentially unfair clauses in online terms of service

Tue, 01 Jan 2019 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Deep Learning for Detecting and Explaining Unfairness in Consumer Contracts

Tue, 01 Jan 2019 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Towards Consumer-Empowering Artificial Intelligence

Sun, 01 Jul 2018 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Argument mining on clinical trials

Mon, 01 Jan 2018 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Automated Processing of Privacy Policies Under the EU General Data Protection Regulation

Mon, 01 Jan 2018 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

automated CLAUse DETectEr (CLAUDETTE)

Sun, 01 Jan 2017 00:00:00 +0000

CLAUDETTE is an interdisciplinary research project hosted at the Law Department of the European University Institute. The research objective is to test to what extent is it possible to automate reading and legal assessment of online consumer contracts and privacy policies, to evaluate their compliance with EU’s unfair contractual terms law and personal data protection law (GDPR), using machine learning and grammar-based approaches.

Argument Mining from Speech: Detecting Claims in Political Debates

Tue, 01 Mar 2016 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Argumentation Mining: State of the Art and Emerging Trends

Tue, 01 Mar 2016 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Constraint Detection in Natural Language Problem Descriptions

Fri, 01 Jan 2016 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

MARGOT: A web server for argumentation mining

Fri, 01 Jan 2016 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Argument Mining: A Machine Learning Perspective

Thu, 01 Jan 2015 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Context-Independent Claim Detection for Argument Mining

Thu, 01 Jan 2015 00:00:00 +0000

Add the full text or supplementary notes for the publication here using Markdown formatting.

Mon, 01 Jan 0001 00:00:00 +0000

Language Technologies Lab

Argument Mining

Argument Mining on Legal Datasets

Hate Speech Detection with Argumentative Reasoning

Interpretability

Knowledge Extraction from Rationalization

Legal Analytics

Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology

Mixture of Experts for Rationalization

Multi-cultural Abusive and Hate Speech Detection

Multimodal Argument Mining

Paper accepted at TACL!

Description

Read More

Rationalization via LLMs

Structured Rationalization via Tree kernel methods

Text Classification with Guidelines Only

Tour

Transformers and LLMs for the detection and classification of unfair clauses

Unstructured Knowledge Integration

AMELR

ArgMining

Argument Mining

Argumentative Fallacies

Multimodality

Dialogues

Interpretability

Reasoning in LLMs

ASAIL

CLEF

CLIC-it

EVALITA

Legal Analytics

Argumentation and Argument Schema

Judgement Prediction

Interpretability

Unfair Clause Detection

Summarization

Cross-linguality

Information Retrieval

LUHME

SemEval

Speech Processing

Argumentation

Clinical domain

Interpretability

Multimodality

EquAl: Equitable Algorithms, Promoting Fairness and Countering Algorithmic Discrimination Through Norms and Technologies - Final Conference

Project info

Useful Links

PRIMA: PRivacy Infringements Machine-Advice - Final Conference

Project info

Useful Links

Leveraging Whisper Embeddings for Audio-based Lyrics Matching

ALMA-AI | Workshop LLM: a debate on technical experiences

Workshop Abstract

Federico Ruggeri’s speech

Useful Links

Do Large Language Models understand how to be judges?

The 13th Workshop on Argument Mining and Reasoning Co-located with ACL 2026

Introduction

Useful Links

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Resources

Abstract

Citation

Sustainable Italian LLM Evaluation: Community Perspectives and Methodological Guidelines

Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification

Interlocking-free Selective Rationalization Through Genetic-based Learning

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Overview

Tasks

Data

Test Set Access 🔍

For Fallacy Classification (afc):

For Fallacy Detection (afd):

References

Evaluation

Baseline Results on Test Set

Argumentative Fallacy Classification (AFC) – Macro F1-score

For Fallacy Classification (`afc`):

For Fallacy Detection (`afd`):

`AFC Task – Argumentative Fallacy Classification`

`AFD Task – Argumentative Fallacy Detection`