Language is Scary when Over-Analyzed: Unpacking Implied Misogynistic Reasoning with Argumentation Theory-Driven Prompts

Fri, 01 Nov 2024 00:00:00 +0000

Research setting

Misogyny detection is a task of hate speech detection: a model reads a social media message and decides whether it is misogynous. Many misogynous messages are implicit. They contain no slur and no insult, and their hate lies in an unstated assumption about women, called an implied assumption. Classifiers trained mostly on explicit hate tend to miss these messages, as the example below shows.

An existing hate speech classifier (bert-hateXplain) flags the explicit message but misses the implicit one (Figure 1 of the paper). Its implied assumptions are that women are not as capable as men and should be told what to do.

The paper reads such messages as arguments, following the argument model of Toulmin. In this model, the claim is what the message states, and the warrant is the unstated link that makes the claim acceptable. A large language model (LLM) is a general text model that follows written instructions, called prompts. In the zero-shot setting the prompt gives only the instructions, and in the few-shot setting it also gives a few solved examples.

Motivation

Most work on hate speech has focused on explicit messages. Some English datasets pair implicit hateful messages with human-written implied statements, but few efforts generate these statements automatically. No study had evaluated whether widely used LLMs can reason their way to good implied assumptions, and Italian had no dataset for implicit misogyny.

Line of work	Targets implicit hate	Generates implied assumptions automatically	Argumentative reasoning framing
Explicit hate speech datasets and systems	no	no	no
Implicit hate datasets with implied statements (Sap et al., 2020; ElSherief et al., 2021)	yes	Human-written only	no
Concept-based explanation generation (Ji et al., 2020; Yang et al., 2023)	partially	yes	no
LLMs on argumentative tasks (Wachsmuth et al., 2024; Chen et al., 2023; Gorur et al., 2024)	no	no	yes
This paper This work	yes	yes	yes

How the paper relates to the prior work it discusses (Sections 1 and 2 of the paper).

Objective. Frame implicit misogyny detection as an argumentative reasoning task, and test whether LLMs can generate the missing warrant of a message and use it to classify the message, in English and Italian.

Approach

The paper prompts the LLM to write out the hidden reasoning before it gives its label. In the Toulmin setting, the LLM states the claim and the warrant, and then decides whether the message is misogynous based on the warrant. In the Assumption setting, the LLM writes the implied assumptions directly, without a claim, and then classifies. The Toulmin prompts ask for the reasoning step by step, a technique called chain-of-thought prompting. In both settings the generated text then informs the decision, a technique called knowledge-augmented prompting.

Message Women football commentators annoy me so much.
Claim Women football commentators are annoying.
Warrant The LLM writes the hidden link. Women do not understand sport.
Label Misogynous, decided from the warrant.

The Toulmin setting on the example of Figure 2 of the paper. The Assumption setting skips the claim and asks for the implied assumptions directly.

Each setting runs zero-shot and few-shot, with prompts in English and in Italian. Two instruction-tuned LLMs are tested: Llama3-8B and Mistral-7B-v02. The baselines are fine-tuned classifiers (bert-hateXplain for English, ALBERTo for Italian) and the same LLMs asked for a label with no reasoning.

Results

The experiments use two datasets of implicit misogyny. ImplicIT-Mis is a new Italian dataset of 1,120 Facebook comments, all misogynous, where 150 comments carry 225 human-written implied assumptions. SBIC+ has 2,409 English messages from two existing corpora, each with human-written warrants. Because every message is misogynous, classification is measured by recall, the share of messages labeled misogynous, from 0 to 1. The generated warrants were also checked by hand on 300 messages, 150 per language, for the best model in each language.

0.725recall of Llama3-8B few-shot with Toulmin warrants on ImplicIT-Mis, against 0.480 with implied assumptions
68% / 50%of generated implied assumptions are wrong, in Italian and in English
Allcorrect Italian labels in the manual check were reached for the wrong reasons

Recall on ImplicIT-Mis (higher is better)

ALBERTo (fine-tuned) 0.380
Llama3-8B, label only 0.738
Llama3-8B, Assumption This work 0.480
Llama3-8B, Toulmin This work 0.725
Mistral-7B-v02, label only 0.259
Mistral-7B-v02, Assumption This work 0.461
Mistral-7B-v02, Toulmin This work 0.556

Classification recall on the Italian ImplicIT-Mis dataset with few-shot prompts and the fine-tuned baseline. Source: Table 1 of the paper.

Show as table

System	Recall on ImplicIT-Mis
ALBERTo (fine-tuned)	0.380
Llama3-8B, label only	0.738
Llama3-8B, Assumption (this work)	0.480
Llama3-8B, Toulmin (this work)	0.725
Mistral-7B-v02, label only	0.259
Mistral-7B-v02, Assumption (this work)	0.461
Mistral-7B-v02, Toulmin (this work)	0.556

Setting	Model	ImplicIT-Mis (IT)	SBIC+ (EN)
Fine-tuned	bert-hateXplain / ALBERTo	0.380	0.342
Zero-shot, label only	Llama3-8B	0.588	0.609
Few-shot, label only	Llama3-8B	0.738	0.719
Few-shot, Assumption	Llama3-8B	0.480	0.616
Few-shot, Assumption	Mistral-7B-v02	0.461	0.685
Zero-shot, Toulmin	Llama3-8B	0.557	0.452
Few-shot, Toulmin	Llama3-8B	0.725	0.594
Few-shot, Toulmin	Mistral-7B-v02	0.556	0.604

Classification recall, selected rows. ALBERTo is the Italian baseline and bert-hateXplain the English one. Source: Table 1 of the paper.

LLMs generally beat the fine-tuned classifiers, and Llama3-8B with a plain few-shot prompt gives the best recall in both languages. In Italian, the Toulmin warrant comes close to that best score and gains 24 points over implied assumptions in the few-shot setting, while in English the warrant-based prompt falls behind. The manual check shows that a correct label does not mean a correct explanation. Only 35% of the Italian warrants were correct, and about half of the English implied assumptions were wrong. The most frequent errors were wrong inferences, missing knowledge of the people a message refers to, figurative language, and sarcasm.

Error in the generated warrant	Italian	English
Wrong inference	42	34
Lack of reference	17	0
Metaphorical and figurative language	14	1
Opposite intention	12	9
Wrong translation	10	N/A
Denial of misogyny	3	4
Sarcasm or irony	2	26

Error categories in the manually checked samples. Source: Table 3 of the paper.

Takeaways