The Alignment Bottleneck in Decomposition-Based Claim Verification

Tue, 01 Sep 2026 00:00:00 +0000

Research setting

Claim verification decides whether a claim is true or false, given evidence such as news articles. A complex claim packs several facts into one text, like the tweet in the figure above. Claim decomposition splits a complex claim into sub-claims, and each sub-claim holds one fact. Each sub-claim gets a veracity label: true (T), false (F), or unverified (U). A model then combines the sub-claims and their labels into one verdict for the whole claim.

Claim A tweet that states several facts.
Decompose The claim is split into one-fact sub-claims.
Verify sub-claims Each sub-claim gets T, F, or U from evidence.
Verdict A model combines sub-claims and labels into true or false.

Decomposition-based claim verification, the pipeline that the paper studies (Sections 1 and 4 of the paper).

The evidence for each sub-claim can come in two forms. In claim-level evidence, every sub-claim sees the same evidence collected for the whole claim. In aligned evidence, each sub-claim sees its own evidence span, the few sentences that justify its label.

Motivation

Studies that test decomposition for fact-checking report conflicting results. Some find it unnecessary for LLMs, while others find that its effect depends on the decomposition method. The paper argues that two overlooked factors explain the conflict: how evidence is aligned with each sub-claim, and whether the sub-claim labels are reliable. Testing these factors needs real-world complex claims with evidence annotated for each sub-claim.

Work	Real-world complex claims	Human evidence per sub-claim	Time-bounded evidence
Decomposition study (Hu et al., 2024)	no	Automatic	no
ClaimDecomp (Chen et al., 2022)	yes	Per claim, by journalists	no
WiCE (Kamoi et al., 2023)	Wikipedia	yes	no
This paper This work	yes	yes	yes

How the paper relates to the decomposition studies and datasets it discusses (Sections 2 and 3.3 of the paper).

Objective. Build a dataset of real-world complex claims with human-annotated evidence for each sub-claim, and measure when decomposition helps claim verification as evidence alignment and sub-claim label quality change.

Approach

The paper first builds a dataset of complex claims from PHEME, a collection of rumours posted on Twitter during five breaking news events. It has 399 complex claims and 1169 sub-claims, with an agreement (Bennett’s S) of 0.81 on 300 double-annotated sub-claims.

Filter Keep 423 PHEME claims with several verbs and sentences.
Decompose gpt-3.5-turbo-instruct splits them; sub-claims are checked by hand.
Link evidence News articles from within 2 days of each event.
This work Annotate Label each sub-claim T, F, or U and mark its evidence span.

How the dataset is built (Section 3 of the paper).

The paper then gives one claim verifier, Qwen3-14B prompted zero-shot, different inputs, always together with the claim. Each tab shows one setup and the inputs it receives.

Vanilla

Sub-claims
Labels
Claim evidence Data
Aligned spans
Verifier Zero-shot

The baseline. The verifier sees only the claim and its claim-level evidence, with no decomposition.

Oracle SRE

Sub-claims Data
Labels Data
Claim evidence Data
Aligned spans
Verifier Zero-shot

Sub-claims with Repeated claim-level Evidence. The same claim evidence is repeated after each sub-claim, together with its gold label.

Oracle SAE

Sub-claims Data
Labels Data
Claim evidence
Aligned spans Data
Verifier Zero-shot

Sub-claims with Aligned Evidence. Each sub-claim comes with its own annotated evidence span and its gold label.

Ablation SRE

Sub-claims Data
Labels
Claim evidence Data
Aligned spans
Verifier Zero-shot

Oracle SRE without sub-claim labels. This is the only setup possible on MMM-Fact and COVID-Fact, which have no annotated sub-claim evidence.

Ablation SAE

Sub-claims Data
Labels
Claim evidence
Aligned spans Data
Verifier Zero-shot

Oracle SAE without sub-claim labels. The verifier must judge the sub-claims from their aligned spans alone.

Noisy SAE and SRE

Sub-claims Data
Labels Predicted
Claim evidence
Aligned spans Data
Verifier Zero-shot

The gold labels are replaced by labels predicted by a graph neural network (GNN) or by Qwen3-14B, which read each sub-claim with the claim-level evidence. The tab shows Noisy SAE; Noisy SRE uses claim evidence instead of aligned spans.

Predicted by a sub-claim model Prompted zero-shot, not trained Data

The claim verification setups of the paper. Oracle setups receive the gold sub-claim labels, Ablation setups receive no sub-claim labels, and Noisy setups receive predicted ones (Section 4 and Figure 1 of the paper).

Results

Claim verification is measured with macro-F1, the F1 score averaged over the true and false classes, and with balanced accuracy, the accuracy averaged over classes. Scores are the mean of three runs (six for ablations), and gains over Vanilla are tested with a paired bootstrap and McNemar’s test.

+0.0625macro-F1 over Vanilla with Oracle SAE, PHEMEPlus
−0.0672macro-F1 against Vanilla with Ablation SRE, MMM-Fact
0.4335macro-F1 of Qwen Noisy SRE, PHEMEPlus

Macro-F1 on PHEMEPlus (higher is better)

Vanilla 0.5643 ± 0.0091
Oracle SRE 0.5872 ± 0.0127
Oracle SAE 0.6268 ± 0.0098
Ablation SRE 0.5808 ± 0.0064
Ablation SAE 0.5485 ± 0.0052

Claim verification on the PHEMEPlus claims of the new dataset, mean ± standard deviation over runs. Source: Tables 2 and 3 of the paper.

Show as table

System	Macro-F1 on PHEMEPlus
Vanilla	0.5643 ± 0.0091
Oracle SRE	0.5872 ± 0.0127
Oracle SAE	0.6268 ± 0.0098
Ablation SRE	0.5808 ± 0.0064
Ablation SAE	0.5485 ± 0.0052

Only aligned evidence with gold labels (Oracle SAE) gives a significant gain over Vanilla: +0.0625 macro-F1 and +0.0486 balanced accuracy, both with bootstrap p = 0.0000 (Table 2). Repeated claim evidence (Oracle SRE) gives +0.0229 macro-F1, with bootstrap p = 0.124 and McNemar p = 0.0227. Aligned spans without labels (Ablation SAE) lose 0.0783 macro-F1 against Oracle SAE (Table 3), so granular evidence helps only with correct sub-claim labels.

Dataset	Vanilla	Ablation SRE	Δ	p
COVID-Fact	0.7365 ± 0.0248	0.7252 ± 0.0161	−0.0113	0.549
MMM-Fact	0.7550 ± 0.0069	0.6878 ± 0.0064	−0.0672	0.0000

Macro-F1 of claim verification without sub-claim labels on two other fact-checking datasets. COVID-Fact has health claims with one evidence source each, and MMM-Fact covers several domains with 6 to 10 evidence sources per claim. Δ is the change against Vanilla, and p is the paired bootstrap p-value. Best values in bold, as in the paper. Source: Table 2 of the paper.

On other domains, decomposition with repeated claim evidence does not help. On COVID-Fact the drop is not significant, while on MMM-Fact it is significant, also under McNemar’s test (p = 0.0000).

Macro-F1 on PHEMEPlus (higher is better)

Oracle SAE (gold labels) 0.6268 ± 0.0098
Qwen Noisy SAE 0.5964 ± 0.0360
GNN Noisy SAE 0.5839 ± 0.1202
Qwen Noisy SRE 0.4335 ± 0.0439
GNN Noisy SRE 0.4416 ± 0.0489

Claim verification on PHEMEPlus when gold sub-claim labels are replaced by labels predicted by Qwen3-14B or a GNN, mean ± standard deviation over runs. Source: Table 3 of the paper.

Show as table

System	Macro-F1 on PHEMEPlus
Oracle SAE (gold labels)	0.6268 ± 0.0098
Qwen Noisy SAE	0.5964 ± 0.0360
GNN Noisy SAE	0.5839 ± 0.1202
Qwen Noisy SRE	0.4335 ± 0.0439
GNN Noisy SRE	0.4416 ± 0.0489

Predicted labels lower performance in every setup. With aligned spans the drop is moderate (−0.0304 macro-F1 for Qwen Noisy SAE), while with repeated claim evidence both models fall to about 0.44.

As sub-claim predictors, Qwen3-14B reaches 56.94% macro-F1 and the GNN 45.79% (Table 4), but they also make different errors (Table 5). The GNN abstains with U on 35.0% of sub-claims against 15.8%, and its refutation recall, the share of false sub-claims it finds, is .04 against .31. The paper reads this abstention as a safety mechanism with aligned evidence, and concludes that sub-claim macro-F1 alone is a poor predictor of downstream success.

Takeaways