The Alignment Bottleneck in Decomposition-Based Claim Verification

Research setting

Claim verification decides whether a claim is true or false, given evidence such as news articles. A complex claim packs several facts into one text, like the tweet in the figure above. Claim decomposition splits a complex claim into sub-claims, and each sub-claim holds one fact. Each sub-claim gets a veracity label: true (T), false (F), or unverified (U). A model then combines the sub-claims and their labels into one verdict for the whole claim.

The evidence for each sub-claim can come in two forms. In claim-level evidence, every sub-claim sees the same evidence collected for the whole claim. In aligned evidence, each sub-claim sees its own evidence span, the few sentences that justify its label.

Motivation

Studies that test decomposition for fact-checking report conflicting results. Some find it unnecessary for LLMs, while others find that its effect depends on the decomposition method. The paper argues that two overlooked factors explain the conflict: how evidence is aligned with each sub-claim, and whether the sub-claim labels are reliable. Testing these factors needs real-world complex claims with evidence annotated for each sub-claim.

Objective. Build a dataset of real-world complex claims with human-annotated evidence for each sub-claim, and measure when decomposition helps claim verification as evidence alignment and sub-claim label quality change.

Approach

The paper first builds a dataset of complex claims from PHEME, a collection of rumours posted on Twitter during five breaking news events. It has 399 complex claims and 1169 sub-claims, with an agreement (Bennett’s S) of 0.81 on 300 double-annotated sub-claims.

The paper then gives one claim verifier, Qwen3-14B prompted zero-shot, different inputs, always together with the claim. Each tab shows one setup and the inputs it receives.

Results

Claim verification is measured with macro-F1, the F1 score averaged over the true and false classes, and with balanced accuracy, the accuracy averaged over classes. Scores are the mean of three runs (six for ablations), and gains over Vanilla are tested with a paired bootstrap and McNemar’s test.

• +0.0625macro-F1 over Vanilla with Oracle SAE, PHEMEPlus
• −0.0672macro-F1 against Vanilla with Ablation SRE, MMM-Fact
• 0.4335macro-F1 of Qwen Noisy SRE, PHEMEPlus

Only aligned evidence with gold labels (Oracle SAE) gives a significant gain over Vanilla: +0.0625 macro-F1 and +0.0486 balanced accuracy, both with bootstrap p = 0.0000 (Table 2). Repeated claim evidence (Oracle SRE) gives +0.0229 macro-F1, with bootstrap p = 0.124 and McNemar p = 0.0227. Aligned spans without labels (Ablation SAE) lose 0.0783 macro-F1 against Oracle SAE (Table 3), so granular evidence helps only with correct sub-claim labels.

Macro-F1 of claim verification without sub-claim labels on two other fact-checking datasets. COVID-Fact has health claims with one evidence source each, and MMM-Fact covers several domains with 6 to 10 evidence sources per claim. Δ is the change against Vanilla, and p is the paired bootstrap p-value. Best values in bold, as in the paper. Source: Table 2 of the paper.

On other domains, decomposition with repeated claim evidence does not help. On COVID-Fact the drop is not significant, while on MMM-Fact it is significant, also under McNemar’s test (p = 0.0000).

Predicted labels lower performance in every setup. With aligned spans the drop is moderate (−0.0304 macro-F1 for Qwen Noisy SAE), while with repeated claim evidence both models fall to about 0.44.

As sub-claim predictors, Qwen3-14B reaches 56.94% macro-F1 and the GNN 45.79% (Table 4), but they also make different errors (Table 5). The GNN abstains with U on 35.0% of sub-claims against 15.8%, and its refutation recall, the share of false sub-claims it finds, is .04 against .31. The paper reads this abstention as a safety mechanism with aligned evidence, and concludes that sub-claim macro-F1 alone is a poor predictor of downstream success.

Takeaways

1. Decomposition needs aligned evidence.Sub-claims with their own evidence spans and gold labels give a significant gain of +0.0625 macro-F1 over verifying the whole claim at once.
2. Repeated claim evidence does not help.Pairing each sub-claim with the same claim evidence brings no significant gain on PHEMEPlus and lowers macro-F1 on MMM-Fact by 0.0672.
3. The kind of sub-claim error matters.Noisy labels hurt much less with aligned spans, and a model that abstains with Unverified behaves differently from one that commits to wrong labels.

@inproceedings{Akhter_2026,
  series = {IJCAI-2026},
  title = {The Alignment Bottleneck in Decomposition-Based Claim Verification},
  url = {https://doi.org/10.24963/ijcai.2026/608},
  DOI = {10.24963/ijcai.2026/608},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  author = {Akhter, Mahmud Elahi and Ruggeri, Federico and Bilal, Iman Munire and Procter, Robert and Liakata, Maria},
  year = {2026},
  month = {September},
  pages = {5458–5467},
  collection = {IJCAI-2026}
}

• Claim Verification
• Claim Decomposition
• Fact-Checking
• Large Language Models