Benchmark

Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification
RECV, a benchmark of real-world claims labeled by reasoning type, shows that LLMs verify claims by deduction well but fail when abduction is needed.
John Dougrez-Lewis, Mahmud Elahi Akhter, Federico Ruggeri, Sebastian Löbbers, Yulan He, Maria Liakata
Cite DOI URL
Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification
LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain
LEXTREME gathers 11 legal datasets in 24 languages into one benchmark with a single score, and shows that legal language understanding remains hard.
Joel Niklaus, Veton Matoshi, Pooja Rani, Andrea Galassi, Matthias Stürmer, Ilias Chalkidis
Cite DOI URL