ALMA-AI | Workshop LLM: a debate on technical experiences
Workshop Abstract
The spread of Large Language Models across every area of human activity makes it necessary to assess the state of the art, limits, obstacles, and barriers of this powerful technology. Rapid updates to these technologies require broader research goals and methods, and the reasoning capabilities of agentic AI raise new challenges. It is increasingly important to assess where these technologies add real value and where, also considering the resources they need, more traditional approaches are preferable. This raises the question of how to compare models (benchmarking). Technical aspects intersect with ethical, legal, sustainability, effectiveness, and explainability concerns about reasoning steps. The workshop investigates these themes, with particular attention to law, where language is a founding pillar of the discipline. The debate also helps define what institutions such as parliaments and public administrations can expect to gain, without ignoring risks and false illusions.
Federico Ruggeri’s speech
The talk addresses three main aspects of LLMs and reasoning capabilities. First, we discuss what kind of reasoning type LLMs are tested for. The short answer is that in the majority of cases, it is unclear which reasoning type(s) is (are) considered. Second, we discuss to what extent LLMs perform reasoning. Some view LLMs as stochastic parrots, while others believe they acquire true reasoning capabilities. Third, we show how reasoning and argumentation are tightly connected and discuss how argumentation is being progressively used as a way to assess reasoning capabilities in LLMs.