Automatic extraction and collection of legal principles from CJEU decisions on Fiscal State Aid

Mon, 22 Jun 2026 00:00:00 +0000

Research setting

The Court of Justice of the European Union (CJEU) interprets and applies EU law, and its decisions guide judges and lawyers in later cases. Fiscal State Aid is the area of EU law on national tax measures that give some companies an undue advantage, and it was developed through CJEU decisions. A legal principle is a criterion stated by the judges, such as how a rule must be interpreted, excluding literal citations of legislation. In CJEU decisions, each principle is usually contained in a single paragraph, so the task is a binary classification of paragraphs: does this paragraph state a legal principle or not?

As regards that condition, that the advantage must be granted directly or indirectly through State resources,Legal principle it must be recalled thatIndicator measures not involving a transfer of State resources may fall within the concept of “aid”,Legal principle within the meaning ofIndicator Article 107(1) TFEU.Legal principle According to the appellant,Indicator the particular nature of the review of State aid means that, when the court carrying out the review is called on to assess the facts of the case in the light of legal criteria, it must limit itself to satisfying itself that there is no manifest error of assessment.Party argument

Two paragraphs quoted from Section 4.2 of the paper. The first, from Case C-656/15 P (para. 43), states a legal principle and contains two of the word patterns that the annotation guidelines list as indicators. The second, from Case C-83/98 P (para. 24), only reports the argument of a party, which the opening words signal, so it is not a principle.

Motivation

CJEU decisions are long and complex, so finding their principles by hand takes time and legal expertise. Collections of extracted principles could help summarize decisions and retrieve precedents based on the same principles. Earlier studies classified legal principles in common law cases, but no empirical work had addressed the CJEU. The only annotated corpus of CJEU decisions on Fiscal State Aid was built for argument structure, not for principles.

Work	Decisions	What is annotated	CJEU decisions	Collections of principles judged by experts
Shulayeva et al. (2017)	50 common law reports	Facts and legal principles	no	no
Valvoda and Ray (2017)	A well-cited case and the cases citing it	Binding and non-binding principles	no	no
Demosthenes corpus (Grundler et al., 2022)	40 CJEU decisions on Fiscal State Aid	Argument structure	yes	no
This paper This work	64 CJEU decisions on Fiscal State Aid	Legal principles	yes	yes

How the paper relates to the work it discusses in Section 2. Columns list the properties that the paper uses to position itself.

Objective. Build an annotated corpus of CJEU decisions on Fiscal State Aid, find the best way to extract their legal principles automatically, and test whether the extracted paragraphs form collections that experts accept as real.

Approach

The corpus holds all 64 decisions of the CJEU on Fiscal State Aid adopted between 1992 and 2022. Two legal experts with a PhD in Law annotated them. On 124 principles annotated by both, their agreement was Cohen’s κ = 0.87, where 1 is perfect agreement. Only 614 paragraphs state a principle, against 8,977 that do not (Table 1), so the authors rebalance the training data. Oversampling repeats each principle paragraph 12 to 13 times. Data augmentation creates about 5,000 new principle paragraphs by replacing 10% or 40% of their words with related words. An encoder then turns each paragraph into a vector. A classifier, such as a support vector machine (SVM), learns to separate principles from other paragraphs.

This work 8 training sets Original, oversampled, and six augmented.
- AugGV GloVe neighbor
- AugLB LEGAL-BERT guess
- AugWN+GV WordNet and GloVe
3 encoders Paragraph to vector.
- BERT general
- LEGAL-BERT legal text
- Sentence-BERT similarity
3 classifiers SVM, logistic regression, feed-forward network.
72 models Compared with 11 baselines and 3 LLMs.

How the 72 classifiers are trained on the paragraphs of the 38 training decisions (Section 5 and Figure 1 of the paper). Every combination of training set, encoder, and classifier is tried.

Accordingly, although a measure by which the officeReplaced authorities grant certain undertakings favorableReplaced tax treatment, which, although not involving the transfer of state resources, places the recipients in a more favourable financial rightReplaced than other taxpayers, quantitiesReplaced to state aid within the meaning of article 107(1) tfeu, that is not the position as regards advantages resulting from a general measure applicable without distinction to all economic operators, which do not constitute state aid within the meaning of that provision (judgment in p, eu:c:2013:525, paragraph 18).

A paragraph augmented with AugGV, replacing 10% of the words, as printed in Figure 3 of the paper. AugGV replaces a word with the word closest to it in GloVe word embeddings. The original words were public, favourable, position, and amounts. The augmented paragraph keeps the label of the original.

The baselines include random guessing, classifiers on word counts (tf-idf), and a similarity threshold to the known principles. The three large language models (LLMs), Mistralv3, Llama3, and Phi3-mini, answer YES or NO for each paragraph from a prompt that defines a legal principle, with no examples (zero-shot). Lastly, the paragraphs that the best classifier flags in a decision are collated in document order into a collection of principles, as in the schema at the top of this page.

Results

All models were tested on 13 held-out decisions with 113 principle paragraphs and 1,805 other paragraphs (Table 1). Precision is the share of flagged paragraphs that state a principle, recall is the share of principles that are flagged, and F1, from 0 to 1, balances the two.

0.72F1 of BERT + SVM + AugGV (Table 6)
0.54F1 of the best baseline, tf-idf (Table 3)
0.34F1 of the best zero-shot LLM, Llama3 (Table 4)

F1 on the principle class (higher is better)

Random, uniform 0.10
Llama3, zero-shot 0.34
Similarity to principles, BERT 0.50
Tf-idf + feed-forward network 0.54
BERT + SVM, no augmentation This work 0.68
BERT + SVM + AugGV 10% This work 0.72

F1 on the test set (higher is better) for the best system of each kind. Source: Tables 3, 4, and 6 of the paper.

Show as table

System	F1 on the principle class
Random, uniform	0.10
Llama3, zero-shot	0.34
Similarity to principles, BERT	0.50
Tf-idf + feed-forward network	0.54
BERT + SVM, no augmentation (this work)	0.68
BERT + SVM + AugGV 10% (this work)	0.72

BERT with an SVM led the ten best combinations, with precision of 0.80 and recall of 0.65 for the best one (Table 6). Augmentation with AugGV at 10% added 0.039 F1 over the same model without augmentation, although no augmentation method was consistently the best. The general-purpose BERT beat LEGAL-BERT, which reached higher recall but lower precision (Table 5). The zero-shot LLMs flagged most paragraphs as principles: Mistralv3 reached a recall of 1.00 with a precision of 0.10 (Table 4). The paper reports no significance tests.

The generated collections were compared with the expert-made collection of each test decision. ROUGE measures word overlap between two texts, here as F1 averaged over the 13 test decisions.

Collections	ROUGE‑1	ROUGE‑2	ROUGE‑L
This work (BERT + SVM + AugGV)	0.73	0.69	0.67
Llama3, zero-shot	0.42	0.38	0.35
Random paragraphs	0.08	0.07	0.07

F1 of each ROUGE variant, averaged over the test decisions (higher is better). Source: Table 7 of the paper.

Two other legal experts then tried to tell generated collections from expert-made ones, for the 26 validation and test decisions. Seeing one collection at a time, they spotted generated collections with an average F1 of 0.40 (precision 0.56, recall 0.31), so most generated collections passed as original. Seeing both versions side by side, they reached an F1 of 0.66 (precision 0.73, recall 0.62). Their error analysis found false positives on paragraphs about facts or appeal procedure, and false negatives where facts and principles are intertwined.

Takeaways