Multilingual training dataset for CAP policy topic classification ParlaCAP-train

  Authors

Kuzman Pungeršek, Taja and Ljubešić, Nikola

  Item identifier

http://hdl.handle.net/11356/2093

 Project URL

https://clarinsi.github.io/parlacap/

 Referenced by

https://doi.org/10.48550/arXiv.2602.16516

 Date issued

2026-02-20

 Type

corpus, text

 Size

35579 utterances, 5341147 words

 Language(s)

Basque , Bosnian , Bulgarian , Catalan , Croatian , Czech , Danish , Dutch , English , Estonian , Finnish , French , Galician , German , Hungarian , Icelandic , Italian , Latvian , Modern Greek (1453-) , Norwegian , Polish , Portuguese , Russian , Serbian , Slovenian , Spanish , Swedish , Turkish , Ukrainian

 Description

The multilingual training dataset for CAP policy topic classification ParlaCAP-train is a collection of parliamentary speeches in 29 European languages, automatically annotated with 21 major policy topic labels from the Comparative Agendas Project (CAP) schema, along with an additional label "Other". The texts were annotated by the GPT-4o large language model, accessed via the OpenAI API (https://openai.com/index/hello-gpt-4o/). Evaluation against manually-annotated test datasets in Bosnian, Croatian, English, and Serbian shows strong performance of the GPT-4o model, with macro-F1 scores ranging from 0.63 to 0.74. Moreover, inter-annotator agreement analysis on the test set indicates that the reliability of the LLM as an annotator is comparable to that of human annotators. The ParlaCAP-train dataset contains 35,579 parliamentary speeches, comprising a total of 5,341,147 words. The instances were drawn from all 29 national and regional parliamentary corpora included in the ParlaMint 4.1 collection (http://hdl.handle.net/11356/1912). For each parliament, a random sample of 1,200 speeches was selected. The dataset is divided into a training set (29,779 instances) and a development set (5,800 instances). The training set includes 1,000 speeches from each ParlaMint corpus, along with 779 additional Public Lands instances to improve the representation of this otherwise rare category. The development set contains 200 speeches per ParlaMint corpus. The splits are stratified by topic label. The dataset uses the 21 major CAP policy topics (https://www.comparativeagendas.net/pages/master-codebook): "Agriculture", "Civil Rights", "Culture", "Defense", "Domestic Commerce", "Education", "Energy", "Environment", "Foreign Trade", "Government Operations", "Health", "Housing", "Immigration", "International Affairs", "Labor", "Law and Crime", "Macroeconomics", "Public Lands", "Social Welfare", "Technology", "Transportation", along with an additional category, "Other". Detailed label descriptions and information on their distribution in the dataset are provided in the accompanying README file. The ParlaCAP-train dataset is distributed in JSONL format. Each entry includes the speech text and the following metadata: document and speech ID, session date, source parliamentary corpus, speaker name and role, speech length, predicted CAP category, data split (train or dev), and an indicator specifying whether the instance was added during the augmentation step to increase the representation of the Public Lands label. Further details on the file format and metadata fields are provided in the README file. This dataset was used to develop the ParlaCAP policy topic classifier (https://www.doi.org/10.57967/hf/6684), a fine-tuned Transformer-based XLM-R-Parla BERT-like model (https://huggingface.co/classla/xlm-r-parla). The classifier can be applied to any language covered by the XLM-RoBERTa pretraining corpus.

 Publisher

Jožef Stefan Institute

 Acknowledgement

OSCARS project ParlaCap "Comparing agenda settings across parliaments via the ParlaMint dataset"

ARIS (Slovenian Research and Innovation Agency) GC-0002 "LLM4DH: Large Language Models for Digital Humanities"

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

Institute of Contemporary History DARIAH "DARIAH-SI"

 Subject(s)

parliamentary debates UK Parliament Portuguese Parliament Basque Parliament Estonian Parliament Spanish Parliament Finnish Parliament Ukrainian Parliament Swedish Parliament Serbian Parliament Norwegian Parliament Greek Parliament Galician Parliament Catalonian Parliament Bosnian Parliament Austrian Parliament French Parliament Slovenian Parliament Polish Parliament Croatian Parliament Bulgarian Parliament Latvian Parliament Hungarian Parliament Italian Parliament Turkish Parliament Dutch Parliament Danish Parliament Belgian Parliament Icelandic Parliament Czech Parliament topic classification topic Comparative Agendas Project CAP CAP topic policy topic agenda-setting

 Collection(s)

CLARIN.SI data & tools

Show full item record