Keyword extraction datasets for Croatian, Estonian, Latvian and Russian 1.0

  Avtorji

Koloski, Boshko ; Pollak, Senja ; Škrlj, Blaž and Martinc, Matej

  Identifikator vnosa

http://hdl.handle.net/11356/1403

 URL projekta

http://embeddia.eu/

 Dokumentirano v

https://www.aclweb.org/anthology/2021.hackashop-1.4.pdf

 Datum objave

2021-06-04

 Vrsta

corpus, text

 Velikost

2000000 bytes

 Jezik(i)

Croatian , Estonian , Latvian , Russian

 Opis

EACL Hackashop Keyword Challenge Datasets In this repository you can find ids of articles used for the keyword extraction challenge at EACL Hackashop on News Media Content Analysis and Automated Report Generation (http://embeddia.eu/hackashop2021/). The article ids can be used to generate train-test split used in paper: Koloski, B., Pollak, S., Škrlj, B., & Martinc, M. (2021). Extending Neural Keyword Extraction with TF-IDF tagset matching. In: Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation, Kiev, Ukraine, pages 22–29. Train and test splits are provided for Latvian, Estonian, Russian and Croatian. The articles with the corresponding ID-s can be extracted from the following datasets: - Estonian and Russian (use the eearticles2015-2019 dataset): https://www.clarin.si/repository/xmlui/handle/11356/1408 - Latvian: https://www.clarin.si/repository/xmlui/handle/11356/1409 - Croatian: https://www.clarin.si/repository/xmlui/handle/11356/1410 dataset_ids folder is organized in the following way: - latvian – containing latvian_train.json: a json file with ids from train articles to replicate the data used in Koloski et al. (2020), the latvian_test.json: a json file with ids from test articles to replicate the data - estonian – containing estonian_train.json: a json file with ids from train articles to replicate the data used in Koloski et al. (2020), the estonian_test.json: a json file with ids from test articles to replicate the data - russian – containing russian_train.json: a json file with ids from train articles to replicate the train data used in Koloski et al. (2020), the russian_test.json: a json file with ids from test articles to replicate the data - croatian - containing croatian_id_train.tsv file with sites and ids (note that just ids are not unique across dataset, therefore site information also needs to be included to obtain a unique article identifier) of articles in the train set, and the croatian_id_test.tsv file with sites and ids of articles in the test set. In addition, scripts are provided for extracting articles (see folder parse containing scripts parse.py and build_croatian_dataset.py, requirements for scripts are pandas and bs4 Python libraries): parse.py is used for extraction of Estonian, Russian and Latvian train and test datasets: Instructions: ESTONIAN-RUSSIAN 1) Retrieve the data ee_articles_2015_2019.zip 2) Create a folder 'data' and subfolder 'ee' 3) Unzip them in the 'data/ee' folder To extract train/test Estonian articles: run function 'build_dataset(lang="ee", opt="nat")' in the parse.py script To extract train/test Russian articles: run function 'build_dataset(lang="ee", opt="rus")' in the parse.py script LATVIAN: 1) Retrieve the latvian data 2) Unzip it in 'data/lv' folder 3) To extract train/test Latvian articles: run function 'build_dataset(lang="lv", opt="nat")' in the parse.py script build_croatian_dataset.py is used for extraction of Croatian train and test datasets: Instructions: CROATIAN: 1) Retrieve the Croatian data (file 'STY_24sata_articles_hr_PUB-01.csv') 2) put the script 'build_croatian_dataset.py' in the same folder as the extracted data and run it (e.g., python build_croatian_dataset.py). For additional questions: {Boshko.Koloski,Matej.Martinc,Senja.Pollak}@ijs.si

 Izdajatelj

Ekspress Meedia Group

 

Styria Media Group

 Zahvala

European Union EC/H2020/825153 "EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media"

 Ključne besede

keyword extraction news corpus Croatian news articles Estonian news articles Latvian news articles Russian news articles

 Zbirke

CLARIN.SI data & tools

Prikaži polni zapis vnosa