| dc.contributor.author | Koloski, Boshko |
| dc.contributor.author | Pollak, Senja |
| dc.contributor.author | Škrlj, Blaž |
| dc.contributor.author | Martinc, Matej |
| dc.date.accessioned | 2021-06-04T07:03:25Z |
| dc.date.available | 2021-06-04T07:03:25Z |
| dc.date.issued | 2021-06-04 |
| dc.identifier.uri | http://hdl.handle.net/11356/1403 |
| dc.description | EACL Hackashop Keyword Challenge Datasets In this repository you can find ids of articles used for the keyword extraction challenge at EACL Hackashop on News Media Content Analysis and Automated Report Generation (http://embeddia.eu/hackashop2021/). The article ids can be used to generate train-test split used in paper: Koloski, B., Pollak, S., Škrlj, B., & Martinc, M. (2021). Extending Neural Keyword Extraction with TF-IDF tagset matching. In: Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation, Kiev, Ukraine, pages 22–29. Train and test splits are provided for Latvian, Estonian, Russian and Croatian. The articles with the corresponding ID-s can be extracted from the following datasets: - Estonian and Russian (use the eearticles2015-2019 dataset): https://www.clarin.si/repository/xmlui/handle/11356/1408 - Latvian: https://www.clarin.si/repository/xmlui/handle/11356/1409 - Croatian: https://www.clarin.si/repository/xmlui/handle/11356/1410 dataset_ids folder is organized in the following way: - latvian – containing latvian_train.json: a json file with ids from train articles to replicate the data used in Koloski et al. (2020), the latvian_test.json: a json file with ids from test articles to replicate the data - estonian – containing estonian_train.json: a json file with ids from train articles to replicate the data used in Koloski et al. (2020), the estonian_test.json: a json file with ids from test articles to replicate the data - russian – containing russian_train.json: a json file with ids from train articles to replicate the train data used in Koloski et al. (2020), the russian_test.json: a json file with ids from test articles to replicate the data - croatian - containing croatian_id_train.tsv file with sites and ids (note that just ids are not unique across dataset, therefore site information also needs to be included to obtain a unique article identifier) of articles in the train set, and the croatian_id_test.tsv file with sites and ids of articles in the test set. In addition, scripts are provided for extracting articles (see folder parse containing scripts parse.py and build_croatian_dataset.py, requirements for scripts are pandas and bs4 Python libraries): parse.py is used for extraction of Estonian, Russian and Latvian train and test datasets: Instructions: ESTONIAN-RUSSIAN 1) Retrieve the data ee_articles_2015_2019.zip 2) Create a folder 'data' and subfolder 'ee' 3) Unzip them in the 'data/ee' folder To extract train/test Estonian articles: run function 'build_dataset(lang="ee", opt="nat")' in the parse.py script To extract train/test Russian articles: run function 'build_dataset(lang="ee", opt="rus")' in the parse.py script LATVIAN: 1) Retrieve the latvian data 2) Unzip it in 'data/lv' folder 3) To extract train/test Latvian articles: run function 'build_dataset(lang="lv", opt="nat")' in the parse.py script build_croatian_dataset.py is used for extraction of Croatian train and test datasets: Instructions: CROATIAN: 1) Retrieve the Croatian data (file 'STY_24sata_articles_hr_PUB-01.csv') 2) put the script 'build_croatian_dataset.py' in the same folder as the extracted data and run it (e.g., python build_croatian_dataset.py). For additional questions: {Boshko.Koloski,Matej.Martinc,Senja.Pollak}@ijs.si |
| dc.language.iso | hrv |
| dc.language.iso | est |
| dc.language.iso | lav |
| dc.language.iso | rus |
| dc.publisher | Ekspress Meedia Group |
| dc.publisher | Styria Media Group |
| dc.relation | info:eu-repo/grantAgreement/EC/H2020/825153 |
| dc.relation.isreferencedby | https://www.aclweb.org/anthology/2021.hackashop-1.4.pdf |
| dc.rights | Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) |
| dc.rights.uri | https://creativecommons.org/licenses/by-nc-nd/4.0/ |
| dc.rights.label | PUB |
| dc.source.uri | http://embeddia.eu/ |
| dc.subject | keyword extraction |
| dc.subject | news corpus |
| dc.subject | Croatian news articles |
| dc.subject | Estonian news articles |
| dc.subject | Latvian news articles |
| dc.subject | Russian news articles |
| dc.title | Keyword extraction datasets for Croatian, Estonian, Latvian and Russian 1.0 |
| dc.type | corpus |
| metashare.ResourceInfo#ContentInfo.mediaType | text |
| has.files | yes |
| branding | CLARIN.SI data & tools |
| contact.person | Matej Martinc matej.martinc@ijs.si Jožef Stefan Institute |
| sponsor | European Union EC/H2020/825153 EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media euFunds info:eu-repo/grantAgreement/EC/H2020/825153 |
| size.info | 2000000 bytes |
| files.count | 1 |
| files.size | 230235 |
Files in this item
This item is
Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
Publicly Available
and licensed under:Creative Commons - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
- Name
- keywords_challenge_datasplits_EE_LV_RU_CRO.zip
- Size
- 224.84 KB
- Format
- application/zip
- Description
- zip file
- MD5
- 074a1985b245b3c067f2a06104c8c769
- keywords_challenge_datasplits_EE_LV_RU_CRO
- readme_keywords_datasplits.txt3 kB
- latvian
- latvian_id_train.json230 kB
- latvian_id_test.json204 kB
- .DS_Store6 kB
- russian
- rus_id_test.json201 kB
- rus_id_train.json243 kB
- croatian
- croatian_id_train.tsv682 kB
- croatian_id_test.tsv77 kB
- estonian
- ee_id_test.json136 kB
- ee_id_train.json188 kB
- parse
- build_croatian_dataset.py1 kB
- parse.py1 kB