Slovenian datasets for contextual synonym and antonym detection

  Avtorji

Pegan, Jasmina ; et al.prikaži vse

Pegan, Jasmina ; Robnik-Šikonja, Marko ; Kosem, Iztok ; Gantar, Polona ; Ponikvar, Primož ; Laskowski, Cyprian

  Identifikator vnosa

http://hdl.handle.net/11356/1694

 URL projekta

https://github.com/jasminapegan/antonym_detection

 Dokumentirano v

https://repozitorij.uni-lj.si/IzpisGradiva.php?id=141456

 Datum objave

2022-10-26

 Vrsta

lexicalConceptualResource, text

 Velikost

79229 entries

 Jezik(i)

Slovenian

 Opis

Slovenian datasets for contextual synonym and antonym detection can be used for training machine learning classifiers as described in the MSc thesis of Jasmina Pegan "Semantic detection of synonyms and antonyms with contextual embeddings" (https://repozitorij.uni-lj.si/IzpisGradiva.php?id=141456). Datasets contain example pairs of synonyms and antonyms in contexts together with additional information on a sense pair. Candidates for synonyms and antonyms were retrieved from the dataset created in the BSc thesis of Jasmina Pegan "Antonym detection with word embeddings" (https://repozitorij.uni-lj.si/IzpisGradiva.php?id=110533). Example sentences were retrieved from The comprehensive Slovenian-Hungarian dictionary (VSMS) (https://www.clarin.si/repository/xmlui/handle/11356/1453). Each dataset is class balanced and contains an equal amount of examples and counterexamples. An example is a pair of example sentences where the two words are synonyms/antonyms. A counterexample is a pair of example sentences where two words are not synonyms/antonyms. Note that a word pair can be synonymous or antonymous in some sense of the two words (but not in the given context). Datasets are divided into two categories, datasets for synonyms and datasets for antonyms. Each category is further divided into base and updated datasets. These contain three dataset files: train, validation and test dataset. Base datasets include only manually-reviewed sense pairs. These are generated from all pairs of VSMS sense examples for all confirmed pairs of antonym and synonym senses. Updated datasets include automatically generated sense pairs while constraining the maximal number of examples per word. In this way, the dataset is more balanced word-wise, but is not fully manually-reviewed and contains less accurate data. A single dataset entry contains the information on the base word, followed by data on synonym/antonym candidate. The last column discerns whether the sense pair is a pair of synonyms/antonyms or not. More details on this can be found inside the included README file.

 Izdajatelj

Centre for Language Resources and Technologies, University of Ljubljana

 Zahvala

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

Republic of Slovenia, Ministry of Culture 3340-21-722002 "Upgrading fundamental dictionary resources and databases of CJVT UL"

 Ključne besede

synonyms antonyms dataset WSD WSI

 Zbirke

CLARIN.SI data & tools

Prikaži polni zapis vnosa