Slovenian datasets for contextual synonym and antonym detection

  Authors

Pegan, Jasmina ; et al.show everyone

Pegan, Jasmina ; Robnik-Šikonja, Marko ; Kosem, Iztok ; Gantar, Polona ; Ponikvar, Primož ; Laskowski, Cyprian

  Item identifier

http://hdl.handle.net/11356/1694

 Project URL

https://github.com/jasminapegan/antonym_detection

 Referenced by

https://repozitorij.uni-lj.si/IzpisGradiva.php?id=141456

 Date issued

2022-10-26

 Type

lexicalConceptualResource, text

 Size

79229 entries

 Language(s)

Slovenian

 Description

Slovenian datasets for contextual synonym and antonym detection can be used for training machine learning classifiers as described in the MSc thesis of Jasmina Pegan "Semantic detection of synonyms and antonyms with contextual embeddings" (https://repozitorij.uni-lj.si/IzpisGradiva.php?id=141456). Datasets contain example pairs of synonyms and antonyms in contexts together with additional information on a sense pair. Candidates for synonyms and antonyms were retrieved from the dataset created in the BSc thesis of Jasmina Pegan "Antonym detection with word embeddings" (https://repozitorij.uni-lj.si/IzpisGradiva.php?id=110533). Example sentences were retrieved from The comprehensive Slovenian-Hungarian dictionary (VSMS) (https://www.clarin.si/repository/xmlui/handle/11356/1453). Each dataset is class balanced and contains an equal amount of examples and counterexamples. An example is a pair of example sentences where the two words are synonyms/antonyms. A counterexample is a pair of example sentences where two words are not synonyms/antonyms. Note that a word pair can be synonymous or antonymous in some sense of the two words (but not in the given context). Datasets are divided into two categories, datasets for synonyms and datasets for antonyms. Each category is further divided into base and updated datasets. These contain three dataset files: train, validation and test dataset. Base datasets include only manually-reviewed sense pairs. These are generated from all pairs of VSMS sense examples for all confirmed pairs of antonym and synonym senses. Updated datasets include automatically generated sense pairs while constraining the maximal number of examples per word. In this way, the dataset is more balanced word-wise, but is not fully manually-reviewed and contains less accurate data. A single dataset entry contains the information on the base word, followed by data on synonym/antonym candidate. The last column discerns whether the sense pair is a pair of synonyms/antonyms or not. More details on this can be found inside the included README file.

 Publisher

Centre for Language Resources and Technologies, University of Ljubljana

 Acknowledgement

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

Republic of Slovenia, Ministry of Culture 3340-21-722002 "Upgrading fundamental dictionary resources and databases of CJVT UL"

 Subject(s)

synonyms antonyms dataset WSD WSI

 Collection(s)

CLARIN.SI data & tools

Show full item record