Word-sense disambiguation corpus SloDicWSD 1.0

  Authors

Škvorc, Tadej and Robnik-Šikonja, Marko

  Item identifier

http://hdl.handle.net/11356/2008

 Date issued

2025-01-24

 Type

corpus, text

 Size

758 entries, 110685 sentences, 2029862 words

 Language(s)

Slovenian

 Description

SloDicWSD is a Slovene word-sense disambiguation (WSD) corpus generated from data contained in SSKJ (Slovar slovenskega knjižnega jezika, the largest dictionary of standard Slovene). The corpus is an automatically constructed WSD dataset based on the sense inventory from the SSKJ dictionary and consists of SSKJ dictionary use-case examples converted to complete sentences using GPT-3.5 Turbo (https://platform.openai.com/docs/models#gpt-3-5-turbo). We limited the corpus to the top 758 lemmas present in the Slovene part of the Elexis-WSD dataset (http://hdl.handle.net/11356/1842). For each lemma, we extracted every usage example from the SSKJ dictionary and labeled it with the matching sense. As these usage examples are likely too short to be useful for the WSD task, we extended them using GPT-3.5. We automatically filtered sentences that contain one of the two errors: 1. The original dictionary lemma was not present in the full sentence. While we prompted GPT-3.5 to generate complete sentences by extending existing examples, GPT-3.5 sometimes omitted the original lemma. 2. The generated sentence was identical to one of the already generated sentences. Thesentences generated by GPT-3.5 are not guaranteed to be unique; therefore, we discarded duplicates.

 Publisher

Faculty of Computer and Information Science, University of Ljubljana

 Acknowledgement

European Union C3.K8.IB "PoVeJMo: Adaptive Natural Language Processing with Large Language Models - Co-financing of research innovation projects in support of green transition and digitalisation"

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

ARRS (Slovenian Research Agency) L2-50070 "Embeddings-based techniques for Media Monitoring Applications"

ARIS (Slovenian Research and Innovation Agency) GC-0002 "LLM4DH: Large Language Models for Digital Humanities"

ARIS (Slovenian Research and Innovation Agency) V5-2297 "Media Landscape in Slovenia between Pluralisation and Homogenisation"

 Subject(s)

WSD word sense disambiguation GPT

 Collection(s)

CLARIN.SI data & tools

Show full item record