Slovene Natural Language Inference Dataset SI-NLI

  Avtorji

Klemen, Matej ; Žagar, Aleš ; Čibej, Jaka and Robnik-Šikonja, Marko

  Identifikator vnosa

http://hdl.handle.net/11356/1707

 Datum objave

2022-11-13

 Vrsta

corpus, text

 Velikost

5937 units

 Jezik(i)

Slovenian

 Opis

SI-NLI (Slovene Natural Language Inference Dataset) contains 5,937 human-created Slovene sentence pairs (premise and hypothesis) that are manually labeled with the labels "entailment", "contradiction", and "neutral". We created the dataset using sentences that appear in the Slovenian reference corpus ccKres (http://hdl.handle.net/11356/1034). Annotators were tasked to modify the hypothesis in a candidate pair in a way that reflects one of the labels. The dataset is balanced since the annotators created three modifications (entailment, contradiction, neutral) for each candidate sentence pair. The dataset is split into train, validation, and test sets, with sizes of 4,392, 547, and 998. We used Slovenian pre-trained language models to create splits, thereby ensuring that difficult and easy instances are evenly distributed in all three subsets. The dataset is released in a tabular TSV format. The README.txt file contains a description of the attributes. Only the hypothesis and premise are given in the test set (i.e. no annotations) since SI-NLI is integrated into the Slovene evaluation framework SloBENCH (https://slobench.cjvt.si/). If you use the dataset to train your models, please consider submitting the test set predictions to SloBENCH to get the evaluation score and see how it compares to others.

 Izdajatelj

Faculty of Computer and Information Science, University of Ljubljana

 Zahvala

Jožef Stefan Institute CLARIN "CLARIN.SI"

 Ključne besede

natural language inference

 Zbirke

CLARIN.SI data & tools

Prikaži polni zapis vnosa