Slovene instruction-following safety dataset for large language models GaMS-Instruct-SAFE 0.5

  Authors

Čibej, Jaka ; et al.show everyone

Čibej, Jaka ; Kos, Sara ; Kastelic, Maja ; Gabrovšek, Dejan ; Trojar, Mitja ; Ježovnik, Janoš ; Bizjak Končar, Aleksandra ; Krvina, Domen ; Petric Žižić, Špela ; Divjak Race, Duša ; Vreš, Domen

  Item identifier

http://hdl.handle.net/11356/2218

 Project URL

http://cjvt.si/povejmo

 Referenced by

https://zenodo.org/records/13912515

 Date issued

2026-06-01

 Type

corpus, text

 Size

501 units

 Language(s)

Slovenian

 Description

GaMS-Instruct-SAFE is a an instruction-following safety dataset designed to fine-tune Slovene large language models to provide safe responses (i.e. to train them to refuse responding to prompts that could lead to physical, economic or psychological harm). It consists of pairs of prompts and responses with various safety topics (e.g. sexual harassment, terrorism, violent crime, drugs). The prompts were written by human annotators using LabelStudio (Tkachenko et al. 2025) based on provided set of criteria (such as topic, expected prompt length, language standardness, different jailbreak strategies) to make the dataset as varied as possible (see Čibej 2024 for more details). In version 0.5, the responses to the prompts were generated using GaMS-27B-Instruct-Nemotron (https://huggingface.co/cjvt/GaMS-27B-Instruct-Nemotron). Only prompt-response pairs in which the model refused to cooperate were included. More responses will be added in future versions. The annotations for this dataset were created using Label Studio, open-source data labeling software developed by Heartex (Tkachenko et al. 2025). References: Čibej, Jaka, 2024: First steps toward the compilation of a safety dataset for Slovene large language models. Jezikovne tehnologije in digitalna humanistika. https://repozitorij.uni-lj.si/IzpisGradiva.php?lang=slv&id=164271 Tkachenko, Maxim, Mikhail Malyuk, Andrey Holmanyuk, Nikolai Liubimov, 2025: Label Studio: Data labeling software. https://github.com/HumanSignal/label-studio

 Publisher

Faculty of Computer and Information Science, University of Ljubljana

 Acknowledgement

ARIS (Slovenian Research and Innovation Agency) NOO "PoVeJMo research project (Adaptive Natural Language Processing with Large Language Models)"

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

 Subject(s)

instruction following dataset large language models safety

 Collection(s)

CLARIN.SI data & tools

Show full item record