Slovene instruction-following dataset for large language models GaMS-Instruct-MED 2.0

Name: Slovene instruction-following dataset for large language models GaMS-Instruct-MED 2.0
License: https://creativecommons.org/licenses/by/4.0/

Tovornik, Robert; Pavlović, Anđela; Radnić, Vuk; Plesnik, Emil; Fabjan, Borut

Slovene instruction-following dataset for large language models GaMS-Instruct-MED 2.0

CLARIN.SI data & tools

Authors: Tovornik, Robert ; Pavlović, Anđela ; Radnić, Vuk ; Plesnik, Emil and Fabjan, Borut

Item identifier: http://hdl.handle.net/11356/2045

Project URL: https://www.cjvt.si/povejmo/en/project/

Date issued: 2025-08-25

Type: corpus, text

Size: 25046 units

Language(s): Slovenian

Description: GaMS-Instruct-MED is an instruction-following dataset designed to fine-tune Slovene large language models to follow instructions in the medical domain. It consists of units of prompts, instrutions and responses from the field of medicine, particularly those pertaining to the use of pharmaceutical drugs and medications. The dataset was generated in several steps (for a more detailed description, please refer to 00README.txt). After consulting with experts from the medical field, a series of prompts was manually compiled containing questions interesting in the context of drug and medication use. For each medication in the PoVeJMo-VeMo-Med 1.0 dataset (http://hdl.handle.net/11356/1983), approximately 10-15 questions were automatically generated using prompt tuning. In version 2.0, the dataset was extended with several other similar datasets for English that were translated into Slovene: MedQuAD, MeQSum, Medication QA, and LiveQA (references are available in 00README.txt). All translations were made automatically using GPT-4.1. The manual validation was made in two phases. In the preparation-evaluation phase, the quality of machine translations were validated on a sample using different machine translation applications (DeepL, OpenAI) to determine the solution with optimal performance. In the second phase, a random sample of 20--40 examples from each translated subset were manually validated (a total of 240 examples). The manual validations were made by two experts from the field of medicine and an expert for dataset compilation. Unlike version 1.0, where the dataset consisted of prompt-response pairs, version 2.0 contains units consisting of three elements (instruction-input-output). The conversion was made using OpenAI GPT-4.1. All final instructions were manually validated by an expert for dataset compilation. Two experts from the field of medicine participated in the design of clinically relevant categories of instructions, the compilation of examples of prompt-response pairs, and the manual validation of test results of the conversion process. Please note that the current version of the dataset (containing 25,046 instruction-input-output units) does not guarantee full clinical accuracy and may contain errors as a consequence of LLM hallucinations.