Pragmatics understanding benchmark for Czech, Slovenian and Croatian PragMega CzeSloCro

 
CLARIN.SI data & tools
  Avtorji
Vintar, Špela ; Brglez, Mojca ; Potočnjak, Mirna ; Žižkova, Hana and Sangawa Hmeljak, Nina
  Identifikator vnosa
http://hdl.handle.net/11356/2261
 Dokumentirano v
https://doi.org/10.63317/4bpncy453r9k
 Datum objave
2026-07-21
 Vrsta
corpus, text
 Velikost
420 items
 Jezik(i)
Croatian , Czech , English , Slovenian
 Opis
PragMega CzeSloCro is a translation and adaptation of a section of the PragMega dataset (Floyd et al., 2026) into Czech, Slovenian and Croatian. The original dataset was manually crafted by psychologists and aimed at discovering whether "pragmatic inferencing" is a result of a single cognitive skill or, on the contrary, of different dissociable skills depending on the type of phenomena encountered. A Slovenian version of the dataset was created first, for which we selected three tasks: Irony, Metaphor, and Humour. These consist of 50, 30, and 25 examples, respectively, or 105 examples in total. The Slovenian benchmark is described in Brglez & Vintar (2026) and is implemented as SloPragMega on the SloBench (https://slobench.cjvt.si) platform. The dataset was then translated into Croatian and Czech by students of Digital Linguistics within a student project, then thoroughly revised by two professional linguists. Due to the highly nuanced and culturally specific nature of the dataset, some tasks were completely rewritten or replaced by more naturally sounding examples in the respective language. For each language, the dataset is divided into 3 subfolders (Metaphor, Irony, Humor), which contain the following files: - stim.csv: The actual localised tasks with possible answers, - stim_en.csv: The original English tasks with possible answers, - stimOrder.csv: Template to create a randomized test with the order of questions and answers reshuffled, - keys.csv: Solutions for the shuffled tasks. References: Brglez, M., & Vintar, S. (2026). From Polyester Girlfriends to Blind Mice: Creating the First Pragmatics Understanding Benchmarks for Slovene. In The Fourth Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL 2026) (pp. 44–54). European Language Resources Association (ELRA). https://doi.org/10.63317/4bpncy453r9k Floyd, S., Gibson, E., Fedorenko, E., & Poliak, M. (2026, January 14). PragMega. https://doi.org/10.17605/OSF.IO/DPGE6
 Izdajatelj
Jožef Stefan Institute
 Ključne besede
pragmatics benchmark metaphor irony humour
 Zbirke
CLARIN.SI data & tools
Prikaži polni zapis vnosa
 

 Datoteke v tem vnosu

To je vnos
Publicly Available
z licenco:
Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Distributed under Creative Commons Attribution Required Share Alike
Icon
Ime
PragMega-EN-SL-HR-CZ.zip
Velikost
152.98 KB
Format
application/zip
Opis
Dataset in CSV format
MD5
d2dfa62d790c252d375d9dce43f02b28
 Prenesi datoteko  Predogled
 Predogled datoteke  
  • PragMega
    • SL
      • HumorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • MetaphorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • IronyText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
    • HR
      • HumorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • MetaphorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • IronyText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
    • PragMegaOrig
      • HumorText
        • keys copy.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • MetaphorText
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • IronyText
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
    • CZ
      • HumorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • MetaphorText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
      • IronyText
        • stim_en.csv-1 B
        • stimOrder.csv-1 B
        • stim.csv-1 B
        • keys.csv-1 B
  • __MACOSX
    • PragMega
      • ._PragMegaOrig-1 B
      • ._SL-1 B
      • SL
        • HumorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._HumorText-1 B
        • MetaphorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • IronyText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._MetaphorText-1 B
        • ._IronyText-1 B
      • ._HR-1 B
      • HR
        • HumorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._HumorText-1 B
        • MetaphorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • IronyText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._MetaphorText-1 B
        • ._IronyText-1 B
      • ._CZ-1 B
      • PragMegaOrig
        • HumorText
          • ._keys copy.csv-1 B
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
        • ._HumorText-1 B
        • MetaphorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
        • IronyText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
        • ._MetaphorText-1 B
        • ._IronyText-1 B
      • CZ
        • HumorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._HumorText-1 B
        • MetaphorText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • IronyText
          • ._stim.csv-1 B
          • ._stimOrder.csv-1 B
          • ._keys.csv-1 B
          • ._stim_en.csv-1 B
        • ._MetaphorText-1 B
        • ._IronyText-1 B
    • ._PragMega-1 B

Partnerji

Partnerji

Repozitorij

Repozitorij uporablja programsko opremo, ki je bila razvita za LINDAT/CLARIAH-CZ jezikoslovni repozitorij in je dostopna na GitHubu.

CLARIN.SI podpira Ministrstvo za izobraževanje, znanost in šport
v okviru programa "Raziskovalne infrastrukture".