CMC training corpus Janes-Tag 2.0

CMC training corpus Janes-Tag 2.0

CLARIN.SI data & tools

Authors: Erjavec, Tomaž ; et al.show everyone
Erjavec, Tomaž ; Fišer, Darja ; Čibej, Jaka ; Arhar Holdt, Špela ; Ljubešić, Nikola ; Zupan, Katja

Item identifier: http://hdl.handle.net/11356/1123

Project URL: http://nl.ijs.si/janes/

Referenced by: http://nl.ijs.si/janes/viri/rocno-oznaceni-korpusi/#Janes-Tag
https://doi.org/10.1007/s10579-018-9425-z

Date issued: 2017-05-14

Type: corpus, text

Size: 2958 texts, 75276 tokens

Language(s): Slovenian

Description: Janes-Tag is a manually annotated corpus of Slovene Computer-Mediated Communication (CMC). It is meant as a gold-standard training and testing dataset for tokenisation, sentence segmentation, word normalisation, morphosyntactic tagging, lemmatisation and named entity annotation of non-standard Slovene. As the corpus has been carefully manually annotated, it is also suitable for detailed linguistic explorations which require highly accurate and reliable annotations. As an update to version 1.2, 2.0 corrects some minor errors and includes named entity annotation. A slightly older version of this corpus is described in: ERJAVEC, Tomaž, ČIBEJ, Jaka, ARHAR HOLDT, Špela, LJUBEŠIĆ, Nikola, FIŠER, Darja. Gold-standard datasets for annotation of Slovene computer-mediated communication. In Proceedings of RASLAN 2016: Recent Advances in Slavonic Natural Language Processing. Brno: Tribun EU, 2016, pp. 29-40, https://nlp.fi.muni.cz/raslan/raslan16.pdf Note that a related corpus, Janes-Norm is also available, cf. http://hdl.handle.net/11356/1084.

Publisher: Jožef Stefan Institute

Subject(s): computer-mediated communication tokenisation word normalisation tagging lemmatisation manual annotation TEI named entities

Collection(s): CLARIN.SI data & tools

This item is replaced by a newer submission:

http://hdl.handle.net/11356/1238

Show full item record

Files in this item

Download all files in item (3.83 MB)

This item is

Publicly Available

and licensed under:
Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)

Name: Janes-Tag.zip
Size: 1.09 MB
Format: application/zip
Description: Corpus in TEI format
MD5: b56533df4d243b2d3c20282083e95118

Download file Preview

File Preview

Janes-Tag
- msd-fslib-sl.xml461 kB
- janes.tag.xml11 kB
- schema
  - tei_janes_doc.html2 MB
  - tei_janes.rng399 kB
  - tei_janes_schema.xml2 kB
  - tei_janes.zip44 kB
  - tei_janes.rnc188 kB
- janes.tag.body.xml6 MB

Name: Janes-Tag.vert.zip
Size: 560.42 KB
Format: application/zip
Description: Derived corpus in vertical format
MD5: 2ce844ec29c48f33099c909c515cd7a0

Download file Preview

File Preview

- janes.tag.vert2 MB
- janes.tag.regi1 kB

Name: Janes-Tag.vert.split.zip
Size: 554.52 KB
Format: application/zip
Description: Corpus split into train, dev and test vertical files
MD5: 8c5e88dc8a2ba848a2bd0c26a4afebeb

Download file Preview

File Preview

- janes.tag.dev.vert250 kB
- janes.tag.train.vert1 MB
- janes.tag.test.vert253 kB

Name: RASLAN16-Janes.pdf
Size: 210.85 KB
Format: PDF
Description: RASLAN'16 paper describing the corpus
MD5: 7487b904191a41f8cb38bbdfe12ba14e

Name: Janes-smernice-v1.0.pdf
Size: 339.69 KB
Format: PDF
Description: Annotation Guidelines - main (in Slovene)
MD5: 39845f938d68b0e3330259eab31c6043

Name: SlovenianNER-slv-v1.1.pdf
Size: 549.4 KB
Format: PDF
Description: Annotation Guidelines - named entities (in Slovene)
MD5: aac3c6f343bc350e1219c00ce232dd87

Name: SlovenianNER-eng-v1.1.pdf
Size: 590.05 KB
Format: PDF
Description: Annotation Guidelines - named entities (in English)
MD5: d3ce678280a4adbe16dfac47a16fc233