Training corpus jos1M 1.2

  Authors

Erjavec, Tomaž ; Krek, Simon and Dobrovoljc, Kaja

  Item identifier

http://hdl.handle.net/11356/1213

 Project URL

http://nl.ijs.si/jos/jos1M-en.html

 Referenced by

http://www.lrec-conf.org/proceedings/lrec2010/summaries/139.html

 Date issued

2019-02-13

 Type

corpus, text

 Size

999961 words, 1182814 tokens, 2564 texts

 Language(s)

Slovenian

 Description

The jos1M corpus contains 1 million words of sampled paragraphs from the Gigafida corpus. It is meant to serve as a training corpus for word-level tagging of Slovene. This silver-standard corpus is annotated for morphosyntactic descriptions and lemmas with about one fourth of the more problematic annotations hand-validated. The morphosyntactic descriptions are given in both the JOS/MULTEXT-East framework (http://nl.ijs.si/ME/V6/msd/), as well as in the framework of Universal Dependencies for Slovene (https://universaldependencies.org/treebanks/sl_ssj/index.html). The corpus is available in source TEI XML with the MSDs in English or Slovene and in the derived vertical format, used by CQP and (no)Sketch Engine concordancers and in CONLL-U, used by Universal Dependencies. Note that the corpus does not contain syntactic dependencies. The texts or paragraphs of the jos1M corpus overlap with this of the ssj500k annotated corpus (http://hdl.handle.net/11356/1210), but the latter has been fully manually annotated, as well as having its tokenisation and sentence segmentation corrected. The texts and paragraphs in the jos1M corpus are marked if they are also included in ssj500k, while the CONLL-U is also split into the part that is included in ssj500k and that which is not. The latter can serve as an additional training set for morphosyntactic tagging and lemmatisation to ssj500k.

 Publisher

Jožef Stefan Institute

 Acknowledgement

ARRS (Slovenian Research Agency) J2-9180 "Linguistic annotation of Slovene"

EU FP6 033917 "SMART “Statistical Multilingual Analysis for Retrieval and Translation”"

Ministry of Higher Education, Science and Technology European Fund for Regional Development "Mobile reader for blind and sight impaired persons"

ARRS (Slovenian Research Agency) P2-103 "Knowledge Technologies"

Jožef Stefan Institute CLARIN "CLARIN.SI"

ARRS (Slovenian Research Agency) P6-0411 "Language Resources and Technologies for Slovene"

 Subject(s)

part-of-speech tagging lemmatisation manual annotation TEI CONLL-U

 Collection(s)

CLARIN.SI data & tools

 Other versions

List all versions  

Show full item record