Show simple item record

 
dc.contributor.author Lenardič, Jakob
dc.contributor.author Čibej, Jaka
dc.contributor.author Arhar Holdt, Špela
dc.contributor.author Erjavec, Tomaž
dc.contributor.author Fišer, Darja
dc.date.accessioned 2022-12-06T13:44:02Z
dc.date.available 2022-12-06T13:44:02Z
dc.date.issued 2022-12-06
dc.identifier.uri http://hdl.handle.net/11356/1733
dc.description Janes-Norm is a manually annotated corpus of Slovene Computer-Mediated Communication (CMC) consisting of about 20,000 short texts (280,000 words), mostly tweets but also blogs, forums and news comments. The corpus is meant as a gold-standard training and testing dataset for tokenisation, sentence segmentation, and word normalisation of non-standard Slovene. The corpus is composed of three parts. One is Janes-Norm 1.2 proper (5,000 texts and 93,000 words, texts to 2016), which has automatically assigned lemmas and morphosyntactic tags. The other two parts constitute the complete Janes-Tag 3.0 (http://hdl.handle.net/11356/1732) corpus, which has manually annotated morphosyntactic tagging, lemmatisation and named entity annotation (15,000 texts and 20,000 words). One part of Janes-Tag 3.0 is the older Janes-Tag 2.1 (texts to 2016) and the newer Janes-RSDO (tweets only, texts up to 2022). Both Janes-Norm and Janes-Tag (but not Janes-RSDO) have texts classified according to their estimated technical (T1-T3) and linguistic (L1-L3) standardness. The data is available in the source TEI encoding and in derived CoNLL-U format. All three parts contain lemmas and JOS/MULTEXT-East morphosyntactic descriptions, while Janes-Tag and Janes-RSDO also contain Universal Dependencies morphological features, and Janes-Tag also named entity annotations. Compared to the previous version, this one corrects some capitalisation errors in normalised words of Janes-Norm, updates the encoding, and adds Janes-RSDO. The first version of this corpus is described in: FIŠER, Darja, LJUBEŠIĆ, Nikola, ERJAVEC, Tomaž. 2018. The Janes project: language resources and tools for Slovene user generated content. Language Resources & Evaluation. https://rdcu.be/7RX4
dc.language.iso slv
dc.publisher Jožef Stefan Institute
dc.relation.isreferencedby https://nl.ijs.si/janes/viri/rocno-oznaceni-korpusi/#Janes-Norm
dc.relation.isreferencedby https://doi.org/10.1007/s10579-018-9425-z
dc.relation.replaces http://hdl.handle.net/11356/1084
dc.rights Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
dc.rights.uri https://creativecommons.org/licenses/by-sa/4.0/
dc.rights.label PUB
dc.source.uri https://nl.ijs.si/janes/
dc.subject computer-mediated communication
dc.subject tokenisation
dc.subject word normalisation
dc.subject manual annotation
dc.subject TEI
dc.title CMC training corpus Janes-Norm 3.0
dc.type corpus
metashare.ResourceInfo#ContentInfo.mediaType text
has.files yes
branding CLARIN.SI data & tools
contact.person Jakob Lenardič jakob.lenardic@inz.si Institute of Contemporary History
sponsor ARRS (Slovenian Research Agency) J6-6842 JANES: Resources, Tools and Methods for the Research of Nonstandard Internet Slovene nationalFunds
sponsor ARRS (Slovenian Research Agency) P2-103 Knowledge Technologies nationalFunds
sponsor Swiss National Science Foundation 160501 ReLDI Other
sponsor Ministry of Culture C3340-20-278001 Development of Slovene in a Digital Environment Other
size.info 19771 texts
size.info 283258 words
size.info 327448 tokens
files.count 2
files.size 12750894
featuredService.kontext search|https://www.clarin.si/kontext/query?corpname=janes_norm30
featuredService.noske search|https://www.clarin.si/ske/#dashboard?corpname=janes_norm30&struct_attr_stats=1


 Files in this item

 Download all files in item (12.16 MB)
This item is
Publicly Available
and licensed under:
Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Distributed under Creative Commons Attribution Required Share Alike
Icon
Name
Janes-Norm.3.0.TEI.zip
Size
3.69 MB
Format
application/zip
Description
Corpus in TEI format
MD5
39ac6cc230fc6229480eeed00e433695
 Download file  Preview
 File Preview  
  • Janes-Norm.3.0.TEI
    • janes-norm.xml5 MB
    • janes-rsdo.xml14 MB
    • Janes-Norm.3.0.xml17 kB
    • janes-tag.xml7 MB
    • schema
      • tei_clarin_example.xml48 kB
      • tei_clarin.rnc321 kB
      • tei_clarin_schema.xml70 kB
      • README.md525 B
      • tei_clarin.rng683 kB
    • 00README.txt1 kB
Icon
Name
Janes-Norm.3.0.CoNLL-U.zip
Size
8.47 MB
Format
application/zip
Description
Corpus in CoNLL-U format
MD5
d5f02ff0c6a7ca43ccf138760b5654d7
 Download file  Preview
 File Preview  
  • Janes-Norm.3.0.CoNLL-U
    • janes-tag.jos.conllu7 MB
    • janes-tag.ud.conllu5 MB
    • janes-rsdo.jos.connlu14 MB
    • janes-norm.jos.conllu9 MB
    • tei2conllu.xsl26 kB
    • janes-rsdo.ud.connlu11 MB
    • 00README.txt1 kB
    • janes-norm.ud.conllu5 MB

Show simple item record