Show simple item record

 
dc.contributor.author Ljubešić, Nikola
dc.date.accessioned 2020-09-15T09:04:33Z
dc.date.available 2020-09-15T09:04:33Z
dc.date.issued 2020-09-15
dc.identifier.uri http://hdl.handle.net/11356/1350
dc.description The model for lemmatisation of non-standard Slovenian was built with the CLASSLA-StanfordNLP tool (https://github.com/clarinsi/classla-stanfordnlp) by training on the ssj500k training corpus (http://hdl.handle.net/11356/1210) and the Janes-Tag corpus (http://hdl.handle.net/11356/1238), using the Sloleks inflectional lexicon (http://hdl.handle.net/11356/1230). These corpora were additionally augmented for handling missing diacritics by repeating parts of the corpora with diacritics removed. The estimated F1 of the lemma annotations is ~98.86. The difference to the previous version of the lemmatizer is that now it relies solely on XPOS annotations, and not on a combination of UPOS, FEATS (lexicon lookup) and XPOS (lemma prediction) annotations.
dc.language.iso slv
dc.publisher Jožef Stefan Institute
dc.relation.isreferencedby http://dx.doi.org/10.18653/v1/W19-3704
dc.relation.replaces http://hdl.handle.net/11356/1338
dc.rights Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
dc.rights.uri https://creativecommons.org/licenses/by-sa/4.0/
dc.rights.label PUB
dc.source.uri https://github.com/clarinsi/classla-stanfordnlp
dc.subject lemmatisation
dc.subject computer-mediated communication
dc.subject language model
dc.title The CLASSLA-StanfordNLP model for lemmatisation of non-standard Slovenian 1.1
dc.type toolService
metashare.ResourceInfo#ContentInfo.detailedType tool
metashare.ResourceInfo#ResourceComponentType#ToolServiceInfo.languageDependent true
has.files yes
branding CLARIN.SI data & tools
contact.person Nikola Ljubešić nikola.ljubesic@ijs.si Jožef Stefan Institute
sponsor ARRS (Slovenian Research Agency) P6-0411 Language Resources and Technologies for Slovene nationalFunds
sponsor ARRS (Slovenian Research Agency) N6-0099 LiLaH: Linguistic Landscape of Hate Speech nationalFunds
files.count 1
files.size 39919841


 Files in this item

This item is
Publicly Available
and licensed under:
Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
Distributed under Creative Commons Attribution Required Share Alike
Icon
Name
sl_all_Sloleks_lemmatizer.pt.zip
Size
38.07 MB
Format
application/zip
Description
Language model
MD5
b08011a0d0370176d390badc3fafd6b2
 Download file  Preview
 File Preview  
    • sl_all_Sloleks_lemmatizer.pt165 MB

Show simple item record