Text collection for training the BERTić transformer model BERTić-data

Name: Text collection for training the BERTić transformer model BERTić-data
License: https://creativecommons.org/licenses/by-sa/4.0/

Ljubešić, Nikola

Prikaži enostavni zapis vnosa

dc.contributor.author	Ljubešić, Nikola
dc.date.accessioned	2021-05-06T07:44:43Z
dc.date.available	2021-05-06T07:44:43Z
dc.date.issued	2021-05-05
dc.identifier.uri	http://hdl.handle.net/11356/1426
dc.description	The BERTić-data text collection contains more than 8 billion tokens of mostly web-crawled text written in Bosnian, Croatian, Montenegrin or Serbian. The collection was used to train the BERTić transformer model (https://huggingface.co/classla/bcms-bertic). The data consists of web crawls before 2015, i.e. bsWaC (http://hdl.handle.net/11356/1062), hrWaC (http://hdl.handle.net/11356/1064), and srWaC (http://hdl.handle.net/11356/1063); previously unpublished 2019-2020 crawls, i.e. cnrWaC, CLASSLA-bs, CLASSLA-hr, and CLASSLA-sr; the cc100-hr and cc100-sr parts of CommonCrawl (https://commoncrawl.org/); and the Riznica corpus (http://hdl.handle.net/11356/1180). All texts were transliterated to the Latin script. The format of the text collection is one-sentence-per-line, empty-line-as-document-boundary. More details, especially on the applied near-deduplication procedure, can be found in the BERTić paper (https://arxiv.org/pdf/2104.09243.pdf).
dc.language.iso	bos
dc.language.iso	hrv
dc.language.iso	cnr
dc.language.iso	srp
dc.publisher	Jožef Stefan Institute
dc.relation.isreferencedby	https://arxiv.org/abs/2104.09243
dc.rights	Creative Commons - Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)
dc.rights.uri	https://creativecommons.org/licenses/by-sa/4.0/
dc.rights.label	PUB
dc.source.uri	https://huggingface.co/classla/bcms-bertic
dc.subject	web corpus
dc.subject	language model
dc.title	Text collection for training the BERTić transformer model BERTić-data
dc.type	corpus
metashare.ResourceInfo#ContentInfo.mediaType	text
has.files	yes
branding	CLARIN.SI data & tools
demo.uri	https://huggingface.co/classla/bcms-bertic-ner
contact.person	Nikola Ljubešić nikola.ljubesic@ijs.si Jožef Stefan Institute
sponsor	ARRS (Slovenian Research Agency) P6-0411 Language Resources and Technologies for Slovene nationalFunds
sponsor	European Union’s Rights,Equality and Citizenship Programme 875263 IMSyPP - Innovative Monitoring Systems and PreventionPolicies of Online Hate Speech Other
sponsor	ARRS (Slovenian Research Agency) N6-0099 LiLaH: Linguistic Landscape of Hate Speech nationalFunds
sponsor	ARRS (Slovenian Research Agency) J7-8280 FRENK: Resources, methods, and tools for the understanding, identification, and classification of various forms of socially unacceptable discourse in the information society nationalFunds
size.info	8387681518 words
files.count	10
files.size	22694988939