{"id":6349,"date":"2023-06-22T10:38:46","date_gmt":"2023-06-22T10:38:46","guid":{"rendered":"https:\/\/www.clarin.si\/info\/?page_id=6349"},"modified":"2023-06-22T11:15:16","modified_gmt":"2023-06-22T11:15:16","slug":"classla-web-bigger-and-better-web-corpora-for-croatian-serbian-and-slovenian-on-clarin-si-concordancers","status":"publish","type":"page","link":"https:\/\/www.clarin.si\/info\/k-centre\/classla-web-bigger-and-better-web-corpora-for-croatian-serbian-and-slovenian-on-clarin-si-concordancers\/","title":{"rendered":"CLASSLA-web: Bigger and Better Web Corpora for Croatian, Serbian and Slovenian on CLARIN.SI Concordancers"},"content":{"rendered":"

A tutorial on how to use a CLARIN.SI tool for easy querying and statistical analysis of text collections, especially appropriate for linguists, language teachers, digital humanists, corpus linguists and others. We show you how you can query new massive web text collections for Croatian, Slovenian and Serbian and find collocations, word statistics, context of non-standard words and more.<\/strong><\/p>\n

Taja Kuzman and Nikola Ljube\u0161i\u0107 \u00b7 June 22, 2023 \u00b7 10-minutes read<\/pre>\n
 <\/p>\n
CLASSLA-web are new massive web corpora for Slovenian (<\/span>CLASSLA-web.sl<\/span><\/a>), Croatian (<\/span>CLASSLA-web.hr<\/span><\/a>) and Serbian (<\/span>CLASSLA-web.sr<\/span><\/a>). These are pilot corpora for the web corpora of all South Slavic languages that will be published later this year by the <\/span>CLASSLA Knowledge Centre for South Slavic languages<\/span><\/a>. In this blog post, we will show you how you can query them on the <\/span>CLARIN.SI concordancers<\/span><\/a> to explore word usages, collocations, good dictionary examples, differences in usage in different genres and more.<\/span><\/p>\n
Concordancers are computer programs that enable effortless searching and statistical treatment of data in big text collections (= corpora) also for those that are less tech-savvy. The Slovenian research infrastructure<\/span> CLARIN.SI<\/span><\/a> provides three open concordancers, and you can access the CLASSLA web corpora on all three of them (read more <\/span>here<\/span><\/a> on how they differ). The examples in this post are from the<\/span> CLARIN.SI noSketchEngine<\/span><\/a> concordancer, an open-source version of the commercial <\/span>Sketch Engine<\/span><\/a>, which was developed by<\/span> Lexical Computing<\/span><\/a>. The corpora are available for querying here: <\/span>CLASSLA-web.sl<\/span><\/a>,<\/span> CLASSLA-web.hr<\/span><\/a> and<\/span> CLASSLA-web.sr<\/span><\/a>. Feel free to open them and try to reproduce the steps in the blog post, while you are reading it, and then experiment some more on your own. We are very interested in your experience with the corpora. Let us know if you noticed anything that you liked or disliked via <\/span>helpdesk.classla@clarin.si<\/span><\/a>.<\/span><\/p>\n
Introducing the CLASSLA web corpora<\/b><\/h3>\n
As part of the\u00a0<\/span>MaCoCu<\/span><\/a> project, we collected monolingual and parallel web corpora for more than 10 European under-resourced languages and made them freely available (you can download them at <\/span>https:\/\/macocu.eu\/<\/span><\/a>). They can be used for training machine translation systems, language models and other language technologies. To make the datasets more usable for linguists and corpus linguists, the<\/span> CLASSLA<\/span><\/a> team linguistically annotated the monolingual datasets for Slovenian, Serbian and Croatian with the <\/span>state-of-the-art pipeline for linguistic annotation CLASSLA Stanza<\/span><\/a>. Additionally, we enriched them with metadata on genre with the <\/span>multilingual genre classifier X-GENRE<\/span><\/a>. Finally, we made them freely available at the<\/span> CLARIN.SI concordancers<\/span><\/a>. In this blog post, we will show you what kind of insights can linguists and digital humanists obtain from the CLASSLA web corpora with just a few clicks.<\/span><\/p>\n
But first, some information on the corpora<\/b><\/h4>\n
The CLASSLA web corpora come from the<\/span> MaCoCu-sl<\/span><\/a>, <\/span>MaCoCu-hr<\/span><\/a> and<\/span> MaCoCu-sr<\/span><\/a> monolingual corpora. They were collected by crawling primarily the national top-level internet domains, that is, \u201c.si\u201d for Slovenian, \u201c.hr\u201d for Croatian and \u201c.rs\u201d and \u201c.\u0441\u0440\u0431\u201d for Serbian, but also crawling generic (\u201c.com\u201d, \u201c.net\u201d etc.) domains well connected to websites on respective top-level domains. You can find more information on the corpora collection and curation at<\/span> https:\/\/macocu.eu\/<\/span><\/a> and at the dataset entries on the CLARIN.SI repository (<\/span>MaCoCu-sl<\/span><\/a>,<\/span> MaCoCu-hr<\/span><\/a> and <\/span>MaCoCu-sr<\/span><\/a>).<\/span><\/p>\n
Each CLASSLA web corpus comprises around 2 billion words and 6 to 8 million texts. The Slovenian and Croatian CLASSLA web corpora are roughly two times bigger than the previous web corpora for these two languages, the <\/span>slWaC<\/span><\/a> and the<\/span> hrWaC<\/span><\/a>, while the Serbian corpus is even 5 times bigger than the previous web corpus, the<\/span>\u00a0srWaC<\/span><\/a>!<\/span><\/p>\n
In addition to being much bigger, the CLASSLA web corpora are much more recent, which means that they include terms that have been only recently introduced, such as the \u201cCOVID-19\u201d, \u201cself-isolation\u201d and so on. They include texts published up to 2021 (CLASSLA-web.sl) or even 2022 (CLASSLA-web.hr and CLASSLA-web.sr). They also provide rich metadata, including information on:<\/span><\/p>\n