CroSloEngual BERT 1.1

  Authors

Ulčar, Matej and Robnik-Šikonja, Marko

  Item identifier

http://hdl.handle.net/11356/1330

 Project URL

http://embeddia.eu

 Referenced by

https://arxiv.org/abs/2006.07890

 Date issued

2020-07-09

 Type

toolService

 Language(s)

Croatian , English , Slovenian

 Description

Trilingual BERT (Bidirectional Encoder Representations from Transformers) model, trained on Croatian, Slovenian, and English data. State of the art tool representing words/tokens as contextually dependent word embeddings, used for various NLP classification tasks by finetuning the model end-to-end. CroSloEngual BERT are neural network weights and configuration files in pytorch format (i.e. to be used with pytorch library). Changes in version 1.1: fixed vocab.txt file, as previous verson had an error causing very bad results during fine-tuning and/or evaluation.

 Publisher

Faculty of Computer and Information Science, University of Ljubljana

 Acknowledgement

European Union EC/H2020/825153 "EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media"

 Subject(s)

word embeddings multilingual contextual embeddings BERT language model

 Collection(s)

CLARIN.SI data & tools

 Other versions

List all versions  

Show full item record