ELMo embeddings models for seven languages

  Avtorji

Ulčar, Matej

  Identifikator vnosa

http://hdl.handle.net/11356/1277

 URL projekta

http://embeddia.eu

 Dokumentirano v

https://arxiv.org/abs/1911.10049

 Datum objave

2019-11-25

 Vrsta

text, toolService

 Velikost

7 files, 1.4 gb

 Jezik(i)

Croatian , Estonian , Finnish , Latvian , Lithuanian , Slovenian , Swedish

 Opis

ELMo language model (https://github.com/allenai/bilm-tf) used to produce contextual word embeddings, trained on large monolingual corpora for 7 languages: Slovenian, Croatian, Finnish, Estonian, Latvian, Lithuanian and Swedish. Each language's model was trained for approximately 10 epochs. Corpora sizes used in training range from over 270 M tokens in Latvian to almost 2 B tokens in Croatian. About 1 million most common tokens were provided as vocabulary during the training for each language model. The model can also infer OOV words, since the neural network input is on the character level. Each model is in its own .tar.gz archive, consisting of two files: pytorch weights (.hdf5) and options (.json). Both are needed for model inference, using allennlp (https://github.com/allenai/allennlp/blob/master/tutorials/how_to/elmo.md) python library.

 Izdajatelj

Faculty of Computer and Information Science, University of Ljubljana

 Zahvala

European Union EC/H2020/825153 "EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media"

 Ključne besede

ELMo contextual embeddings word embeddings

 Zbirke

CLARIN.SI data & tools

 Druge različice

Seznam vseh različic  

Prikaži polni zapis vnosa