ELMo embeddings models for seven languages

  Authors

Ulčar, Matej

  Item identifier

http://hdl.handle.net/11356/1277

 Project URL

http://embeddia.eu

 Referenced by

https://arxiv.org/abs/1911.10049

 Date issued

2019-11-25

 Type

text, toolService

 Size

7 files, 1.4 gb

 Language(s)

Croatian , Estonian , Finnish , Latvian , Lithuanian , Slovenian , Swedish

 Description

ELMo language model (https://github.com/allenai/bilm-tf) used to produce contextual word embeddings, trained on large monolingual corpora for 7 languages: Slovenian, Croatian, Finnish, Estonian, Latvian, Lithuanian and Swedish. Each language's model was trained for approximately 10 epochs. Corpora sizes used in training range from over 270 M tokens in Latvian to almost 2 B tokens in Croatian. About 1 million most common tokens were provided as vocabulary during the training for each language model. The model can also infer OOV words, since the neural network input is on the character level. Each model is in its own .tar.gz archive, consisting of two files: pytorch weights (.hdf5) and options (.json). Both are needed for model inference, using allennlp (https://github.com/allenai/allennlp/blob/master/tutorials/how_to/elmo.md) python library.

 Publisher

Faculty of Computer and Information Science, University of Ljubljana

 Acknowledgement

European Union EC/H2020/825153 "EMBEDDIA - Cross-Lingual Embeddings for Less-Represented Languages in European News Media"

 Subject(s)

ELMo contextual embeddings word embeddings

 Collection(s)

CLARIN.SI data & tools

 Other versions

List all versions  

Show full item record