| dc.contributor.author | Žitnik, Slavko |
| dc.contributor.author | Dragar, Frenk |
| dc.date.accessioned | 2021-12-09T17:44:53Z |
| dc.date.available | 2021-12-09T17:44:53Z |
| dc.date.issued | 2021-11-30 |
| dc.identifier.uri | http://hdl.handle.net/11356/1469 |
| dc.description | The evaluation framework contains public evaluation scripts. All the scripts contain additional Dockerfiles that allow for platform-independent evaluation and exact comparison of results. Pre-built Docker images are available in slobench/eval DockerHub repository. The evaluation framework is used and maintained by the SloBENCH leaderboard Web site team. SloBENCH submitters are able to check their compliance of submissions and evaluate theri model on training/validation data prior to submission. The initial version of SloBENCH contains evaluation scripts with examples of training and testing datasets for nine different tasks: named entity recognition, part-of-speech tagging, lemmatization, dependency parsing, semantic role labeling, translation (ENG-SLO, SLO-ENG), summarization and question answering. |
| dc.publisher | CLARIN.SI |
| dc.rights | Apache License 2.0 |
| dc.rights.uri | https://opensource.org/licenses/Apache-2.0 |
| dc.rights.label | PUB |
| dc.source.uri | https://github.com/clarinsi/slobench-eval-docker |
| dc.subject | evaluation |
| dc.subject | SloBENCH |
| dc.subject | Docker |
| dc.title | SloBENCH evaluation framework |
| dc.type | toolService |
| metashare.ResourceInfo#ContentInfo.detailedType | tool |
| metashare.ResourceInfo#ResourceComponentType#ToolServiceInfo.languageDependent | false |
| has.files | yes |
| branding | CLARIN.SI data & tools |
| demo.uri | https://slobench.cjvt.si |
| contact.person | Slavko Žitnik slavko.zitnik@fri.uni-lj.si Faculty of Computer and Information Science, University of Ljubljana |
| sponsor | Jožef Stefan Institute CLARIN SloBENCH: Design and implementation of natural language processing evaluation framework nationalFunds |
| files.count | 1 |
| files.size | 3877091 |
Files in this item
- Name
- slobench-eval-docker-main-2021-12-04.zip
- Size
- 3.7 MB
- Format
- application/zip
- Description
- Commit ID: 4e88c0cc2111dc89602
- MD5
- d3d8f1c98e980aebfc1bed323dbfad62
- slobench-eval-docker-main
- .gitignore12 B
- run.py2 kB
- cjvt-logo.png22 kB
- README.md3 kB
- evaluation_scripts
- eval_prototype
- submission.zip274 B
- README.md658 B
- evaluate.py1 kB
- ground_truth.zip273 B
- Dockerfile147 B
- eval_translation
- sample_submission.zip283 B
- .gitignore38 B
- translation_eval.py2 kB
- sample_reference.zip241 B
- README.md1 kB
- evaluate.py422 B
- requirements.txt28 B
- Dockerfile429 B
- eval_summarization
- README.md1 kB
- .gitignore38 B
- sample_reference.zip1 MB
- evaluate.py740 B
- sample_submission.zip1 MB
- requirements.txt14 B
- Dockerfile734 B
- dependencies
- ROUGE-1.5.5
- XML
- RegExp.pm5 kB
- Handler
- BuildDOM.pm7 kB
- DOM
- Attr.pod2 kB
- Parser.pod2 kB
- Notation.pod874 B
- Comment.pod438 B
- NodeList.pm695 B
- Entity.pod1 kB
- AttDef.pod662 B
- Text.pod1 kB
- ProcessingInstruction.pod802 B
- DOMImplementation.pod665 B
- DOMException.pm1 kB
- ElementDecl.pod529 B
- Node.pod12 kB
- AttlistDecl.pod1 kB
- Element.pod4 kB
- PerlSAX.pm940 B
- Document.pod5 kB
- CDATASection.pod1 kB
- DocumentFragment.pod2 kB
- CharacterData.pod2 kB
- EntityReference.pod1 kB
- NamedNodeMap.pm5 kB
- NodeList.pod1 kB
- DocumentType.pod4 kB
- XMLDecl.pod686 B
- NamedNodeMap.pod3 kB
- DOM.pm108 kB
- ROUGE-1.5.5.pl101 kB
- runROUGE-test.pl2 kB
- RELEASE-NOTE.txt14 kB
- README.txt15 kB
- data
- smart_common_words.txt3 kB
- WordNet-2.0-Exceptions
- buildExeptionDB.pl679 B
- adj.exc22 kB
- verb.exc37 kB
- noun.exc37 kB
- adv.exc85 B
- WordNet-1.6-Exceptions
- buildExeptionDB.pl679 B
- adj.exc19 kB
- verb.exc81 kB
- noun.exc108 kB
- adv.exc85 B
- XML
- ROUGE-1.5.5
- summarization_eval.py2 kB
- eval_question_answering
- qa_validation.zip35 kB
- README.md2 kB
- .gitignore38 B
- sample_reference.zip1 kB
- sample_submission.zip1 kB
- evaluate.py3 kB
- qa_eval.py7 kB
- requirements.txt49 B
- Dockerfile348 B
- qa_test.zip123 kB
- qa_train.zip160 kB
- eval_sequence_tagging_conllu
- sample_submission.zip384 kB
- sample_reference.zip509 kB
- README.md1 kB
- evaluate.py1 kB
- conll18_ud_eval.py27 kB
- Dockerfile258 B
- eval_sequence_tagging_tab
- sample_submission.zip50 kB
- .gitignore38 B
- sample_reference.zip50 kB
- README.md1 kB
- evaluate.py422 B
- tagging_tab_eval.py1 kB
- requirements.txt19 B
- Dockerfile377 B
- eval_prototype
- clarin-logo.png28 kB
- cjvt-header.png15 kB