Document-level aspect-based sentiment analysis dataset AspectBench (HBS Subset) 1.0

  Authors

Chatterjee, Nishan ; Koloski, Boshko ; Doucet, Antoine ; Pollak, Senja and Purver, Matthew

  Item identifier

http://hdl.handle.net/11356/2356

 Project URL

https://github.com/nishan-chatterjee/aspect-based-sentiment-analysis

 Referenced by

https://hal.science/hal-05583597v2/document
https://www.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2026.1844418

 Date issued

2026-08-04

 Type

corpus, text

 Size

67547 articles, 20461137 tokens

 Language(s)

Bosnian , Croatian , Serbian , Serbo-Croatian

 Description

AspectBench 1.0 (HBS Subset) is an HBS language dataset (hbs, ISO 639-3) containing online news articles, mainly in Serbian and Croatian, for studying sentiment towards target entities like companies and brand names. It is a document-level aspect-based sentiment analysis dataset containing long articles and unseen or new target entities that are not encountered during training. The article text has been filtered to remove URLs. The dataset contains 82,714 distinct records covering 65 targets. The articles were published between 31 December 2012 and 30 December 2023. The held-out test set contains 12,407 records. The other 70,307 records form one training and validation pool. Three random splits of this pool are provided. Each split contains 57,901 training records and 12,406 validation records. Only the source-system internal identifier was removed. The article text, sentiment labels, aspect names, and aspect tags are unchanged. Each record is one document-aspect pair. It contains the full article headline and body, one target company or brand, the target's tracked keyword patterns, the number of matched mentions, publication metadata, and one sentiment label: negative (-1), neutral (0), or positive (1). A single article can occur in more than one record when it has more than one target. The literal <aspect>...</aspect> tags in the article mark the mentions that refer to that record's target. These tags are part of the annotation. Examples: { "uuid": "11855", "article": "bankama u srbiji bolje nego ikada ... neke banke poput sosijete zeneral banke i <aspect>erste banke</aspect> pohvalile su se da su postigle najbolji rezultat otkako posluju u srbiji. ...", "translation": "Banks in Serbia are doing better than ever ... Some banks, such as Societe Generale Bank and Erste Bank, highlighted that they achieved their best result since they began operating in Serbia. ...", "aspect": "Erste banka", "keywords": ["erst* bank*", "ерст* банк*"], "sentiment": 1, "language": "Serbo-Croatian", "published": "2017-05-02T22:00:00.000Z", "country": "RS" } { "uuid": "11854", "article": "bankama u srbiji bolje nego ikada ... <aspect>telenor banka</aspect> ponovila je los rezultat iz prve godine poslovanja, a strucnjaci kazu da je to ocekivano zbog ogromnih ulaganja i da bi odrzivost ovog modela poslovanja mogla da se oceni posle cetiri, pet godina. ...", "translation": "Banks in Serbia are doing better than ever ... Telenor Bank repeated the weak result from its first year of operation, while experts said this was expected because of substantial investments and that the sustainability of this business model could be assessed after four or five years. ...", "aspect": "Telenor banka", "keywords": ["telenor bank*", "теленор банк*"], "sentiment": -1, "language": "Serbo-Croatian", "published": "2017-05-02T22:00:00.000Z", "country": "RS" } Although both records originate from the same article text, their target aspects and corresponding sentiment labels differ. The first record is positive because the article reports that Erste banka achieved its best result since entering the Serbian market. The second record is negative because Telenor banka is discussed as repeating a weak business result. This illustrates why the task is formulated as document-level entity-targeted sentiment analysis: sentiment is predicted with respect to a specific target entity rather than assigned broadly to the document as a whole. The same training and validation pool is supplied under three splits. Therefore, the four partition files contain 223,328 record entries in total. This is not the number of distinct records. The target catalogue contains 65 targets and no dataset records. Files in the submitted archive: - hbs_train_val_0.json, hbs_train_val_1.json, and hbs_train_val_2.json: the three random train and validation splits of the same 70,307-record pool. - hbs_test.json: the held-out test set with 12,407 records. - hbs_aspects.json: the 65 targets, their forms, and keyword patterns. Use this file to identify seen and unseen targets when calculating Macro-F1, quadratic weighted kappa, or other scores. - statistics.json: exact counts for records, articles, labels, languages, characters, and tokens. - hbs-dataloader.ipynb: examples for loading and inspecting the data, displaying sentiment-label tables by target, and tallying seen, unseen, and train/validation-only targets. - README.md: the dataset description, file guide, record-field guide, and usage examples. - CITATION.cff: structured citation information, including the Frontiers article, GitHub repository, and Hugging Face model collection. - LICENSE-and-access.md: a plain-language summary of the access conditions. - release-validation.json: the machine-readable report of the schema, split, target-catalogue, and tag checks. - MANIFEST.sha256: SHA-256 checksums for all files in the archive.

 Publisher

Jožef Stefan Institute

 Acknowledgement

Slovenian Research and Innovation Agency (ARIS) P2-0103 "Core research program Knowledge Technologies"

ARRS (Slovenian Research Agency) L2-50070 "Embeddings-based techniques for Media Monitoring Applications"

ARIS (Slovenian Research and Innovation Agency) GC-0002 "LLM4DH: Large Language Models for Digital Humanities"

Young Researcher Grant PR-12394 "ARIS"

Nouvelle-Aquitaine Region, France 2022-2021-17014610 "ACTUADA"

European Union HORIZON-WIDERA-2023-TALENTS-01-01 101186647 "EU Era Chair (AI4DH)"

 Subject(s)

aspect-based sentiment analysis targeted sentiment analysis entity-level sentiment analysis document-level sentiment analysis news corpus media monitoring

 Collection(s)

CLARIN.SI data & tools

Show full item record