Collective Human Opinions in Semantic Textual Similarity
Transactions of the Association for Computational Linguistics, ISSN: 2307-387X, Vol: 11, Page: 997-1013
2023
- 5Citations
- 205Usage
- 8Captures
- 2Mentions
Metric Options: Counts1 Year3 YearSelecting the 1-year or 3-year option will change the metrics count to percentiles, illustrating how an article or review compares to other articles or reviews within the selected time period in the same journal. Selecting the 1-year option compares the metrics against other articles/reviews that were also published in the same calendar year. Selecting the 3-year option compares the metrics against other articles/reviews that were also published in the same calendar year plus the two years prior.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Example: if you select the 1-year option for an article published in 2019 and a metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019. If you select the 3-year option for the same article published in 2019 and the metric category shows 90%, that means that the article or review is performing better than 90% of the other articles/reviews published in that journal in 2019, 2018 and 2017.
Citation Benchmarking is provided by Scopus and SciVal and is different from the metrics context provided by PlumX Metrics.
Metrics Details
- Citations5
- Citation Indexes5
- Usage205
- Downloads198
- Abstract Views7
- Captures8
- Readers8
- Mentions2
- News Mentions2
- News2
Most Recent News
PRC-UAE Collaboration and US Technology Transfer Concerns in Abu Dhabi
Executive Summary: The Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) poses a high risk of technology transfer due to its deep connections to the
Article Description
Despite the subjective nature of semantic textual similarity (STS) and pervasive disagreements in STS annotation, existing benchmarks have used averaged human ratings as gold standard. Averaging masks the true distribution of human opinions on examples of low agreement, and prevents models from capturing the semantic vagueness that the individual ratings represent. In this work, we introduce USTS, the first Uncertainty-aware STS dataset with ∼15,000 Chinese sentence pairs and 150,000 labels, to study collective human opinions in STS. Analysis reveals that neither a scalar nor a single Gaussian fits a set of observed judgments adequately. We further show that current STS models cannot capture the variance caused by human disagreement on individual instances, but rather reflect the predictive confidence over the aggregate dataset.
Bibliographic Details
http://www.scopus.com/inward/record.url?partnerID=HzOxMe3b&scp=85173061422&origin=inward; http://dx.doi.org/10.1162/tacl_a_00584; https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00584/117217/Collective-Human-Opinions-in-Semantic-Textual; https://dclibrary.mbzuai.ac.ae/nlpfp/85; https://dclibrary.mbzuai.ac.ae/cgi/viewcontent.cgi?article=1084&context=nlpfp; https://dx.doi.org/10.1162/tacl_a_00584
MIT Press
Provide Feedback
Have ideas for a new metric? Would you like to see something else here?Let us know