PlumX Metrics
Embed PlumX Metrics

Agreement models for multiraters

Turkish Journal of Medical Sciences, ISSN: 1300-0144, Vol: 41, Issue: 5, Page: 939-944
2011
  • 2
    Citations
  • 49
    Usage
  • 0
    Captures
  • 0
    Mentions
  • 0
    Social Media
Metric Options:   Counts1 Year3 Year

Metrics Details

Article Description

Aim: Agreement between 2 or more independent raters evaluating the same items and same scale can be measured by kappa coefficient. In recent years, modeling agreement among raters rather than summarizing indices has been preferred. In this study, the disadvantages of kappa are reviewed. Agreement models are introduced and these models are applied to a real data set. Materials and methods: Three pathologists classified each of 118 slides in terms of carcinoma in situ of the uterine cervix, based on the most involved lesions. Using log-linear agreement models, agreement between 3 pathologists according to their evaluations was investigated. Results: Coefficient of kappa was found to be 0.48 among the 3 pathologists, which indicates a moderate agreement. Models were applied to the data. The agreement parameter was estimated for the best model among models. The probability of giving the same decision by the 3 pathologists was 2.5 times higher than that of giving a different decision. Conclusion: Log-linear models can be used to measure the agreement among more than 2 raters. Modeling agreement can provide more information than kappa. © TÜBİTAK.

Bibliographic Details

Provide Feedback

Have ideas for a new metric? Would you like to see something else here?Let us know