Dataset and label checks · Question 19 of 60
Which metrics can be used to measure inter-annotator agreement (IAA)?
1 answer is correct. Easy
Show answer
Answer
- B Cohen's Kappa or simple percentage agreement
Explanation
How much annotators agree (inter-annotator agreement, IAA) is measured, for example, with Cohen's Kappa or simply as the percentage of identical labels. If agreement is low, the guidelines are often unclear, the data ambiguous or the annotation sloppy.