For the latest version of JMP Help, visit JMP.com/help.


Publication date: 04/21/2023

Example of Rater Agreement

Use the Categorical platform to summarize data from multiple raters. In the data, each column contains ratings for a single rater.

1. Select Help > Sample Data Folder and open Attribute Gauge.jmp.

2. Select Analyze > Consumer Research > Categorical.

3. Select the Related tab.

4. Select A, B, and C, and click Rater Agreement on the Related tab.

5. Click OK.

Figure 3.33 Rater Agreement ReportĀ 

Rater Agreement Report

The rater agreement is strong as shown by the Kappa statistics. The Kappa statistic can take on a value between 0 (no agreement) to 1.0 (perfect agreement). The details section provides 2x2 tables for each pair of raters. The Bowker test of symmetry tests the null hypothesis that cell proportions are symmetric for all pairs of cells (pij = pji for all i, j). Here, p-values for the Bowker test are all greater than 0.05, indicating no strong evidence of asymmetry between raters.

Want more information? Have questions? Get answers in the JMP User Community (community.jmp.com).