For the latest version of JMP Help, visit JMP.com/help.

The Student’s t method controls only the error rate for an individual comparison. As such, it is not a true multiple comparison procedure. All other methods provided control the overall error rate for all comparisons of interest. Each of these methods uses a multiple comparison adjustment in calculating p-values and confidence limits.
An example of the control window for the Multiple Comparisons option is shown in Figure 3.28. This example is based on the Big Class.jmp data table, with weight as Y and age, sex, and height as model effects. Two classes of estimates are available for comparisons: Least Squares Means Estimates and User-Defined Estimates.
Figure 3.28 Launch Window for Least Squares Means Estimates
The specification of User-Defined Estimates is illustrated in Figure 3.29. Three levels of age and both levels of sex have been selected. Also, two values of height have been manually entered. The Add Estimates button has been clicked, resulting in the listing of all possible combinations of the specified levels. At this point, you can specify more estimates and click the Estimates button again to add them to the list of Estimates for Comparison.
Figure 3.29 Launch Window for User-Defined Estimates
Note: In this section, we use the term mean to refer to either estimates of least squares means or user-defined estimates.
Select Show Least Squares Means Plot to obtain a least square means plot. If your effect is an interaction term, then you have the option to create an interaction plot. You select the term for the overlay. If you do not select the interaction plot, then the least squares plot will nest the effect terms. See Least Squares Means Plot Options.
The t ratio for the significance test. This column appears only if you right-click in the report and select Columns > t Ratio.
The p-value for the significance test. This column appears only if you right-click in the report and select Columns > Prob>|t|.
Note: You can obtain t ratios and p-values by right-clicking in the table and selecting Columns.
The value of Nelson’s h statistic used in constructing the decision limits.
Specifically, the average least squares mean is a weighted average with weights inversely proportional to the diagonal entries of the matrix . Here L is the matrix of coefficients used to compute the group least squares means. For a technical definition of least squares means and the average least squares mean, see the GLM Procedure chapter in the SAS/STAT 14.3 User’s Guide (SAS Institute Inc. 2017).
For user-defined estimates, the average mean is defined similarly. However, in this case, L is the matrix of coefficients used to define the estimates.
Nelson: Provides exact critical values and p-values. Used whenever possible, in particular, when the estimates are uncorrelated.
Nelson-Hsu: Provides approximate critical values and p-values based on Hsu’s factor analytical approximation is used (Hsu 1992). Used when exact values cannot be obtained.
For technical details, see the GLM Procedure chapter in the SAS/STAT 14.3 User’s Guide (SAS Institute Inc. 2017).
Adds a column that contains p-values (Prob>|t|) to the Comparisons with Overall Average report. Note that computing exact critical values and p-values for unbalanced designs requires complex integration and can be computationally challenging. When calculations for such a quantile fail, the Sidak quantile is computed but p-values are not available.
Consider the Lipid Data.jmp sample data table. You are interested in whether any of the four Smoking History categories are unusual in that their mean Coffee intake (cups/day) differ from the overall average coffee intake while controlling for alcohol use and heart history. You specify a model with Coffee intake (cups/day) as the response and Smoking History, Alcohol Use, and Heart History as model effects.
1.
Select Help > Sample Data Library and open Lipid Data.jmp.
2.
Select Analyze > Fit Model.
3.
Select Coffee intake (cups/day) and click Y.
4.
Select Smoking History, Alcohol Use, and Heart History, and click Add.
5.
Click Run.
8.
In the Choose Initial Comparisons list, select Comparisons with Overall Average - ANOM.
9.
The results shown in Figure 3.30 indicate that the least squares means for non-smokers and cigarette smokers differ significantly from the overall average in terms of coffee intake.
Figure 3.30 Comparisons with Overall Average for Ratings
When exact calculation of p-values and confidence intervals is not possible, Hsu’s factor analytical approximation is used (Hsu 1992). Note that computing exact critical values and p-values for unbalanced designs requires complex integration and can be computationally intensive. When calculations for such a quantile fail, the Sidak quantile is computed.
Dunnett: Provides exact critical values and p-values. Used whenever possible, in particular, when the estimates are uncorrelated.
Dunnett-Hsu: Provides approximate critical values and p-values based on Hsu’s factor analytical approximation (Hsu 1992). Used when exact values cannot be obtained.
For technical details, see the GLM Procedure chapter in the SAS/STAT 14.3 User’s Guide (SAS Institute Inc. 2017).
Adds a column that contains p-values (Prob>|t|) to the Comparisons with Control report. Note that computing exact critical values and p-values for unbalanced designs requires complex integration and can be computationally challenging. When calculations for such a quantile fail, the Sidak quantile is computed but p-values are not available.
The All Pairwise Comparisons option shows either a Tukey HSD All Pairwise Comparisons or Student’s t All Pairwise Comparisons report (Hsu 1996; Westfall et al. 2011). Tukey HSD comparisons are constructed so that the significance level applies jointly to all pairwise comparisons. In contrast, for Student’s t comparisons, the significance level applies to each individual comparison. When making several pairwise comparisons using Student’s t tests, the risk that one of the comparisons incorrectly signals a difference can well exceed the stated significance level.
The critical value for the test. Note that, for Tukey HSD, the quantile is , where q is the appropriate percentage point of the Studentized range statistic.
Tukey: Provides exact critical values and p-values. Used when the means are uncorrelated and have equal variances, or when the design is variance-balanced.
Tukey-Kramer: Provides approximate critical values and p-values. Used when exact values cannot be obtained.
For technical details, see the GLM Procedure chapter in the SAS/STAT 14.3 User’s Guide (SAS Institute Inc. 2017).
Both Tukey HSD and Student’s t compare all pairs of levels. For each pairwise comparison, the All Pairwise Differences report shows:
t Ratio - the t ratio for the test of whether the difference is zero
Prob > |t| - the p-value for the test
This plot, sometimes called a diffogram or a mean-mean scatterplot, displays the confidence intervals for all means pairwise differences. (See Figure 3.32 for an example.) Colors indicate which differences are significant.
Use this option to conduct one or more equivalence tests. Equivalence tests are useful when you want to detect differences that are of practical interest. You are asked to specify a threshold difference for group means for which smaller differences are considered practically equivalent. In other words, if two group means differ by this amount or less, you are willing to consider them equivalent.
The Two One-Sided Tests (TOST) method is used to test for a practical difference between the means (Schuirmann 1987). Two one-sided pooled-variance tests are constructed for the null hypotheses that the true difference exceeds the threshold values. If both tests reject, the difference in the means does not statistically exceed either threshold value. Therefore, the groups are considered practically equivalent. If only one or neither test rejects, then the groups might not be practically equivalent.
Lower Bound t Ratio, Upper Bound t Ratio - the lower and upper bound ratios for the two one-sided pooled-variance significance tests
Lower Bound p-Value, Upper Bound p-value - p-values corresponding to the lower and upper bound t ratios
Consider the Lipid Data.jmp sample data table. You are interested in Cholesterol differences for gender and non-smokers versus former smokers (Smoking History equal to no and quit, respectively) across two ages (25 and 35) and average height.
1.
Select Help > Sample Data Library and open Lipid Data.jmp.
2.
Select Analyze > Fit Model.
3.
Select Cholesterol and click Y.
4.
Select Gender, Age, Height, and Smoking History, and click Add.
5.
Click Run.
7.
8.
From the Choose Gender levels list, select female (it should already be selected by default) and male.
10.
In the Age list, enter the ages 25 and 35 in the first two rows.
Do not enter any values in the list entitled Height. Because no values for Height are specified, the mean value of the Height column is used in the multiple comparisons report.
11.
Click Add Estimates.
12.
In the Choose Initial Comparisons list, select All Pairwise Comparisons - Tukey HSD.
Figure 3.31 Populated Used-Defined Estimates Window
13.
The All Pairwise Differences report indicates that two of the 28 pairwise comparisons are significant. The All Pairwise Comparisons Scatterplot, shown in Figure 3.32, shows the confidence intervals for these comparisons in red. You can place your pointer over any of the points to determine which pairwise comparison the point represents. The tooltips also contain the difference between the two levels in the comparison. The two red points in Figure 3.32 represent the points comparing 35-year-old former smokers to 25-year-old non-smokers, for both females and males.
Figure 3.32 All Pairwise Comparisons Scatterplot for User-Defined Comparisons

Help created on 3/19/2020