For the latest version of JMP Help, visit JMP.com/help.


Publication date: 04/21/2023

Power for Two Independent Sample Means

Use the Power for Two Independent Sample Means Explorer to determine a sample size for a hypothesis test about the means from two groups. Select DOE > Sample Size Explorers > Power > Power for Two Independent Sample Means. Explore the trade offs between sample size, power, significance, and the hypothesized difference to detect. Sample size and power are associated with the following hypothesis test:

Equation shown here

versus the two-sided alternative:

Equation shown here

or versus a one-sided alternative:

Equation shown here or Equation shown here

where μ1 and μ2 are the true means of the two populations. It is assumed that the populations of interest are normally distributed and that you want to detect a difference of δ between the means.

Power Explorer for Two Independent Sample Means Settings

Set study assumptions and explore sample sizes using the radio buttons, text boxes, and menus. The profiler updates as you make changes to the settings. Alternatively, change settings by dragging the cross hairs on the profiler curves.

Test Type

Specifies a one or two-sided hypothesis test.

Preliminary Information

Alpha

The probability of a type I error, which is the probability of rejecting the null hypothesis when it is true. It is commonly referred to as the significance level of the test. The default alpha level is 0.05.

Group Population Standard Deviations

Specifies the distribution for calculations.

Yes

Specifies known group standard deviations, calculations use the z distribution.

No

Specifies unknown group standard deviations, calculations use the t distribution.

Power Explorer for Two Independent Sample Means Profiler

The profiler enables you to visualize the impact of sample size assumptions on the power calculations.

Total Sample Size

Specifies the total number of observations (runs, experimental units, or samples) needed for your experiment. Select Lock to lock the total sample size

Solve for

Enables you to solve for a sample size, difference to detect, or an assumed group standard deviation.

Power

Specifies the probability of rejecting the null hypothesis when it is false. With all other parameters fixed, power increases as sample size increases.

Group 1 Sample Size

Specifies the number of observations (runs, experimental units, or samples) needed for Group 1 in your experiment.

Group 2 Sample Size

Specifies the number of observations (runs, experimental units, or samples) needed for Group 2 in your experiment.

Difference to Detect

Specifies the smallest difference between the group means that you want to be able to declare statistically significant.

Group 1 StdDev (σ1)

Specifies the assumed standard deviation for one of your groups, Group 1. An estimate of the error standard deviation could be the root mean square error (RMSE) from a previous model fit.

Group 2 StdDev (σ2)

Specifies the assumed standard deviation for the second group, Group 2. An estimate of the error standard deviation could be the root mean square error (RMSE) from a previous model fit.

Note: Adjusting the sample size for one group adjusts the total sample size unless the total sample size is locked. In that case, adjusting the sample size for one group adjust the sample size for the second group. Use the text boxes to specify group sample sizes.

Power Explorer for Two Independent Sample Means Options

The Explorer red triangle menu and report buttons provide additional options:

Simulate Data

Opens a data table of simulated data based on the explorer settings. View the simulated response column formula for the settings used.

Make Data Collection Table

Creates a new data table that you can use for data collection. The table includes scripts to facilitate data analysis.

Save Settings

Saves the current settings to the Saved Settings table. This enables you to save a set of alternative study plans. See Saved Settings in the Sample Size Explorers.

Help

Opens JMP help.

Statistical Details for the Power Explorer for Two Independent Sample Means

The power calculations for testing the difference in means of two sample groups are based on the traditional t test, or if σ1 and σ2 are known, the z test.

Variances Unknown

For the case when the group variances are unknown, it is assumed that σ1=σ2=σ and the power is calculated based on the form of the alternative hypothesis. For a one-sided, higher alternative (μ1 > μ2):

Equation shown here

For a one-sided, lower alternative (μ1 < μ2):

Equation shown here

For a two-sided alternative (μ1μ2):

Equation shown here

Equation shown here

where:

α is the significance level

n1 and n2 are the group sample sizes

σ is the pooled standard deviation

δ is the difference to detect

t1-α,ν is the (1 - α)th quantile of the central t-distribution with ν degrees of freedom

T(t; ν, λ) is the cumulative distribution function of the non-central t distribution with ν degrees of freedom and non-centrality parameter λ.

The degrees of freedom are approximate degrees of freedom, also known as Satterthwaite degrees of freedom:

Equation shown here

Variances Known

When σ1 and σ2 are known the z distribution is used for the power calculation. The power is calculated based on the form of the alternative hypothesis. For a one-sided, higher alternative (μ1 > μ2):

Equation shown here

For a one-sided, lower alternative (μ1 < μ2):

Equation shown here

For a two-sided alternative (μ1μ2):

Equation shown here

where:

α is the significance level

n1 and n2 are the group sample sizes

σ1 and σ2 are known group standard deviations

δ is the difference to detect

z1-α is the (1 - α)th quantile of the z-distribution

Φ(x) is the cumulative distribution function of the normal distribution.

Want more information? Have questions? Get answers in the JMP User Community (community.jmp.com).