Design of Experiments Guide • Prospective Sample Size and Power • One-Sample and Two-Sample Proportions
Previous
 • 
Next
One-Sample and Two-Sample Proportions
The Sample Size windows and computations to test sample sizes and power for proportions are similar to those for testing means. You enter a true Proportion and choose an Alpha level. Then, for the one-sample proportion case, enter the Sample Size and Null Proportion to obtain the Power. Or, enter the Power and Null Proportion to obtain the Sample Size. Similarly, to obtain a value for Null Proportion, enter values for Sample Size and Power. For the two-sample proportion case, either the two sample sizes or the desired Power must be entered. (See Power and Sample Window for One-Sample Proportions and Difference Between Two Proportions for a Two-Sided Test.)
The computations to determine sample size and null proportions use exact methods based on the binomial distribution. Such methods are more reliable than using the normal approximation to the binomial, which can provide erroneous results for small samples or proportions close to 0 or 1.
For a single proportion, results are based on exact power calculations used in conjunction with one of the following:
•
The “Add two successes and two failures” adjusted Wald test statistic described in Agresti and Coull (1998). See also Section 3.3 in the white paper by Barker on the JMP website: https://www.jmp.com/blind/whitepapers/wp_jmp_powersample_104887.pdf.
•
The Clopper-Pearson exact confidence interval method (Clopper and Pearson, 1934; Agresti and Coull, 1998, Section 1)
For two proportions, results are based on exact power calculations for the adjusted Wald statistic proposed by Agresti and Caffo (2000).
JMP Learning Library – Basic Inference - Proportions and Means
Watch a brief video on how to calculate sample size and power for tests involving one or two sample proportions.
Actual Test Size
The results also show the Actual Test Size. This is the actual Type I error rate for a given situation. Since the binomial distribution is discrete, the actual test size can differ significantly from the stated Alpha level for small samples or proportions near 0 or 1.
One Sample Proportion
Clicking the One Sample Proportion option on the Sample Size and Power window yields a One Proportion window. In this window, you can specify the alpha level and the true proportion. The sample size, power, or the hypothesized proportion is calculated. If you supply two of these quantities, the third is computed, or if you enter any one of the quantities, you see a plot of the other two.
For example, if you have a hypothesized proportion of defects, you can use the One Sample Proportion window to estimate a large enough sample size to guarantee that the risk of accepting a false hypothesis (β) is small. That is, you want to detect, with reasonable certainty, a difference in the proportion of defects.
For the one sample proportion, the hypothesis supported is
and the two-sided alternative is
where p is the population proportion and p0 is the null proportion to test against. Note that if you are interested in testing whether the population proportion is greater than or less than the null proportion, you use a one-sided test. The one-sided alternative is either
or
One-Sample Proportion Window Specifications
In the top portion of the Sample Size window, you can specify or enter values for:
Alpha
is the significance level of your test. The default value is 0.05.
Proportion
is the true proportion, which could be known or hypothesized. The default value is 0.1.
Method
is the method to determine the exact confidence interval. Choices are Exact Agresti-Coull or Exact Clopper-Pearson. For more details, see One-Sample and Two-Sample Proportions.
One-Sided or Two-Sided
Specify either a one-sided or a two-sided test. The default setting is the two-sided test.
In the bottom portion of the window, enter two of the following quantities to see the third, or a single quantity to see a plot of the other two.
Null Proportion
is the proportion to test against (p0) or is left blank for computation. The default value is 0.2.
Sample Size
is the sample size, or is left blank for computation. If Sample Size is left blank, then values for Proportion and Null Proportion must be different.
Power
is the desired power, or is left blank for computation.
One-Sample Proportion Example
As an example, suppose that an assembly line has a historical proportion of defects equal to 0.1, and you want to know the power to detect that the proportion is different from 0.2, given an alpha level of 0.05 and a sample size of 100.
1.
Select DOE > Sample Size and Power.
2.
Click One Sample Proportion.
3.
Leave Alpha as 0.05.
4.
Leave 0.1 as the value for Proportion.
5.
Leave the Method as Exact Agresti-Coull.
6.
Accept the default option of Two-Sided. (A one-sided test is selected if you are interested in testing if the proportion is either greater than or less than the Null Proportion.)
7.
Leave 0.2 as the value for Null Proportion.
8.
Enter 100 as the Sample Size.
9.
Click Continue.
The Power is calculated and is shown as approximately 0.7 (see Power and Sample Window for One-Sample Proportions). Note the Actual Test Size is 0.0467, which is slightly less than the desired 0.05.
Power and Sample Window for One-Sample Proportions
Two Sample Proportions
The Two Sample Proportions option computes the power or sample sizes needed to detect the difference between two proportions, p1 and p2.
For the two sample proportion, the hypothesis supported is
and the two-sided alternative is
where p1 and p2 are the population proportions from two populations, and D0 is the hypothesized difference in proportions.
The one-sided alternative is either
or
Two Sample Proportion Window Specifications
Specifications for the Two Sample Proportions window include:
Alpha
is the significance level of your test. The default value is 0.05.
Proportion 1
is the proportion for population 1, which could be known or hypothesized. The default value is 0.5.
Proportion 2
is the proportion for population 2, which could be known or hypothesized. The default value is 0.1.
One-Sided or Two-Sided
Specify either a one-sided or a two-sided test. The default setting is the two-sided test.
Null Difference in Proportion
is the proportion difference (D0) to test against, or is left blank for computation. The default value is 0.2.
Sample Size 1
is the sample size for population 1, or is left blank for computation.
Sample Size 2
is the sample size for population 2, or is left blank for computation.
Power
is the desired power, or is left blank for computation.
If you enter any two of the following three quantities, the third quantity is computed:
•
Null Difference in Proportion
•
Sample Size 1 and Sample Size 2
•
Power
Example of Determining Sample Sizes with a Two-Sided Test
As an example, suppose you are responsible for two silicon wafer assembly lines. Based on the knowledge from many runs, one of the assembly lines has a defect rate of 8%; the other line has a defect rate of 6%. You want to know the sample size necessary to have 80% power to reject the null hypothesis of equal proportions of defects for each line.
To estimate the necessary sample sizes for this example:
1.
Select DOE > Sample Size and Power.
2.
Click Two Sample Proportions.
3.
Accept the default value of Alpha as 0.05.
4.
Enter 0.08 for Proportion 1.
5.
Enter 0.06 for Proportion 2.
6.
Accept the default option of Two-Sided.
7.
Enter 0.0 for Null Difference in Proportion.
8.
Enter 0.8 for Power.
9.
Leave Sample Size 1 and Sample Size 2 blank.
10.
Click Continue.
The Sample Size window shows sample sizes of 2554. (see Difference Between Two Proportions for a Two-Sided Test.) Testing for a one-sided test is conducted similarly. Simply select the One-Sided option.
Difference Between Two Proportions for a Two-Sided Test
Example of Determining Power with Two Sample Proportions Using a One-Sided Test
Suppose you want to compare the effectiveness of a two chemical additives. The standard additive is known to be 50% effective in preventing cracking in the final product. The new additive is assumed to be 60% effective. You plan on conducting a study, randomly assigning parts to the two groups. You have 800 parts available to participate in the study (400 parts for each additive). Your objective is to determine the power of your test, given a null difference in proportions of 0.01 and an alpha level of 0.05. Because you are interested in testing that the difference in proportions is greater than 0.01, you use a one-sided test.
1.
Select DOE > Sample Size and Power.
2.
Click Two Sample Proportions.
3.
Accept the default value of Alpha as 0.05.
4.
Enter 0.6 for Proportion 1.
5.
Enter 0.5 for Proportion 2.
6.
Select One-Sided.
7.
Enter 0.01 as the Null Difference in Proportion.
8.
Enter 400 for Sample Size 1.
9.
Enter 400 for Sample Size 2.
10.
Leave Power blank.
11.
Click Continue.
Difference Between Two Proportions for a One-Sided Test shows the Two Proportions windows with the estimated Power calculation of 0.82.
Difference Between Two Proportions for a One-Sided Test
You conclude that there is about an 82% chance of rejecting the null hypothesis at the 0.05 level of significance, given that the sample sizes for the two groups are each 400. Note the Actual Test Size is 0.0513, which is slightly larger than the stated 0.05.