Example of a Covering Array with No Factor Level Restrictions

The data in this example pertain to interoperability in the area of software testing. There are four factors of interest:

•	Web Browser (Safari, IE, Firefox, Chrome, Other)

•	Operating System (Windows or Macintosh)

•	RAM (4, 8, or 16 MB)

•	Connection Speed at three settings (0-1, 1-5, or greater than 5 Mbps)

You are interested in finding out which combinations of these factors are likely to cause failures.

The response is whether the system functions properly for each combination of factor settings.

Testing each combination of settings would require 90 (5x2x3x3) trials. To keep the run size manageable, you decide to require Strength 3 coverage, indicating that all combinations of any three factors are tested.

Create the Design

Create the Strength 3 covering array by following these steps.

1.	Select Help > Sample Data Library and open Design Experiment/Software Factors.jmp.

The Software Factors.jmp data table contains the factors and their settings.

2.	Select DOE > Covering Array.

3.	From the menu next to Strength: t = , select 3.

4.	From the Covering Array red triangle menu, select Load Factors and click Continue.

The Factors outline is populated with the four factors and their levels.

Factors Outline for Software Factors

5.	Click Continue.

The Restrict Factor Level Combinations outline opens, where you can enter restrictions on the design settings. Because there are no restrictions for this design, do not change the default selection of None.

6.	Click Make Design.

The Design outline opens to show a 45-run design.

Design and Metrics Outlines for Software Factors

In the Metrics outline, consider the row that corresponds to t = 3. The Coverage is 100%, indicating that the design covers 100% of the three-factor interactions. This is what you want, because you requested a Strength 3 design. For t = 3, the Diversity column indicates that 68.33% of the three-factor interactions that appear are distinct. There is some minor repetition of three-factor combinations.

For t = 4, the Coverage is 50%, indicating that the design covers half of the four-factor interactions. There are 90 possible distinct combinations of the four factor settings. The 45 runs in the design comprise one-half of these distinct combinations. The Diversity value of 100% reinforces the fact that none of the four-way interactions are repeated.

7.	Click Make Table.

Design Table for Software Factors

The design is presented in a data table. Notice the following in the Table panel at the top left:

•	The Design note indicates that this is a strength 3 covering array.

•	The DOE Dialog script reproduces the Covering Array window settings.

•	The Analysis script analyzes the experimental data.

Analyze the Experimental Data

Now that you have your design table, you can conduct your experiment and record your data in the Response column of the design table (Design Table for Software Factors). Your experimental results are in the Software Data.jmp sample data table.

1.	Select Help > Sample Data Library and open Design Experiment/Software Data.jmp.

2.	In the Table panel, click the red triangle next to Analysis and select Run Script.

Analysis of Software Experimental Data

The Summary outline indicates that three tests failed and four tests did not result in a pass or fail outcome.

The Failure Analysis Details outline gives a breakdown of failures in terms of the associated three-way interactions. The outline lists only combinations of factor levels where all tests resulted in failure. If any test that involves a given three-way combination of settings results in success, then that three-way combination of settings cannot be responsible for system failure.

Two failures were associated with Web Browser set to Firefox, Operating System set to Macintosh, and RAM set to 8 MB. Notice that this combination led to failure regardless of the setting of Connection Speed.

3.	Select the first line in the 3 Factor Interactions report.

Selection of an Interaction in the Analysis Report

This action selects the corresponding rows and columns in the data table.

Corresponding Selection of Rows and Columns in the Data Table

Three failures were associated with combinations of Web Browser, RAM, and Connection Speed. Note that two of these failures, Firefox, 8 MB, 1-5 Mbps and Firefox, 8 MB, >5 Mbps, are among the two failures for the Web Browser, Operating System, and RAM interaction. Selecting any of these rows in the report selects the corresponding rows and columns in the data table.