Analysis Script
Use the Analysis Script in a data table generated by the Covering Array platform to identify suspicious factors or combination of factors contributing to failures. Use a deterministic ranking or a probabilistic ranking.
The Analysis script assumes the following about the Response column:
• The responses are recorded as 0 for failure and 1 for success.
• At least one failure is needed in order to run the analysis.
Missing values are permitted. The Response column can be renamed.
The Analysis Script produces a report that contains two sections.
Summary
The Summary section of the Covering Array Analysis report includes the number of runs resulting in Success and Failures, and the number of runs for which the response is Missing.
Failure Analysis Details
The Failure Analysis Details section of the Covering Array Analysis report contains a summary of failures by factors to help identify the cause of failures. Use the default rankings by count or select Rank by Probability for Bayesian Fault Localization (BayesFLo) analysis. See Fault Localization.
Factor Interactions
The Factor Interactions report is labeled by the value of k that is the smallest number of interactions that detect a failure. (For a definition of detect, see Covering Arrays and Strength.) The table contains the following three columns:
Factors
All k-factor combinations that detect failures.
Failure Levels
The values of the k factors in the Factors column that detect failures.
Failure Count
The number of failures corresponding to the k-factor combination of Failure Levels.
Note: A failure observation can appear in more than one of the k-factor combinations listed in the Failure Levels column.
The rows in the <k> Factor Interactions report are dynamically linked to the data table. If you select one or more rows in the report, the corresponding rows are selected in the data table.
Fault Localization
Fault localization is a method available when you select Rank by Probability in the Failure Analysis Details section. This method uses a Bayesian procedure to compute probabilities that a factor combinations is the root cause of failures based on prior information.
Initialize Priors
Specify how to initialize the priors.
Unweighted priors (uneditable)
Applies equal initial probabilities to all factors after you click Continue.
Unweighted priors (editable)
Opens the Prior Table after you click Continue. You can assign relative importance to each factor level.
Weighted priors (editable)
Opens the Ranked Factors / Levels table. Define high, medium, and low rankings. Assign the high, medium, or low ranking to each factor. Click Continue to apply settings to initialize the priors and open the Prior Table.
Tip: Select one or more rows in the Prior Table or the Ranked Factors / Levels Table, right-click, and select Quick Entry to open a window for setting table values.
Maximum Combination Strength
Define the maximum strength, that is the maximum level of interactions considered.
Back
Return to the initialize priors step.
Update Ranking
Updates rankings to reflect changes made to the priors.
After updating the rankings, the Factor Interactions report is replaced with the Fault Localization (Rank by Probability) and includes a column of posterior probabilities.
The following options are available only if you select Yes to enable defining priors for combinations:
Multipliers for Selected Combinations
Enables the adjustment of priors to account for suspected combinations of factors.
All combinations of selected factors (degree k)
Select factor names from the Names column, click Go. A table of factor combinations of degree k opens, adjust the Multiplier to increase or decrease the impact of each combination.
All combinations of selected factors (degree 2 to k)
Select factor names from the Names column, click Go. A table of factor combinations of degree 2 to k opens, adjust the Multiplier to increase or decrease the impact of each combination.
Flexible combinations of selected factors and/or levels
Select factor names or factor levels from the Names or Values columns, click Go. A table of combinations opens, adjust the Multiplier to increase or decrease the impact of each combination.
Remove Combinations
Enables you to removed selected combinations from the combination table.
After updating the rankings, the <k> Factor Interactions report is replaced with the Fault Localization (Rank by Probability) and includes a column of posterior probabilities.