Stepwise Regression Control Panel
P-value Threshold uses p-values (significance levels) to enter and remove effects from the model. Two other options appear when P-value Threshold is chosen: Prob to Enter is the maximum p-value that an effect must have to be entered into the model during a forward step. Prob to Leave is the minimum p-value that an effect must have to be removed from the model during a backward step.
Minimum AICc uses the minimum corrected Akaike Information Criterion to choose the best model.
Minimum BIC uses the minimum Bayesian Information Criterion to choose the best model.
Max Validation RSquare uses the maximum R-square from the validation set to choose the best model. This is available only when a validation column is used, and the validation column has two or three distinct values. For more information about validation, see Using Validation.
Max K-Fold RSquare uses the maximum R-square from K-fold cross validation to choose the best model. This is available only when K-Fold cross validation is used. For more information about validation, see Using Validation.
Forward brings in the regressor that most improves the fit, given that term is significant at the level specified by Prob to Enter. See Forward Selection Example.
Backward removes the regressor that affects the fit the least, given that term is not significant at the level specified in Prob to Leave. See Backward Selection Example.
Mixed alternates the forward and backward steps. It includes the most significant term that satisfies Prob to Enter and removes the least significant term satisfying Prob to Leave. It continues removing terms until the remaining terms are significant and then it changes to the forward direction.
Description of Current Model Statistics describes the statistics for the current model, which appear below the Stepwise Regression Control panel.
Adjusts R2 to make it more comparable over models with different numbers of parameters by using the degrees of freedom in its computation. The adjusted R2 is useful in stepwise procedure because you are looking at many different models and want to adjust for the number of terms in the model.
Mallow’s Cp criterion for selecting a model. It is an alternative measure of total squared error and can be defined as follows:
where s2 is the MSE for the full model and SSEp is the sum-of-squares error for a model with p variables, including the intercept. Note that p is the number of x-variables+1. If Cp is graphed with p, Mallows (1973) recommends choosing the model where Cp first approaches p.
k is the number of estimated parameters, including intercept and error terms in the model
n is the number of observations in the data set.
Burnham and Anderson (2004) discuss using AICc for model selection. The best model has the smallest value, as discussed in Akaike (1974).
k is the number of parameters
n is the sample size.
2.
Click Step.
From the top figure in Current Estimates Table for Forward Selection, you can see that after one step, the most significant term, Runtime, is entered into the model.
3.
The bottom figure in Current Estimates Table for Forward Selection shows that all of the terms have been added, except RstPulse and Weight.
Current Estimates Table for Forward Selection
2.
Click Enter All.
All Effects Entered Into the Model
3.
For Direction, select Backward.
4.
Click Step two times.
The first backward step removes RstPulse and the second backward step removes Weight.
Current Estimates with Terms Removed and Step History Table
Current Estimates Table
Traditional test statistic to test that the term effect is zero. It is the square of a t-ratio. It is in quotation marks because it does not have an F-distribution for testing the term because the model was selected as it was fit.
Significance level associated with the F statistic. Like the “F Ratio,” it is in quotation marks because it is not to be trusted as a real significance probability.
As each step is taken, the Step History report records the effect of adding a term to the model. For example, the Step History report for the Fitness.jmp example shows the order in which the terms entered the model and shows the statistics for each model. See Example Using Stepwise Regression.
Step History Report