In JMP, you can perform cross-validation by selecting the K-Fold Crossvalidation option from the Stepwise Fit red triangle menu.
If you specify a Validation column with two or three values, Stepwise fits models based on the training set. Model fit statistics are reported for the validation and test sets. SeeValidation and Test Set Statistic Definitions for details on how these statistics are defined.
Note: Max Validation RSquare considers only the models defined by p-value entry (Forward direction) or removal (Backward direction). It does not consider all possible models.
To use the Backward direction, you must first click Enter All to enter all terms. The Backward direction behaves in a similar fashion to the Forward direction. If you click Go rather than Step, the process of entering terms proceeds automatically. The model designated as Best is the one with the largest RSquare Validation that can be followed by as many as ten models with lower RSquare Validation values.
‒
|
‒
|
‒
|
‒
|
An Entropy RSquare measure (also known as McFadden’s R2) for the validation set computed as follows:
‒
|
‒
|
K-fold cross validation randomly divides the data into k subsets. In turn, each of the k sets is used as a validation set while the remaining data are used as a training set to fit the model. In total, k models are fit and k validation statistics are obtained. The model giving the best validation statistic is chosen as the final model. This method is useful for small data sets, because it makes efficient use of limited amounts of data.
In JMP, select K-Fold Crossvalidation from the red triangle options for Stepwise Fit.
In JMP Pro, you can access k-fold cross validation in two ways:
•
|
From the red triangle options for Stepwise Fit, select K-Fold Crossvalidation.
|
When you use k-fold cross validation, the Stopping Rule defaults to Max K-Fold RSquare. This rule attempts to maximize the RSquare K-Fold statistic.
Note: Max K-Fold RSquare considers only the models defined by p-value entry (Forward direction) or removal (Backward direction). It does not consider all possible models.
The Max K-Fold RSquare stopping rule behaves in a fashion similar to the Max Validation RSquare stopping rule. See Max Validation RSquare. Replace references to RSquare Validation with RSquare K-Fold.