In JMP, you can perform cross-validation by selecting the K-Fold Crossvalidation option from the Stepwise Fit red triangle menu.
Note: Max Validation RSquare considers only the models defined by p-value entry (Forward direction) or removal (Backward direction). It does not consider all possible models.
To use the Backward direction, you must first click Enter All to enter all terms. The Backward direction behaves in a similar fashion to the Forward direction. If you click Go rather than Step, the process of entering terms proceeds automatically. The model designated as Best is the one with the largest RSquare Validation that can be followed by as many as ten models with lower RSquare Validation values.
An Entropy RSquare measure (also known as McFadden’s R2) for the validation set computed as follows:
K-fold cross validation randomly divides the data into k subsets. In turn, each of the k sets is used as a validation set while the remaining data are used as a training set to fit the model. In total, k models are fit and k validation statistics are obtained. The model giving the best validation statistic is chosen as the final model. This method is useful for small data sets, because it makes efficient use of limited amounts of data.
In JMP, select K-Fold Crossvalidation from the red triangle options for Stepwise Fit.
In JMP Pro, you can access k-fold cross validation in two ways:
When you use k-fold cross validation, the Stopping Rule defaults to Max K-Fold RSquare. This rule attempts to maximize the RSquare K-Fold statistic.
Note: Max K-Fold RSquare considers only the models defined by p-value entry (Forward direction) or removal (Backward direction). It does not consider all possible models.