Specialized Models • Model Comparison • Example of Model Comparison
Previous
 • 
Next
Example of Model Comparison
This section provides an example of using the Model Comparison platform. The example uses demographic data to build a model for median home price. A regression model and a bootstrap forest model are compared.
Begin by selecting Help > Sample Data Library and opening Boston Housing.jmp.
Create a Validation Column
1.
Create a column called validation.
2.
On the Column Info window, select Random from the Initialize Data list.
3.
Select the Random Indicator radio button.
4.
Click OK.
The rows assigned a 0 are the training set. The rows assigned a 1 are the validation set.
Create the Regression Model and Save the Prediction Formula to a Column
1.
Select Analyze > Fit Model.
2.
Select mvalue and click Y.
3.
Select the other columns (except validation) and click Add.
4.
Select Stepwise in the Personality list.
5.
Select validation and click Validation.
6.
Click the Run button.
7.
Select P-value Threshold from the Stopping Rule list.
8.
Click the Go button.
9.
Click the Run Model button.
The Fit Group report appears, a portion of which is shown in Fit Model Report.
10.
Save the prediction formula to a column by selecting Save Columns > Prediction Formula on the Response red triangle menu.
Fit Model Report
Create the Bootstrap Forest Model and Save the Prediction Formula to a Column
1.
Select Analyze > Modeling > Partition.
2.
Select mvalue and click Y, Response.
3.
Select the other columns (except validation) and click X, Factor.
4.
Select validation and click Validation.
5.
Select Bootstrap Forest in the Method list.
6.
Click OK.
7.
Select the Early Stopping check box.
8.
Select the Multiple Fits over number of terms check box.
9.
Click OK.
The Bootstrap Forest report appears, a portion of which is shown in Bootstrap Forest Model.
10.
Save the prediction formula to a column by selecting Save Columns > Save Prediction Formula on the Bootstrap Forest red triangle menu.
Bootstrap Forest Model
Compare the Models
1.
Select Analyze > Modeling > Model Comparison.
2.
Select the two prediction formula columns and click Y, Predictors.
3.
Select validation and click Group.
4.
Click OK.
The Model Comparison report appears (Model Comparison Report).
Note: Your results differ due to the random assignment of training and validation rows.
Model Comparison Report
The rows in the training set were used to build the models, so the RSquare statistics for Validation=0 might be artificially inflated. In this case, the statistics are not representative of the models’ future predictive ability. This is especially true for the bootstrap forest model.
Compare the models using the statistics for Validation=1. In this case, the bootstrap forest model predicts better than the regression model.
Related Information 
•
Introduction to Fitting Models in the Fitting Linear Models book
•
Partition Models section
Launch the Model Comparison Platform
The Model Comparison Platform is available in JMP Pro only.
To launch the Model Comparison platform, select Analyze > Modeling > Model Comparison.
The Model Comparison Launch Window
Y, Predictors
The columns that contain the predictions for the models that you want to compare. They can be either formula columns or just data columns. Prediction formula columns created by JMP platforms have either the Predicting or Response Probability column property. If you specify a column that does not contain one of these properties, the platform prompts you to specify which column is being predicted by the specified Y column.
For a categorical response with k levels, most model fitting platforms save k columns to the data table, each predicting the probability for a level. All k columns need to be specified as Y, Predictors. For platforms that do not save k columns of probabilities, the column containing the predicted response level can be specified as a Y, Predictors column.
If you do not specify any Y, Predictors columns, JMP uses the prediction formula columns in the data table that have either the Predicting or Response Probability column property.
Group
The column that separates the data into groups, which are fit separately.
The other role buttons are common among JMP platforms. See the Using JMP book for details.