For the latest version of JMP Help, visit JMP.com/help.


Predictive and Specialized Modeling > Model Comparison > Model Comparison Platform Options
Publication date: 04/21/2023

Image shown hereModel Comparison Platform Options

The options in the Model Comparison red triangle menu depend on whether the response is continuous or categorical.

Continuous Responses

Categorical Responses

Image shown hereContinuous Responses

When the response is continuous, the Model Comparison red triangle menu contains the following options:

Model Averaging

Creates a new column of the arithmetic mean of the predicted values across models.

Plot Actual by Predicted

Shows or hides a scatterplot of the actual versus the predicted values. The plots for the different models are overlaid.

Plot Residual by Row

Shows or hides a plot of the residuals by row number. The plots for the different models are overlaid.

Profiler

Shows or hides a profiler for each response based on prediction formula columns in your data. The profilers have a row for each model being compared.

Image shown hereCategorical Responses

When the response is categorical, the Model Comparison red triangle menu contains the following options:

Model Averaging

Creates a new column of the arithmetic mean of the predicted probabilities across models.

ROC Curve

Shows or hides ROC curves for each level of the response variable. The curves for the different models are overlaid.

AUC Comparison

Shows or hides a comparison of the area under the ROC curve (AUC) from each model. The area under the curve is the indicator of the goodness of fit, where 1 is a perfect fit.

The report includes the following information:

standard errors and confidence intervals for each AUC

standard errors, confidence intervals, and hypothesis tests for the difference between each pair of AUCs

an overall hypothesis test for testing whether all AUCs are equal

Lift Curve

Shows or hides lift curves for each level of the response variable. The curves for the different models are overlaid.

Cum Gains Curve

Shows or hides cumulative gains curves for each level of the response variable. A cumulative gains curve is a plot of the proportion of a response level that is identified by the model against the proportion of all responses. A cumulative gains curve for a perfect model would reach 1.0 at the overall proportion of the response level. The curves for the different models are overlaid.

Confusion Matrix

Shows or hides confusion matrices for each model. A confusion matrix is a two-way classification of actual and predicted responses. Count and rate confusion matrices are shown. Separate confusion matrices are produced for each level of the Group variable.

If the response has a Profit Matrix column property, then Actual by Decision Count and Actual by Decision Rate matrices are shown to the right of the confusion matrices. For more information about these matrices, see Additional Examples of the Partition Platform.

Profiler

Shows or hides a profiler for each response based on prediction formula columns in your data. The profilers have a row for each model being compared.

Decision Threshold

(Available only for binary categorical responses.) Shows or hides Decision Thresholds reports for the training, validation, and test sets, if specified. Each report contains a graph of the distribution of fitted probabilities for each model, confusion matrices for each model, classification graphs to compare the model fits, and a table of classification accuracy metrics. See Decision Thresholds Report.

Related Information

ROC Curve

Lift Curve

Want more information? Have questions? Get answers in the JMP User Community (community.jmp.com).