For the latest version of JMP Help, visit JMP.com/help.


Predictive and Specialized Modeling > Neural Networks > Model Reports > Training and Validation Measures of Fit
Publication date: 06/21/2023

Training and Validation Measures of Fit

The Neural model report contains tables of measures of fit for the training and validation sets (Figure 3.8). The following measures of fit are included in each table:

Generalized RSquare

A measure that can be applied to general regression models. It is based on the likelihood function L and is scaled to have a maximum value of 1. The value is 1 for a perfect model, and 0 for a model no better than a constant model. The Generalized RSquare measure simplifies to the traditional RSquare for continuous normal responses in the standard least squares setting. Generalized RSquare is also known as the Nagelkerke or Craig and Uhler R2, which is a normalized version of Cox and Snell’s pseudo R2. See Nagelkerke (1991).

Entropy RSquare

(Appears only when the response is nominal or ordinal.) A measure of fit that compares the log-likelihoods from the fitted model and the constant probability model. Entropy RSquare ranges from 0 to 1, where values closer to 1 indicate a better fit. See Entropy RSquare.

RSquare

Gives the RSquare for the model.

RASE

Gives the root average squared error. When the response is nominal or ordinal, the differences are between 1 and p (the fitted probability for the response level that actually occurred).

Mean Abs Dev

The average of the absolute values of the differences between the response and the predicted response. When the response is nominal or ordinal, the differences are between 1 and p (the fitted probability for the response level that actually occurred).

Misclassification Rate

The rate for which the response category with the highest fitted probability is not the observed category. Appears only when the response is nominal or ordinal.

-LogLikelihood

Gives the negative of the log-likelihood. See Likelihood, AICc, and BIC in Fitting Linear Models.

SSE

Gives the error sums of squares. Available only when the response is continuous.

Sum Freq

Gives the number of observations that are used. If you specified a Freq variable in the Neural launch window, Sum Freq gives the sum of the frequency column.

If there are multiple responses, fit statistics are given for each response, and an overall Generalized RSquare and negative Log-Likelihood is given.

Confusion Statistics

When nominal or ordinal responses are fit in the Neural platform, a Confusion Matrix report and Confusion Rates report are given (Figure 3.8). The Confusion Matrix report shows a two-way classification of the actual response levels and the predicted response levels. For a categorical response, the predicted level is the one with the highest predicted probability. The Confusion Rates report is equal to the Confusion Matrix report, with the numbers divided by the row totals.

Want more information? Have questions? Get answers in the JMP User Community (community.jmp.com).