Decision Matrix Report

Note: This report is available only if the response has a Profit Matrix column property or if you specify costs using the Specify Profit Matrix option.

When a profit matrix is defined, the partition algorithm uses the values in the matrix to calculate the profit for each decision. The level of the response with the largest profitability is considered the final decision. When you select Save Columns > Save Prediction Formula, this decision is given in the column Most Profitable Prediction for <column name>. See Specify Profit Matrix.

In the Decision Matrix report, the decision counts reflect the most profitable prediction decisions based on the weighting in the profit matrix. The report gives Decision Count and Decision Rate matrices for the training set and for validation and test sets (if defined). For reference the profit matrix is also shown.

Note: If you change the weights in your Profit Matrix using the Specify Profit Matrix option, the Decision Matrix report automatically updates to reflect your changes.

Decision Count Matrix

Shows a two-way classification with actual responses in rows and classification counts in columns.

Specified Profit Matrix

Gives the weights that define the Profit Matrix.

Decision Rate Matrix

Shows rate values corresponding to the proportion of a given row’s observations that are classified into each category. If all observations are correctly classified, the rates on the diagonal are all equal to one.

Tip: To obtain decision rate matrices for a response using the default profit matrix with costs of 1 and -1, select Specify Profit Matrix from the red triangle menu, make no changes to the default values, and click OK.

The matrices are arranged in two rows:

•	The Decision Count matrices are in the first row.

•	The Specified Profit Matrix is to the right in the first row.

•	The Decision Rate matrices are in the second row.

Example of Profit Matrix and Decision Matrix Report

Consider a study of liver cancer patients. Based on various measurements and markers, you want to classify patients according to their disease severity (high or low). However, classifying a subject who has high severity into the low group is a costly error, as that patient may not receive the necessary more aggressive treatment. Classifying a patient with low severity into the high severity group is a less costly error. That patient will receive the more aggressive treatment, but its side effects are an unnecessary consequence.

In the following example, you define a profit matrix in the context of a liver cancer study and obtain a Decision Matrix report. The Decision Matrix report helps you assess your classification rates relative to the costs in your profit matrix.

1.	Select Help > Sample Data Library and open Liver Cancer.jmp.

2.	Select Analyze > Modeling > Partition.

3.	Select Severity and click Y, Response.

4.	Select BMI through Jaundice and click X, Factor.

Click OK.

6.	Click Split ten times.

Check that the Number of Splits is 10 in the panel beneath the plot.

Panel Showing Summary of Split

7.	From the red triangle menu next to Partition for Severity, select Specify Profit Matrix.

8.	Change the entries as follows:

Completed Profit Matrix

Tip: You can save this profit matrix as a column property for use in later analyses.

Note the following:

‒	Each value of 1 reflects your profit when you make a correct decision.

‒	The -3 value indicates that if you classify a Low severity patient as High severity, your loss is 3 times as much as the profit of a correct decision.

‒	The -5 value indicates that if you classify a High severity patient as Low severity, your loss is 5 times as much as the profit of a correct decision.

Click OK.

10.	From the red triangle menu next to Partition for Severity, select Show Fit Details.

Confusion Matrix and Decision Matrix Reports

The Confusion Matrix and Decision Matrix reports follow the list of Measures in the Fit Details report. Notice that the Confusion Matrix and the first matrix in the Decision Matrix report show different counts. This is because the weighting in the profit matrix results in different decisions than do the predicted probabilities without weighting.

11.	From the red triangle menu next to Partition for Severity, select Save Columns > Save Prediction Formula.

Eight columns are added to the data table.

Tip: To quickly return to the data table, click the View Associated Data icon

in the bottom-right corner of the report window.

‒

The first three columns involve only the predicted probabilities. The confusion matrix counts are based on the Most Likely Severity column, which classifies a patient into the level with the highest predicted probability. These probabilities are given in the Prob(Severity == High) and Prob(Severity == Low) columns.

‒

The last five columns involve the profit matrix weighting. The column called Most Profitable Prediction for Severity contains the decision based on the profit matrix. The decision for a patient is the level that results in the largest profit. The profits are given in the Profit for High and Profit for Low columns.

The Confusion Matrix in Confusion Matrix and Decision Matrix Reports shows classifications based on predicted probabilities alone. Based on these, 33 High severity patients would be classified as Low severity and 5 Low severity patients would be classified as High severity. The Decision Matrix report incorporates the weights in the profit matrix. Using those weights, only 5 High severity patients are classified as Low severity. However, this comes at the expense of misclassifying 38 Low severity patients into the High severity group.