The linear regression model is formulated as . Here X is a matrix whose first column consists of 1s, and whose remaining columns are the values of the non-intercept terms in the model. If the model consists of p terms, including the intercept, then X is an n by p matrix, where n is the number of observations. The parameter estimates, denoted by the vector b, are typically given by the formula:
However, this formula presumes that exists, in other words, that the p x p matrix is invertible, or equivalently, of full rank. Situations often arise when is not invertible because there are linear dependencies among the columns of X.
Singularity and Parameter Estimates Report for Model with Linear Dependencies shows reports for the Reactor 8 Runs.jmp sample data table. To obtain these reports, fit a model with Percent Reacted as Y. Enter Feed Rate, Catalyst, Stir Rate, Temperature, Concentration, Catalyst*Stir Rate, Catalyst*Concentration, and Feed Rate*Catalyst as model effects.
Singularity and Parameter Estimates Report for Model with Linear Dependencies
If there are degrees of freedom available for an estimate of error, t tests for parameters estimated using biased estimates are conducted. These tests should be interpreted with caution, though, given that the estimates are not unique.
An F test is conducted if the degrees of freedom for an effect are nonzero, assuming that there are degrees of freedom for error. Whenever DF is less than Nparm, the description LostDFs is displayed to the far right in the row corresponding to the effect (Singularity and Parameter Estimates Report for Model with Linear Dependencies). These effects have the opportunity to explain only model sums of squares that have not been attributed to the aliased effects that have absorbed their lost degrees of freedom. It follows that the sum of squares given in the Effect Tests report most likely underrepresents the “true” sum of squares associated with the effect. If the test is significant, its significance is meaningful. But lack of significance should be interpreted with caution.
Open the Singularity.jmp sample data table. There is a response Y, four predictors X1, X2, X3, and A, and five observations. The predictors are continuous except for A, which is nominal with four levels. Also note that there is a linear dependency among the continuous effects, namely, X3 X1 X2.
1.
Select Help > Sample Data Library and open Singularity.jmp.
2.
Run the script Model 1. The script opens a Fit Model launch window where the effects are entered in the order X1, X2, X3.
3.
Click Run and leave the report window open.
4.
Run the script Model 2. The script opens a Fit Model launch window where the effects are entered in the order X1, X3, X2.
5.
Click Run and leave the report window open.
Compare the two reports (Fit Least Squares Reports for Model 1 (on left) and Model 2 (on right)). The Singularity Details report at the top of both reports displays the linear dependency, indicating that X1 X3 - X2.
Now compare the Parameter Estimates reports for both models. Note, for example, that the estimate for X1 for Model 1 is –1.25 while for Model 2 it is 2.75. In both models, only two of the terms associated with effects are estimated, because there are only two model degrees of freedom. See the Analysis of Variance report. The estimates of the two terms that are estimated are labeled Biased while the remaining estimate is set to 0 and labeled Zeroed.
Fit Least Squares Reports for Model 1 (on left) and Model 2 (on right)
1.
Select Help > Sample Data Library and open Singularity.jmp.
2.
Click Analyze > Fit Model.
3.
Select Y and click Y.
4.
Select X1 and A and click Add.
5.
Set the Emphasis to Minimal.
6.
Click Run.
Portions of the report are shown in (Fit Least Squares Report for Model with X1 and A). The Singularity Details report shows that there is a linear dependency involving X1 and the three terms associated with the effect A. (For details about how a nominal effect is coded, see Details of Custom Test Example). The Analysis of Variance report shows that there are three model degrees of freedom. The Parameter Estimates report shows Biased estimates for the three terms X1, A[a], and A[b] and a Zeroed estimate for the fourth, A[c].
The Effect Tests report shows that X1 cannot be tested, because A must be entered first and A accounts for the three model degrees of freedom. However, A can be tested, but with only two degrees of freedom. (X1 must be entered first and it accounts for one of the model degrees of freedom.) The test for A is partial, so it must be interpreted with care.
Fit Least Squares Report for Model with X1 and A