Specialized Models • Nonlinear Regression with Custom Models • Additional Examples
Previous
 • 
Next
Additional Examples
This section provides several examples of the broad usefulness of the Nonlinear platform.
Example of Maximum Likelihood: Logistic Regression
In this example, we show several variations of minimizing a loss function. The loss function is the negative of a log-likelihood function, thus producing maximum likelihood estimates.
The Logistic w Loss.jmp data table in the Nonlinear Examples sample data folder has an example for fitting a logistic regression using a loss function. The Y column contains ones for events and zeros for non-events. The Model Y column has the linear model, and the Loss column has the loss function. In this example, the loss function is the negative log-likelihood for each observation, or the negative log of the probability of getting the observed response.
Run the model by following the steps below:
1.
Select Help > Sample Data Library and open Nonlinear Examples/Logistic w Loss.jmp.
2.
Select Analyze > Modeling > Nonlinear.
3.
Assign Model Y to the X, Predictor Formula role.
4.
Assign Loss to the Loss role.
Nonlinear Launch Window
5.
Click OK.
The Nonlinear Fit Control Panel appears.
Nonlinear Fit Control Panel
6.
Click Go.
The parameter estimates are shown in the Solution report. See Solution Report.
Solution Report
The Loss value in the Solution report is the negative log-likelihood evaluated at the parameter estimates.
The same problem can be handled differently by defining a model column formula that absorbs the logistic function. Also, define a loss function that uses the model to form the probability for a categorical response level. Model2 Y holds the model, and the loss function is Loss2.
The loss function is used in this example, so the second derivative as well as the first one is calculated for the optimization. With least squares, the second derivatives are multiplied by residuals, which are usually near zero. For custom loss functions, second derivatives can play a stronger role. Follow these steps to fit the model:
1.
Display the Logistic w Loss.jmp sample data table again.
2.
Select Analyze >Modeling > Nonlinear.
3.
Assign Model2 Y to the X, Predictor Formula role.
4.
Assign Loss2 to the Loss role.
5.
Select the Second Derivatives option.
Nonlinear Launch Window for Second Derivatives
6.
Click OK.
The Nonlinear Fit Control Panel appears.
7.
Type 1000 for the Iteration Stop Limit.
Specify the Stop Limit
8.
Click Go.
The Solution report shows the same Loss and parameter estimates as before.
The Standard Error is different
Example of a Probit Model with Binomial Errors: Numerical Derivatives
The Ingots2.jmp sample data table includes the numbers of ingots tested for readiness after different treatments of heating and soaking times. The response variable, NReady, is binomial, depending on the number of ingots tested (Ntotal) and the heating and soaking times. Maximum likelihood estimates for parameters from a probit model with binomial errors are obtained using:
•
numerical derivatives
•
the negative log-likelihood as a loss function
•
the Newton-Raphson method.
The average number of ingots ready is the product of the number tested and the probability that an ingot is ready for use given the amount of time it was heated and soaked. Using a probit model, the P column contains the model formula:
Normal Distribution(b0+b1*Heat+b2*Soak)
The argument to the Normal Distribution function is a linear model of the treatments.
To specify binomial errors, the loss function, Loss, has the formula
-(Nready*Log(p) + (Ntotal - Nready)*Log(1 - p))
Follow these steps to fit the model:
1.
Select Analyze > Modeling > Nonlinear.
2.
Assign P to the X, Predictor Formula role,
3.
Assign Loss to the Loss role.
4.
Select the Numeric Derivatives Only option.
5.
Click OK.
6.
Click Go.
The platform used the Numerical SR1 method to obtain the parameter estimates shown in Solution for the Ingots2 Data.
Solution for the Ingots2 Data
Example of a Poisson Loss Function
A Poisson distribution is often used to model count data.
, n = 0, 1, 2, …
where μ can be a single parameter, or a linear model with many parameters. Many texts and papers show how the model can be transformed and fit with iteratively reweighted least squares (Nelder and Wedderburn 1972). However, in JMP it is more straightforward to fit the model directly. For example, McCullagh and Nelder (1989) show how to analyze the number of reported damage incidents caused by waves to cargo-carrying vessels.
The data are in the Ship Damage.jmp sample data table. The model formula is in the model column, and the loss function (or –log-likelihood) is in the Poisson column. To fit the model, follow the steps below:
1.
Select Analyze > Modeling > Nonlinear.
2.
Assign model to the X, Predictor Formula role.
3.
Assign Poisson to the Loss role.
4.
Click OK.
5.
Set the Current Value (initial value) for b0 to 1, and the other parameters to 0 (Enter New Parameters).
Enter New Parameters
6.
Click Go.
7.
Click the Confidence Limits button.
The Solution report is shown in Solution Table for the Poisson Loss Example. The results include the parameter estimates and confidence intervals, and other summary statistics.
Solution Table for the Poisson Loss Example
Note: The standard errors, confidence intervals, and hypothesis tests are correct only if least squares estimation is done, or if maximum likelihood estimation is used with a proper negative log likelihood.
Example of Setting Parameter Limits
The Fit Curve personality enables you to fit a model and then use the prediction equation in the full personality of the Nonlinear platform. This method requires more steps and user input but allows any nonlinear model to be fit.
Complete Example Using the Fit Curve Personality in Nonlinear Regression with Built-In Models to fit the model. This example shows how to save the prediction formula from Fit Curve and then set parameter limits in Nonlinear.
1.
From the Logistic 4P red triangle menu, select Save Formulas > Save Parametric Prediction Formula.
A new column named Toxicity Predictor appears in the data table.
2.
Select Analyze > Modeling > Nonlinear.
3.
Assign Toxicity to the Y, Response role.
4.
Assign Toxicity Predictor to the X, Predictor Formula role.
5.
Assign Formulation to the Group role.
6.
Click OK.
The Nonlinear Fit window appears (Nonlinear Fit Control Panel). In the Control Panel, parameter values and locking options are shown. The letters listed before each parameter correspond to variables from the Prediction Model in the Fit Curve function.
Nonlinear Fit Control Panel
Tip: You can lock parameters if you know the values from prior information.
7.
Select the red triangle menu next to Nonlinear Fit and then select Parameter Bounds.
Options for setting the lower and upper parameters appear next to the parameters.
8.
Set the lower bounds for the parameters as shown in Setting Parameter Bounds. You know from prior experience that the maximum toxicity of the drug is at least 1.1.
Setting Parameter Bounds
9.
Click Go.
The final parameter estimates are shown in the Solution report, along with other fit statistics (Nonlinear Fit Plot and Parameter Estimates). The fitted model is shown on the plot.
Nonlinear Fit Plot and Parameter Estimates
Options below the plot allow for adjusting parameter limits and estimates (Nonlinear Fit Plot and Parameter Estimates).