Borehole Model: A Sphere-Packing Example

Worley (1987) presented a model of the flow of water through a borehole that is drilled from the ground surface through two aquifers. The response variable y is the flow rate through the borehole in m3/year and is determined by the following equation:

There are eight inputs to this model:

rw = radius of borehole, 0.05 to 0.15 m

r = radius of influence, 100 to 50,000 m

Tu = transmissivity of upper aquifer, 63,070 to 115,600 m2/year

Hu = potentiometric head of upper aquifer, 990 to 1100 m

Tl = transmissivity of lower aquifer, 63.1 to 116 m2/year

Hl = potentiometric head of lower aquifer, 700 to 820 m

L = length of borehole, 1120 to 1680 m

Kw = hydraulic conductivity of borehole, 9855 to 12,045 m/year

This example is atypical of most computer experiments because the response can be expressed as a simple, explicit function of the input variables. However, this simplicity is useful for explaining the design methods.

Create the Sphere-Packing Design for the Borehole Data

To create a Sphere-Packing design for the borehole problem:

1.	Select DOE > Space Filling Design.

2.	Click the red triangle icon on the Space Filling Design title bar and select Load Factors.

3.	Select Help > Sample Data Library and open Design Experiment/Borehole Factors.jmp (Factors Panel with Factor Values Loaded for Borehole Example).

Factors Panel with Factor Values Loaded for Borehole Example

Note: The logarithm of r and rw are used in the following discussion.

4.	Click Continue.

5.	Specify a sample size (Number of Runs) of 32 as shown in Space-Filling Design Method Panel Showing 32 Runs.

Space-Filling Design Method Panel Showing 32 Runs

6.	Click the Sphere Packing button to produce the design.

7.	Click Make Table to make a table showing the design settings for the experiment.

To see a completed data table for this example, select Help > Sample Data Library and open Design Experiment/ Borehole Sphere Packing.jmp. Because the designs are generated from a random seed, the settings that you obtain will differ from those shown in the completed table.

The Borehole Sphere Packing.jmp data table contains a Fit Model script that you can use to analyze the data. Columns containing the true model, the prediction formula, and the prediction bias are included in the data table.

Guidelines for the Analysis of Deterministic Data

It is important to remember that deterministic data have no random component. As a result, p-values from fitted statistical models do not have their usual meanings. A large F statistic (low p-value) is an indication of an effect due to a model term. However, you cannot make valid confidence intervals about the size of the effects or about predictions made using the model.

Residuals from any model fit to deterministic data are not a measure of noise. Instead, a residual shows the model bias for the current model at the current point. Distinct patterns in the residuals indicate new terms to add to the model to reduce model bias.

Results of the Borehole Experiment

The example described in the previous sections produced the following results:

•	A stepwise regression of the response, log y, versus the full quadratic model in the eight factors, led to the prediction formula column.

•	The prediction bias column is the difference between the true model column and the prediction formula column.

•	The prediction bias is relatively small for each of the experimental points. This indicates that the model fits the data well.

In real world examples, the true model is generally not available in a simple analytical form. As a result, it is impossible to know the prediction bias at points other than the observed data without doing additional runs.

In this case, the true model column contains a formula that allows profiling the prediction bias to find its value anywhere in the region of the data. To understand the prediction bias in this example:

1.	Select Graph > Profiler.

2.	Highlight the prediction bias column and click the Y, Prediction Formula button.

3.	Check the Expand Intermediate Formulas box, as shown at the bottom on the Profiler dialog in Profiler Dialog and Profile of the Prediction Bias in the Borehole Sphere-Packing Data. The prediction bias formula is a function of columns that are also created by formulas.

Click OK.

The profile plots at the bottom in Profiler Dialog and Profile of the Prediction Bias in the Borehole Sphere-Packing Data show the prediction bias at the center of the design region. If there were no bias, the profile traces would be constant between the value ranges of each factor. In this example, the variables Hu and Hl show nonlinear effects.

Profiler Dialog and Profile of the Prediction Bias in the Borehole Sphere-Packing Data

The range of the prediction bias on the data is smaller than the range of the prediction bias over the entire domain of interest. To see this, look at the distribution analysis (Analyze > Distribution) of the prediction bias in Distribution of the Prediction Bias. Note that the maximum bias is 1.826 and the minimum is –0.684 (the range is 2.51).

Distribution of the Prediction Bias

The top plot in Prediction Plots Showing Maximum and Minimum Bias over Factor Domains shows the maximum bias (2.91) over the entire domain of the factors. The plot at the bottom shows the comparable minimum bias (–4.84). This gives a range of 7.75. This is more than three times the size of the range over the observed data.

Prediction Plots Showing Maximum and Minimum Bias over Factor Domains

Keep in mind that, in this example, the true model is known. In any meaningful application, the response at any factor setting is unknown. The prediction bias over the experimental data underestimates the bias throughout the design domain.

There are two ways to assess the extent of this underestimation:

•	Cross validation refits the data to the model while holding back a subset of the points and looks at the error in estimating those points.

•	Verification runs (new runs performed) at different settings to assess the lack of fit of the empirical model.