Design of Experiments Workflow

Style

section-padding-none

What is the design of experiments (DOE) workflow?

Design of experiments (DOE) is a structured approach to planning, conducting, and analyzing experiments to better understand how multiple input variables (factors) affect an output variable (response). A designed experiment can help us identify cause-and-effect relationships and optimize our processes or products. The DOE workflow describes a typical sequence of steps for properly executing a designed experiment.

The DOE workflow consists of six steps: define, model, design, data entry, analyze, and predict. The framework for designing an experiment remains the same regardless of the method of experimental design you choose.

Define: Identify the experimental purpose, response(s), and factor(s).
Model: Propose or specify an initial statistical model.
Design: Generate and evaluate an experimental design that can support the initial statistical model.
Data entry: Collect data for each row (run) in the design.
Analyze: Fit a statistical model to the experimental data to answer your research questions.
Predict: Use your confirmed statistical model to predict future values of the responses.

While these steps relate to a single experiment, a sequence of experiments may be required to achieve the experimental purpose. Subject matter knowledge is vital to all the steps in DOE.

Figure 1: The DOE framework consists of define, model, design, data entry, analyze, and predict.

Define

The purpose of the experiment should guide your design choices throughout the DOE process. There are a series of questions to answer during the define step, and all of them relate to what information you would like to have at the end of the experiment.

Questions to answer during the define step of the DOE framework include:

What is the purpose of the experiment?
What are the responses that you want to measure?
What are the response goals (e.g., maximize, minimize)?
What are the factors that affect the responses?
What are meaningful ranges or levels for the factors that will produce measurable effects on the response(s) of interest?

Two common purposes of DOE are identifying important factors from a large set of factors (commonly referred to as “screening”) or characterizing and optimizing a process. Your experiment might have more than one purpose.

After you have established the purpose, the response and factor are defined. The response is what is measured during the experiment; the factor is what changes during the experiment to understand its impact on the response.

Figure 2: For this example, we are interested in optimizing a process by identifying which factor settings produce the greatest yield and lowest impurity. Yield and impurity are the responses; pH, temperature, and vendor are the factors.

Model

During the model step, an initial statistical model is specified. A designed experiment is a collection of trials performed to support a proposed statistical model. The statistical model you specify is directly related to the purpose of the experiment. First-order models (main effects only) are commonly used to identify active factors (screening), while second-order models (including interaction and quadratic terms) offer greater flexibility for predicting and optimizing responses. Depending on the design approach you use, the statistical model will be implicit in your design choice, or you can specify a statistical model that only includes the model effects that you want to estimate.

Figure 3: An initial model of a Response Surface Design is chosen with 18 runs.

Design

During the design step, a design is generated based on the choices made during the model step. Each row in the design is a run and contains the factor combinations you want to test. The design consists of the number of runs needed to estimate the proposed statistical model, plus a few extra runs to estimate experimental error. Design evaluation also takes place during this step. Design evaluation is a set of tools we use to understand the design’s strengths and limitations and to ensure that our design provides the information needed given the purpose of the experiment. Design evaluation can be used to compare two or more designs to understand the tradeoffs.

Figure 4: The design data table for an 18-run experiment.

Data entry

For the data entry step, the experiment is executed following the design run order; the responses for each run are recorded in the data table.

Figure 5: The experiment is run by testing each of the rows of factor combinations in the design table. The response values are recorded in the data table.

Analyze

During the analyze step, the initial "full" specified statistical model is fit to the experimental data. When screening, the model consists of main effects, or sometimes some or all of the two-factor interactions. When you want to predict or optimize the process, the model typically includes second-order effects (two-factor interactions and quadratics). A common statistical model is a multiple linear regression model. Inactive (non-significant) effects can be removed from the initial, full model to create a reduced model. If more than one response is collected during the experiment, an individual model is fit for each response.

Figure 6: A regression model is created using the initial model of the effects to estimate. Solid green lines are terms that are statistically significant and influence the response yield. The “Full” model is shown.

Figure 7: The "Best" model button will reduce the model by eliminating the inactive terms from the "Full" model. The “Best” model is shown.

Predict

During this step, the reduced model from the analyze step is used to predict future values of the response, and, if response goals have been specified, find factor settings that are predicted to meet those goals. The model is an interpolating model, meaning that it can make predictions for any levels of the factors within the factor range, even if the design did not test those particular levels.

Figure 8: The Prediction Profiler explains which factor settings produce the greatest yield and lowest impurity. In this case, a pH of 7.14, temperature of 33.70, and the fast vendor will produce the greatest yield and lowest impurity.

layout

2 column

Style

columns-75-25, section-top-padding-xsmall