Types of Experimental Designs
Which designs are commonly taught in design of experiments courses?
Full factorial, fractional factorial, central composite, and Box-Behnken designs are common designs taught in university courses and industry short courses on experimental design. These designs are examples of classic, or textbook designs.
How are algorithmic designs different from classic or textbook designs?
Algorithmic designs, sometimes called modern designs, use an algorithm and an optimality criterion to determine the best set of runs to meet your budget, types of factors, and the statistical model you want to fit.
Which designs can help identify the important factors in a relatively small experiment?
Screening designs are primarily used to find important factors (main effects) and, in some cases, to test for interactions among the factors (interaction effects). The results from screening experiments guide the choice of factors and effects to include in subsequent experiments.
Which designs can help you find the best settings to optimize one or more responses?
Response surface designs are used to understand potential interactions among important factors and any curvature in the relationship between continuous factors and the response (quadratic effects). These designs are used to identify optimum factor settings to meet response goals.
There are many types of experimental designs, and the design that you use depends largely on your experimental goal. It also depends on other factors, such as the cost of running the experiment, resource constraints, and practical limitations that you might encounter when conducting the experiment.
Here we describe some popular types of experimental designs and when you might use them.
Classic designs
Full factorial designs
In a full factorial design, all possible combinations of the factor levels are tested.
The most common full factorial design is the 2k full factorial. In a 2k full factorial design, there are k factors and two levels for each continuous factor. This results in 2k combinations of factor levels, or treatments. Full factorial designs can get very large if you have a lot of factors, so they are typically only used when you want to study a very limited number of factors and their interactions. In practice, you generally won't start with a full factorial design. Early in your experimentation, you might have a long list of potentially important factors.
Take a deeper dive
Watch this tutorial (5:11) to learn more about designing a full factorial experiment.
Classic screening designs
Conducting a screening experiment can help you narrow down a long list of potentially important factors and interactions to only a few important effects. Screening experiments are usually small and efficient, involving many factors. They are often used for exploratory purposes (for example, to identify a handful of important effects) before conducting subsequent designed experiments for process improvement or optimization.
Fractional factorial designs
One widely used family of screening designs is fractional factorial designs. Fractional factorial designs are created by splitting 2k factorial designs in half "r" times. Suppose that you have a 2k factorial design with seven factors. A 27 full factorial design has 128 treatments, even with only two levels per factor! In most cases, it isn't practical, necessary, or even possible to perform 128 experimental runs. A 2k-r fractional factorial design uses a subset of runs from the full factorial design.
Take a deeper dive
Watch this tutorial (2:10) to learn more about designing a fractional factorial experiment.
Imagine a 25 full factorial design, where each factor can be run at two levels, –1 and +1. This design has 32 treatments. In a 25–1 design, the full 25 design has been split in half one time, resulting in 16 runs instead of 32. If you split the design in half a second time, you have a 25–2 design, requiring eight runs.
This 25–2 design enables you to study the five factors, and two of the possible two-way interactions, in only eight runs.
There’s another type of fractional factorial design called a Plackett-Burman design. The number of runs for a 2k-r fractional factorial design is a power of two, so the number of runs for these designs increases quickly as the number of factors increases. The number of runs for a Plackett-Burman design is a multiple of four, so they can be a nice alternative. However, in any type of fractional factorial design, the estimates for some – or even many – effects will be aliased, or confounded, with one another. Learn more about fractional factorial designs and aliasing of effect (Lesson 2 of JMP’s Classic Design of Experiments course).
Classic response surface designs
Response surface designs are used when you have identified important factors and their interactions and your experimental goal is optimization. For example, you might want to find settings of your factors that minimize or maximize a response or that enable you to hit a target.
When you’re trying to find a response optimum, you also want to consider that there might be curvature in the response. These designs are used with continuous factors to model potential curvature in the relationship between the factors and the response. To estimate curvature, the design requires at least three levels for the factors. As a result, response surface designs can get extremely large unless the number of factors is limited.
The most common types of classic response surface designs are central composite designs and Box-Behnken designs.
Algorithmic designs
Full factorial, fractional factorial, central composite, and Box-Behnken designs are often called "classic" or "textbook" designs, because of their long history and widespread use. But there have been many advancements in the field of DOE, specifically the development of algorithmic designs, which are often referred to as "modern" or "computer-generated" designs. Two important types of algorithmic designs are custom (or optimal) designs and definitive screening designs.
Custom designs
Custom, or optimal, designs use an algorithmic approach to generate a design based on your experimental goal (e.g., screening or optimization), budget (how many runs you can afford), and particular problem (types of factors you want to include or specific effects that you want to estimate). For example, consider a scenario where you are studying four factors, and your experimental goal is optimization. Three of the factors are continuous, and the fourth is a two-level categorical variable. You can only afford to conduct 20 runs.
Which type of experimental design should you use? None of the classic designs can accommodate this situation. First, classic designs don’t allow you to include categorical factors. You would have to perform two experiments (one for each level of the categorical factor) of at least 15 runs each. Second, even if all of your factors were continuous, the smallest classic response surface design for four factors is 26 runs.
Instead, you can generate a custom design that meets your specific experimental requirements. In this example, a custom design that allows you to estimate the main effects, interaction effects, and quadratic effects can be performed in as few as 14 runs, well below your budget! Custom designs are more flexible and can be much more efficient than classic designs.
We’ve discussed both classic and custom designs. As a practitioner you will likely use custom designs more often because they support such a broad class of experiments, from screening to optimization, and give you far more flexibility. And in fact, classic designs are actually a subset of custom designs.
Definitive screening designs
In 2011, a new class of experimental designs was introduced by Bradley Jones and Christopher Nachtsheim. Definitive screening designs are highly efficient designs, where each continuous factor has three levels and each categorical factor has two levels. Definitive screening designs enable you to study many factors at once, to help identify the most important ones. They have many advantages over classic screening designs.
While definitive screening designs are generally considered specialized screening designs, they enable you to estimate main effects and quadratic effects (for continuous factors), and when only a few of the factors are important, you can also estimate some of the interaction effects. This means that definitive screening designs can also be used for optimization.
Summary
Here we’ve discussed some experimental designs commonly used in industrial settings and when you might use them. However, there are many designs not discussed here. It’s also worth noting that designed experiments are not limited to industrial environments. DOE is widely used in other settings, such as marketing, agriculture, and the health and life sciences.