Understanding Response Variables in Design of Experiments
What are the critical aspects of the experimental response?
-
Each specific question for your study requires one or more specific observable response(s).
-
Each response should be a measurable outcome.
- Quantitative responses are more informative than qualitative responses.
-
The measurement system should be valid for each response.
What are the goals the experiment can investigate for your response?
The experimental goals for a response could be to maximize, minimize, or match a target for that response.
How do you manage multiple responses in an experiment?
It is common to collect data for multiple responses in the same experiment. You can model each response separately (ignoring the correlation between the responses); or you can allow the statistical model to balance the needs of all your responses in a single model through optimization.
Identifying the responses
After you've defined your experimental objectives, you need to identify your response of interest. You also need to ensure that your measurement systems are capable.
Your response is the output variable of interest. For a given experiment, you might have more than one response. The response variables should provide useful information about the process characteristic that you are studying.
Wherever possible, you should use continuous responses. This is because experiments are usually small and efficient. You are trying to learn as much as you can with a limited number of runs.
Continuous data contain much more information than categorical data. So, if you have a limited amount of data, you can usually learn more from continuous data than you can from categorical data.
If your primary response is categorical, you should consult with an expert for advice on how to proceed.
For each response, you need to determine how the response will be measured. For example, which measurement gauge or instrument will be used, what is the unit of measurement, who will take the measurements, and how many decimal places will you record.
Beyond the basics: Special analyses for your measurement system
For each response, you also need to ensure that the gauges, or measurement systems, are capable. When a measurement system is capable, it means the system measures the characteristic of interest both accurately and precisely enough for the process. If a measurement system is not capable, you can't trust your data.
You might have a lot of variation in the response due entirely to the measurement system, or the measurements might be biased (or inaccurate). You might need to conduct a measurement system study before running the experiment. If the measurement system is incapable, you might need to improve it or find an alternative measurement system.
For more information about measurement system studies, see the Quality Methods module in JMP’s free, online statistics course.
Examples
Here are some descriptions of experiments with special attention on their responses.
- Tablet Potency. When developing a pharmaceutical tablet, you want to ensure that the final product has the correct potency of the active ingredient present, within an acceptable range above or below that amount.
- Cost. You manufacture the weather-stripping system used to seal car doors. There are several components that make up the seal, and your goal is to minimize the overall cost of the car door seal.
- Paint Viscosity. You want to find broad ranges of factor settings under which your process for creating paint results in stable viscosity. That is, even when varying the factor settings within those ranges you need to keep the viscosity response consistent.
Response goals
Different experiments have different goals for the behavior of the response. Some common goals are:
- Finding the best settings of the factors in order to hit a target with the response. For the tablet potency example, you want to match a specific target range for the drug potency (e.g., a “200 mg caffeine tablet” should have between 198 and 202 mg caffeine).
- Finding the best settings of the factors in order to maximize or minimize the response. For the cost example, you want to minimize the cost of the door seal system.
- Finding the best settings of the factors in order to make the process robust to small changes in the factor settings. For the paint viscosity example, you want to find settings that achieve a good viscosity result and then study the stability by perturbing the settings with some random variation and testing whether those setting ranges remain stable.
Experiments with Multiple Responses
It is possible (and often desirable) to include multiple responses in the same experiment. That is, when you are interested in more than one outcome from the same set of experimental factors, you can collect data on these multiple responses in each run of the experiment.
Consider the car door seal experiment described above. The response was the cost of the weather-stripping system. But to be effective, you also want a door seal with minimal leaks of rain or wind that doesn’t take too much force to close. In this example, meeting the goals for these three responses at the same time (minimizing the cost while also minimizing leaks and keeping the force required to close the door below some threshold) might require tradeoffs.
Those tradeoffs can mean that you can’t perfectly optimize all the responses at the same time, and you might need to consider whether one response is more important than the others. Statistical software will often require you to specify this in the form of a ratio or an importance weight. For example, you might decide that the goal for the door seal leaks is five times as important as the goals for the cost and force. In this case you could say that the importance of both cost and force is one and the importance of leaks is five. The software will now weight the goal for the leaks five times more than the other goals in the optimization.