Testing the Prediction Profiler with Disallowed Combinations

Combinatorial testing is a design-of-experiments inspired approach to software testing that is based on a mathematical object known as a covering array. This approach to testing is both effective and efficient in that the resulting test cases have a high probability of discovering faults that are due to the interaction of two or more inputs, and this is achieved with fewer test cases than other approaches. For complex software systems, finding faults due to the interaction of two or more inputs is a particularly challenging problem for test engineers.

An example of such a complex software system is the JMP Profiler, an interactive tool that allows a user to explore the relationship between multiple factors and responses as well as to find optimal factor settings for those responses. In addition, the tool allows for exploration even when the factor space is constrained by disallowed combinations of settings for those factors. In this talk, we present a case study that describes how test engineers used a combinatorial testing approach to test the profiler.