LAUSR

Covering arrays are increasingly being used by test engineers to derive test cases to test complex engineered systems. This approach to testing is known as combinatorial testing and has proven to be a cost-efficient way to determine test cases that are highly effective at identifying faults in the system that are due to the combination of several inputs. However, when such faults are encountered and failures occur, the test engineer is tasked with determining the inputs and associated values that triggered the failures. This exercise typically involves examining a long list of potential causes and may even require performing follow-up tests to reduce the number of potential causes. This paper addresses this issue by considering the prior knowledge of the system under test that test engineers often have. We show how this knowledge can be used to evaluate and improve the effectiveness of a suite of test cases before any test cases are executed. Finally, we address the case where failures occur and show how this prior knowledge can aid in determining the inputs, and associated values, that triggered the failures. In addition, throughout the paper, we compare and contrast the use of covering arrays for testing complex engineered systems to the use of factorial experiments in traditional experimental design settings.