There’s a problem with configuration. It’s called the combinatory explosion.
When combining the building blocks of a typical product, number of possible combinations will explode. Let’s assume there are a quadrillion ways to configure your product. Out of these more than 50 % will be invalid.
If we were to test quadrillion solutions we will test something very close to infinity. The big drawback is that we will not live to see the result!
How to survive
So how do we do it? How do you test configuration logic? How do we survive BA-AM of the combinatory explosion?
Use the correct data
First of all, we need to confirm that the data we use is correct. If we don’t define our building blocks in a correct way we will always be wrong. Right?
Say we’re configuring a car. Then we need to check that the specified power for each variant of the motor has the correct feature value. The same goes for every feature of every single building block.
Validating the data is fundamental. We need to make sure we specify the connecting features correctly. Otherwise the constraints connecting the bits and pieces will not do any good.
Piece by piece
Secondly we need to test the logic piece by piece. We need to split the logic and isolate separate parts. This eliminates the domino effect often seen when we combine the logic of this and that.
If we’re configuring a car there will be a few features connecting and restricting the selection of engine and gearbox. Once we consider the chassis and other parts of the vehicle these connections become more and more intricate. Therefore we should isolate and test the different islands of logic before we connect all the dots.
Manual validation
Thirdly, we should do some manual validation. We just do some representative selections and review the outcome. The propagation – the configurator’s ability to see the consequences of the choices we make – is a very good way to understand the consequences of what we just selected.
Define test-cases
The fourth and final thing is to define some test-cases.
The more general we can make the test-case the better. In plain text a general case will be something like “no matter what type of soft drink we put in the container 50 cl and 12 oz. should always be selectable”.
Think of test cases that will stand the test of time and that are relevant to actual sales (now and hopefully forever).
This is how to survive the BA-AM of the combinatory explosion. We know from experience that we will come a long way with a few hundred test cases.
That’s how you avoid the 32 000 years of full scale testing.
Perspective:
With 1015 solutions (and for sure many of our customers exceed this number) and a typical test cycle of 1 MS it will take 277 million hours or 11,5 million days or 32 thousand days to finish the testing.