Failings of software reliability theory
There are lots of bugs in the theory.
- Operational profiles are very difficult to obtain in practice.
- Automatic random testing requires a mechanical oracle to decide
if the test result is correct.
- In any case, it takes far too many tests to obtain reasonable
confidence in a useful reliability.
- The assumption that test points are independent is questionable.
Intuitively, since many inputs cause the program to do roughly
the same thing, a collection of such inputs should not lead to
the statistical confidence predicted.