Failings of software reliability theory

• Operational profiles are very difficult to obtain in practice.
• Automatic random testing requires a mechanical oracle to decide if the test result is correct.
• In any case, it takes far too many tests to obtain reasonable confidence in a useful reliability.
• The assumption that test points are independent is questionable. Intuitively, since many inputs cause the program to do roughly the same thing, a collection of such inputs should not lead to the statistical confidence predicted.