Buggy Software Costs $59.5 Billion

by Ed Sawicki

June 29, 2002

The National Institute of Standards and Technology (NIST) has issued the findings of a study it conducted on the cost of buggy software. The cost, $59.5 billion, is borne mostly by consumers of the buggy software. They say "few products of any type other than software are shipped with such high levels of errors."

The study found that over half of all bugs were discovered well after the code was written - mostly when the software was already in the hands of users. Clearly, this suggests a lack of testing but the NIST report says that about 80% of software costs are spent on testing. Something is really wrong. Why isn't testing finding the bugs?

NIST blames this mostly on growing complexity. Popular software is now measured in millions of lines of code. Both programmers and their testing tools are challenged when code size grows very large. But there are other reasons as well:

1. Marketing strategies - Time-to-market pressures force programmers to take shortcuts to meet agressive schedules. This helps explain why, for example, a company can hire brilliant programmers but still ship awful products.
2. Limited liability by software vendors - Since software vendors are largely insulated from lawsuit, there's little incentive for market leaders (monopolists) to reduce bugs.
3. Decreasing average product life expectancy - Many commercial software vendors keep profits high by eating their young. They make their products obsolete within a relatively short time and force consumers to upgrade to the next version of buggy software. Each new version may fix some old bugs but introduce new ones. The cycle is never-ending.

The study does not mention vendors, hardware/software platforms, or software categories as the worst offenders. This is unfortunate because readers could easily get the impression that the entire industry suffers from these problems equally, which is not the case. Generally, open source software quality is much higher than commercial software because its creators are unconcerned about marketing, profits, and liability.

The problems mentioned above are self-inflicted. Vendors choose to have these problems. A vendor chooses to keep growing a program until its complexity is such that it can no longer be properly maintained. No programming class or book in the world instructs programmers that "bigger is better". In fact, the reverse is true. Every Unix programmer has been taught that "small is beautiful".

The problem of buggy software was created by the way the commercial software world does business. The solution must come from vendors making business decisions. This is not likely to happen. Buggy software means higher profits.

Reference

The NIST report is here: Software Errors Cost U.S. Economy $59.5 Billion Annually and a plain text copy is here.

Another article on the cost of software bugs is The True Cost of a Software Bug