Big Software Projects Fail

Note the following is not unique to government. I firmly believe that organizations of people are more similar than different.

The FBI ran into problems with its big software project, leading to an op-ed in

The New York Times > Opinion > Op-Ed Contributor: Does Not Compute

"Research by the Standish Group, a software research and consulting firm, illustrates the troubled fates of most big software initiatives. In 1994, researchers found, only 16 percent were completed on time, on budget and fulfilling the original specifications. Nearly a third were canceled outright, and the remainder fell short of their objectives. More than half of the cost overruns amounted to at least 50 percent of the original budget. Of the projects that went off schedule, almost half took more than twice as long as originally planned. A follow-up survey in 2003, however, showed that corporate software projects were doing better; researchers found that the percentage of successful projects had risen to 34 percent."

I was involved in several such projects at USDA (i.e., large computer systems covering multiple offices and agencies). That led to my formulating Harshaw's rule one (you never do it right the first time). Other factors for failure:

1. the culture gap between information technology (IT) (used to be EDP, then ADP) types and the operational types who really call the shots.
2. McConnell's rule -- consider the goals of a project: cheapness, speed, quality. It's easy to achieve one, difficult to achieve two, and all but impossible to achieve all three. (I think I stole that from Steve McConnell's "Code Complete", published in 1993.) It fits my experience, although Dan Goldin, the former NASA head had the motto: Faster, Better, Cheaper.
3. hubris. See the classic tragedies. See "A Bridge Too Far." See the Spruce Goose.