Just finished reading The Confidence Game,--anyone who enjoyed The Sting and the David Mamet film House of Games (which inspired the book) will enjoy it. The author views con artists running con games as employing human traits, the desire to believe, the desire for meaning, the reluctance to cut one's losses, etc.we all share. With that perspective, I was struck by the question in my title.
How did I get there? It's true, I believe, that most massive IT projects, possibly especially those in government, fail; the success rate is maybe 30 percent. With that sort of track record, why do we in government keep creating and funding the projects and why can IT contractors get contracts to run them? Surely if Elon Musk's space venture only got into orbit 30 percent of the time, he'd fail to attract venture capital. But as far as I can tell (not very far), no big IT contractor has gone out of business because they can't get any more contracts. So why?
Maybe they're running a con game? After all that in the beginning there's lots of enthusiasm, enough to sweep agency employees, agency officials, even OMB and Congress into supporting the project. A big project may paradoxically be easier to sell than a small one: a big project has meaning, it offers to change many things, to solve lots of problems, etc. etc. For IT projects it's likely that management and Congress don't really understand the nuts and bolts; they just know that people who should know, who seem to know, claim it can work, can succeed. In the early stages it's easy to use the project focus to find more improvements to make, problems to solve, things to be folded into the project scope. And once you're committed to a project, your reputation is involved, there's money been spent, meetings have been held, promises made. And the problems surely are fixable, no need to abandon hope, just spend a little more money here, work some more hours there, move the schedule back just a little.
Finally there's a loss of confidence by those who should know, an increasing desperation, and Congress and management cut their losses, a process made much easier because there's been turnover in both areas so they aren't killing their own baby, it's some else's bastard child. That can in turn make it easier for those who know (who haven't retired or moved to higher paying private jobs) to blame the big shots for not keeping the faith.
Meanwhile the IT contractors can move on to run another con.
Note: I don't necessarily think IT contractors are knowingly con artists; they may be conning themselves as much as their customers and they do have the occasional success.