They've taken hits for not fully testing, which I can agree with. On the other hand, remembering the test process we had for System/36 software, I can only imagine the problems they would have had. If my imagination is right, they had these choices for beginning to end testing:
- use live data--i.e., have all the 20-something IT types try to sign up for health insurance for real using their software. That has some obvious problems, particularly when you have to cover 36 state exchanges.
- create test data. The problem here is while you can create applicants, you need to have SS numbers which meet the SSA criteria, and/or you need to create credit histories over at Experian, then you need to tack on test data for those SSN's with IRS, etc.
- use a subset of live data for test data. That's what we used to do--get a copy of a counties files in and modify the data to create test conditions. That's very problematic, both from a security standpoint and from a Privacy Act standpoint. And our FSA system was simple compared to the sort of system ACA requires.