ACA IT and Testing

I can't resist the temptation to comment on the healthcare software process.  (BTW, here's a link to their blog.)

They've taken hits for not fully testing, which I can agree with.  On the other hand, remembering the test process we had for System/36 software, I can only imagine the problems they would have had. If my imagination is right, they had these choices for beginning to end testing:

• use live data--i.e., have all the 20-something IT types try to sign up for health insurance for real using their software.  That has some obvious problems, particularly when you have to cover 36 state exchanges. 
• create test data.  The problem here is while you can create applicants, you need to have SS numbers which meet the SSA criteria, and/or you need to create credit histories over at Experian, then you need to tack on test data for those SSN's with IRS, etc. 
• use a subset of live data for test data.  That's what we used to do--get a copy of a counties files in and modify the data to create test conditions. That's very problematic, both from a security standpoint and from a Privacy Act standpoint. And  our FSA system was simple compared to the sort of system ACA requires.