Yale and Public Information-Sshhh

Colin McKay at SoSaidThe Organization secretly tells us of a document we are commanded not to cite from Yale Law. It's well worth reading. It proposes [emphasis removed]:

In order for public data to benefit from the same innovation and dynamism that characterize private parties’ use of the Internet, the federal government must reimagine its role as an information provider. Rather than struggling, as it currently does, to design sites that meet each end-user need, it should focus on creating a simple, reliable and publicly accessible infrastructure that “exposes” the underlying data. Private actors, either nonprofit or commercial, are better suited to deliver government information to citizens and can constantly create and reshape the tools individuals use to find and leverage public data. The best way to ensure that the government allows private parties to compete on equal terms in the provision of government data is to require that federal websites themselves use the same open systems for accessing the underlying data as they make available to the public at large.

Some comments:

• the authors could productively cite the GPO's "reengineering" initiative--as GPO is officially responsible for all government documents (repository libraries) it particularly aggravates me that their effort seems to be a solo silo, (see this link) particularly as their aim is to please their library stakeholders, not the public.
  
• the authors do not deal with the problem of private data, that is, data that can't be made public. Their examples include FCC dockets, regulations, Congressional actions (bills, votes, etc.), SEC filings--all things that are supposed to be public. In other words, they're viewing government as lawyers, and the data they want is lawyers' data. They might profitably look at the EWG's database of federal payments to farmers--a long-existing example of the problems (and gains) in providing government data on-line.
  
• I doubt the practicality of the suggestion (that is, considered as a government-wide, top down initiative). They note the number of constraints agencies have to deal with in handling data. Each of the constraints was the result of some interest group and/or Congressional members putting their oar in. That's the way our government works. Perhaps in a parliamentary system the proposal is feasible, but not here. Appropriations committees will not give dollars to such good government suggestions. And a President Obama or McCain is unlikely to use political capital to take real action.
• I think the most likely outcome is a gradual, evolutionary, scattershot approach which, after 20-30 years or so, ends up maybe close to what the authors want.

(I'm struck by this language: "The best way to ensure that the government allows private parties to compete on equal terms in the provision of government data is to require that federal websites themselves use the same open systems for accessing the underlying data as they make available to the public at large." 10 years ago I was proposing the same thing to FSA, NRCS, and RD--an Internet-based front end to access documents and data with a gradual back end migration from the legacy databases and indexed files to modern web-servers. Even had a working mockup website to this effect. Alas, I'm no salesman. As far as I can see, the key technical constraint was the need to provide security at the data element level--i.e., maybe you want to show to everyone that John Doe owns a section of land in Mills County, Iowa, but you don't want to show his SSN to anyone except him and one or two people who serve him. That's tough to get the "business rules" decided on. The political constraint was the necessary impact on the organizations involved. Agencies whose raison d'etre was to serve local farmers would have to consider whether and how to change. And the politicians whose livelihood depends on customer service, meaning being able to see that the local office works for their farmers, would have to cede influence.)