Saturday, August 01, 2015

Good Data Modeling Is Important--MP3s

I was usually ambivalent about the IT contractors working for ASCS/FSA. On the one hand I always thought I we could do the job better and faster ourselves, if management would only give us the dollars/people.  On the other hand I have to admit I did learn a lot, even if all the contractors didn't accomplish much (at least in my fallible memory).

One of the things I did learn was data modeling, normalizing data structures and the importance of having it right.  For example, the System/36 just used flat files, with multiple indexes to them.  While the IT people in IRMD and KCMO did a pretty good job in capturing the data needed for some of our activities, one big thing was missed: time, most notably time as reflected in crop/fiscal/program years, and the idea that we could be operating programs for different years at the same time. 

Of course you never get the thing totally right.  For example, our "name" structure assumed the standard WASP structure: first name, middle name, last name, and failed to allow for the naming structures present in other parts of the world or other cultures.

Mistakes in data modeling can have big consequences, as seen in the case of the metadata scheme for MP3 files, as discussed in this piece at The Atlantic.  Classical music is particularly affected.