Housing Codes--Needed or Not?

Some bloggers, maybe all bloggers, have a hobbyhorse.  Matt Yglesias fights against zoning codes, arguing that home prices would be lower if anyone could build anything on any land they own. [In fairness to him, I'm sure I'm exaggerating his position. 

Meanwhile FCW has a post praising the "faceless bureaucrats", which is my hobbyhorse, for their work in enforcing housing codes.  Matt hasn't commented on that.  I suppose he'd probably argue that building codes are essential,  that when he writes "build anything" he doesn't really mean "anything", but buildings built to fulfill their purpose.  That may be a slippery slope, however.