That's the theme of this NYTimes article, describing farms which went organic in 2005 or 6, and now are having problems. I never got into the business details of our farm, so I'm not sure who we shipped our milk to and whether there was a contract. I suspect not. Apparently organic dairies have contracts with their processor, presumably to ensure compliance with organic standards?
One of the problems the organic people run into is the math of a niche market. Generally speaking, the bigger the market, the more fluctuations will damp themselves out. (Unless, that is, you have a bubble like the subprime or dot-com ones--then the bigger the market the harder the fall.) So the article mentions the possibility of selling milk into the conventional milk market, or trying to sell locavore/raw milk.
One result of the problems will be the less efficient organic producers will fail, meaning the average size will increase, moving organic dairy further away from the organic ideal.
[Updated: Of course things aren't good for conventional dairy either, as this LA Times story says.]
[Updated II: John Phipps comments on the same article. I'm struck by the fact that even for organic dairies the cost of [bought, I assume] feed is 50 percent or more of total costs.]