The issue with this model, and some others like it, is what’s known in the statistical business as overfitting. This occurs when the number of variables is large relative to the sample size: in this case, the full version of Mr. Enten’s model contains six variables, but is used to explain only 15 cases (Congressional elections in presidential years since 1952).Seems to me there's a relationship with our construction of narratives. The more detail, the more variables, we can stick in and still have a cohesive story the more satisfying it is. So what Silver says is that stories aren't scientific explanations, they're history.
A general rule of thumb is that you should have no more than one variable for every 10 or 15 cases in your data set. So a model to explain what happened in 15 elections should ideally contain no more than one or two inputs. By a strict interpretation, in fact, not only should a model like this one not contain more than one or two input variables, but the statistician should not even consider more than one or two variables as candidates for the model, since otherwise he can cherry-pick the ones that happen to fit the data the best (a related problem known as data dredging).
If you ignore these principles, you may wind up with a model that fits the noise in the data rather than the signal.
Blogging on bureaucracy, organizations, USDA, agriculture programs, American history, the food movement, and other interests. Often contrarian, usually optimistic, sometimes didactic, occasionally funny, rarely wrong, always a nitpicker.
Thursday, March 24, 2011
What Wisdom Do Statisticians Have
From a Nate Silver post((on a theory predicting the Republicans are almost sure to maintain control of the House in 2012):
Simulators for Everything
University Diaries points to this dairy cow simulator for vet training, and for artificial inseminators.
The Question of the Day
From Joel Achenbach:
How do we become the organized people who are ready for the Big One when the ordinary tasks of daily life — the mundane stuff — the tedious grind of being a taxpaying citizen — are already overwhelming us?
How do we become the organized people who are ready for the Big One when the ordinary tasks of daily life — the mundane stuff — the tedious grind of being a taxpaying citizen — are already overwhelming us?
Wednesday, March 23, 2011
Second Thoughts on "Industrial Farming"
I usually resist the meme among the food movement of dissing the "industrial farmer", ". However, I saw this "Agrosecurity Checklist" on the extension.org site and noted the extensive references to "employees", which is a reminder that farming has changed from my mental picture of it. I just don't think of farmers as having employees, at least not full-time employees, but many do now.
The Ride of the Valkyries--and the Furies?
Maureen Dowd writes about the idea that the Obama people who ended up pushing for the no-fly zone were women: Rice, Power, Clinton. She missed the fact that two of the people commanding the effort were Major Gen. Margaret Woodward and Rear Adm. Peg Klein. For Furies, see this.
Tuesday, March 22, 2011
The Washing Machine, and Clean Water
Via Ezra Klein, a TED talk claiming the washing machine is the great invention, by Hans Rosling. Actually, he uses it more metaphorically to discuss growth of wealth and population and green concerns. But it triggered my memory:
My mother would remember Mondays on the farm. Monday was wash day, of course. There were ironing day and baking day and I forget what else. When she was young, they had a "dog power" to run the washing machine, and their dog would know and hide on Mondays, which was a story she'd repeat regularly over the years. I never asked, but I assume they heated water for washing as we did, using a coal/wood stove, possibly with a boiler on the stove, or through a heating coil contained within the stove. I assume the washer simply agitated the clothes in the water, with a separate wringer (set of rollers to squeeze water out of the clothes. I vaguely remember the two big wash tubs used for rinsing the clothes, which then would be run through the wringer to wring out the water (could almost make a tongue-twister out of that). Then of course the clothes would be hung on the line to dry.
Mom's washing machine was a wringer washer, with no dog to run away and a wringer as part of the machine. It still presupposed a supply of heated water. And clothes were still hung to dry. It definitely required more work than today's washer which simply requires loading soap and clothes and pressing buttons.
So my point: while Rosling is right to talk about the importance of the washing machine in freeing women to learn to read, and to read books to their children, the machine itself and the detergents available to us, assume the presence of clean, preferably heated water. In that sense, public utilities, taking the human waste away by keeping it separate from the clean water provided for drinking and washing, become the greatest invention.
My mother would remember Mondays on the farm. Monday was wash day, of course. There were ironing day and baking day and I forget what else. When she was young, they had a "dog power" to run the washing machine, and their dog would know and hide on Mondays, which was a story she'd repeat regularly over the years. I never asked, but I assume they heated water for washing as we did, using a coal/wood stove, possibly with a boiler on the stove, or through a heating coil contained within the stove. I assume the washer simply agitated the clothes in the water, with a separate wringer (set of rollers to squeeze water out of the clothes. I vaguely remember the two big wash tubs used for rinsing the clothes, which then would be run through the wringer to wring out the water (could almost make a tongue-twister out of that). Then of course the clothes would be hung on the line to dry.
Mom's washing machine was a wringer washer, with no dog to run away and a wringer as part of the machine. It still presupposed a supply of heated water. And clothes were still hung to dry. It definitely required more work than today's washer which simply requires loading soap and clothes and pressing buttons.
So my point: while Rosling is right to talk about the importance of the washing machine in freeing women to learn to read, and to read books to their children, the machine itself and the detergents available to us, assume the presence of clean, preferably heated water. In that sense, public utilities, taking the human waste away by keeping it separate from the clean water provided for drinking and washing, become the greatest invention.
A Political Scientist (International Relations) on How To Arrange a Marriage
Dan Drezner passes on a great lesson in marriage planning, dressed up as commentary on Libya.
Monday, March 21, 2011
High on the Hog, Surprising Factoids
High on the Hog, subtitled "A Culinary Journey from Africa to America" is a broadbrush history of slavery and race relations focused through the prism of food, food crops, food preparation, cuisines, etc. It's well-written, although I'd quibble with a couple items where I think an urbanite showed lack of agricultural background. One was a reference to a slave being given 17 "stalks" of corn to subsist on. Possible, but more likely "ears". Another was a reference to an early writer (circa 1600?) who claimed that native Americans could raise 200 English bushels of wheat per acre. The cite may be accurate, but it shows credulity by the writer.
A couple factoids: It has the surprising claim that the death rate for sailors on ships engaged in the slave trade was higher than the rate for the Africans held captive. Although the author, Jessica Harris, is a professor, it's not footnoted within the book.
I could explain it: if the analysis includes the whole trip for the sailors, time spent off the coast of Africa waiting to fill the slave ships was notoriously unhealthy. And, there was a definite economic incentive to keep captives healthy enough to survive the Middle Passage. So the factoid might be right, but I'm still uncomfortable
Another factoid: France's Code Noir in 1685 prescribed the diet to be provided to French slaves. The U.S. federal government never had such a provision and apparently no states did either. That's a reflection of the difference in government between France and the U.S.: our governments are weaker and less prescriptive; French governments, whether monarcharies or democracies, are more centralized and prescriptive.
A couple factoids: It has the surprising claim that the death rate for sailors on ships engaged in the slave trade was higher than the rate for the Africans held captive. Although the author, Jessica Harris, is a professor, it's not footnoted within the book.
I could explain it: if the analysis includes the whole trip for the sailors, time spent off the coast of Africa waiting to fill the slave ships was notoriously unhealthy. And, there was a definite economic incentive to keep captives healthy enough to survive the Middle Passage. So the factoid might be right, but I'm still uncomfortable
Another factoid: France's Code Noir in 1685 prescribed the diet to be provided to French slaves. The U.S. federal government never had such a provision and apparently no states did either. That's a reflection of the difference in government between France and the U.S.: our governments are weaker and less prescriptive; French governments, whether monarcharies or democracies, are more centralized and prescriptive.
Politicians Break Promises, Even Their Wives
From Obamafoodorama:
In an interview with the New York Times given the day before the groundbreaking [of the White House garden], Mrs. Obama announced that the entire First Family, including President Obama, would be pulling weeds in the Kitchen Garden, “whether they like it or not." With one exception.But when an aide was asked about follow-through:
“Now Grandma, my mom, I don’t know,” Mrs. Obama said of her mother, Marian Robinson, working in the Kitchen Garden.
Mrs. Robinson "would probably sit back and say: “Isn’t that lovely. You missed a spot,”" Mrs. Obama said.
The aide said she wasn't aware of any Presidential or First Daughter weeding or harvesting action, but added that it might be going on "during private family time."If I remember correctly, I was always dubious about the premise. Given the schedule I presume Mrs. Obama maintains, it would be very hard for her to herd her family to the garden on a regular basis.
"You never know," the aide said. She added that Mrs. Obama, in her interview, "may have been joking" about the First Family weeding the Kitchen Garden.
Saturday, March 19, 2011
When Is an Earmark an Earnmark
The Sustainable Ag Coalition believes that ATTRA lost its funding because Congress thought it was an earmark:
One very distressing casualty of the continuing series of Continuing Resolutions that are keeping the government open but cutting funding week by week is the National Sustainable Agriculture Information Service, known as ATTRA. ATTRA’s $2.8 million was cut entirely in H.R. 1, the House-passed full-year Continuing Resolution from mid-February and that proposed program termination was unfortunately including among the $6 billion in cuts adopted by Congress this week in the new short-term Continuing Resolution keeping the government operating through April 8.What they don't mention is that the National Sustainable Agriculture Information Service is not a federal agency, as the name might imply (and I first thought). It's the outcome of a cooperative agreement with the Rural Development Service--in other words federal money provided to a cooperative. Here's the general blurb from RD:
The justification for cutting ATTRA appears to be a misperception that it is an earmark. Indeed, like earmarks, many Senators and Members of Congress request funding for ATTRA every year, as they do for many programs. However, unlike earmarks for projects in specific congressional districts, ATTRA is a nationwide program, authorized in the 2008 Farm Bill, and it has been included in presidential budgets through many administrations over several decades.
We have over 80 years of experience working with the cooperative sector and remain the only federal agency charged with that responsibility. USDA Rural Development has been providing support to cooperatives since the Cooperative Marketing Act of 1926, promoting the knowledge of cooperative principles and practices as well as collecting statistics on cooperative activities. The Cooperative Program provides assistance for rural residents interested in forming new cooperatives and administers programs that fund value-added producer grants, rural cooperative development centers, and small socially-disadvantaged producers.Now if the appropriation is specifically for that cooperative agreement, it comes pretty close in my mind to an earmark. If RD is given a lump sum of money for cooperative agreements and decides to give $2.8 million to ATTRA, then it's not.
We also provide resources to local cooperatives to support a department-wide effort known as, 'Know Your Farmer, Know Your Food'. This initiative, led by Deputy Secretary Kathleen Merrigan puts an increased emphasis on regional food systems, which will have direct and significant benefits to rural communities. Lear [sic] more here:
Subscribe to:
Posts (Atom)