What he doesn’t (need to) mention is the way that journalists, myself included, read economics papers: we generally have no ability or inclination to try to understand the details of the formulae and regression analyses, so we confine ourselves to reading the stuff in English, and work on the general assumption that the mathematics is reasonably solid.Thankfully, he sees a problem with this:

The problem of course is that we really have no basis for making that general assumption: we make it not because we think it’s particularly justified or justifiable, but because we don’t have any choice. What’s more, because we’re always interested in what’s new, and because we have easy access to the internet and little access to expensive journals, we gravitate to preprints at sites like SSRN, rather than papers which have gone through peer review.Coincidentally, over the last few days, I have had an email discussion about this very same subject with Iris Mack, who has a Doctorate in Applied Mathematics from Harvard. She wrote to me:

I worry about this. The blogosphere is full of interesting debates between people who understand and respond to what everybody else is saying. But the minute that economic papers get cited, the degree of understanding plunges, and most bloggers and journalists are cowed by all those equations into simply assuming that it all stands up somehow.

...what I see over and over again are folks (esp in social sciences - who usually were not good at math to begin with) take a little bit of math and run with it as if they know more than the top PhD mathematicians. No offence, but IMHO economists are the worse.This is pretty much what Murray Rothbard used to say, serious mathematicians laugh at the math employed by economists. Indeed, the math does little than snow those who have an even weaker math background such as most journalists. Fortunately, I think Salmon senses this, but doesn't understand the underlying methodological problems with using math for empirical theory building in the social sciences.

I recommended to Iris, the best, in my opinion, book on the proper methodological foundation for the social sciences and the problem with empirical theory building, Friedrich Hayek's The Counter Revolution of Science. Maybe Salmon should read it, also.

The assumption ..."to give the mathematics the benefit of the doubt"... and to focus attention on the written language narrative is not just a problem for journalists. Nor is it a problem confined to economics.

ReplyDeleteI would strongly recommend A.W. Montford's recent book "The Hockey Stick Illusion" for a detailed and in-depth unpacking of the pre-eminent statistical conjuring trick of our era.

Just to add a bit more...

ReplyDeleteThe following extract from an Amazon review illustrates some of the detail "The Hockey Stick Illusion" examines. The reviewer gives his own interpretation but it strikes me as the reasonable conclusion.

"What is now clear is that the Mann conclusions, far from being based on coherent evidence across a geographical widespread range of proxies all showing similar patterns across the Northern hemisphere, were based on a tiny subset of proxies, bristlecone and foxtail pines, from California whose anomalous 20th century growth was almost certainly not caused by high temperature. The apparently broad evidence was an illusion created by an eccentric implementation of a standard statistical technique called principal components analysis. Mann's version of this (which appears to be his own creation) effectively mined his hundred plus proxies for any which had hockeystick shapes and then gave them huge weight in the analysis. What is worrying about all this is not so much the fact that a paper is wrong. It is the failure to admit this when it is perfectly clear that it is wrong. Montford documents the evasions of debate and the consistent misrepresentation of what McIntyre and McKitrick actually said, as well as multiple refusals of access to data and clear descriptions of what had actually been done. By the time of the 2006 Wegman report it was clear that the hockeystick was broken, but it seems too much had been invested in it for people in paleoclimate to admit outright that it was just wrong..."

Bob Murphy linked to this article a while back tittle, "On Farmers Who Solve Equations". I think it's really good.

ReplyDelete