Bad Economics Hits the Big Screen

I have never been a big fan of the book, Freakonomics, so news that there is now a  Freakonomics documentary film didn't exactly cause me to react with the kind of excitement that you could detect on a Richter scale.

Co-author Stephen Dubner announces the film at the Freakonomics blog.

Judging from Dubner's comments it appears the film focuses considerable time on what I view as some of the books weakest points.

Econometricians as Bad Football Coaches

Econometricians are all over themselves congratulating a losing move in this week's Indianapolis Colts versus New England Patriots football game. Econometrician Steven Levitt sets the scene:

I respect Bill Belichick more today than I ever have.

Last night he made a decision in the final minutes that led his team the New England Patriots to defeat. It will likely go down as one of the most criticized decisions any coach has ever made. With his team leading by six points and just over two minutes left in the game, he elected to go for it on fourth down on his own side of the field. His offense failed to get the first down, and the Indianapolis Colts promptly drove for a touchdown.

Here's his defense of Belichick's move:

The data suggest that he actually probably did the right thing if his objective was to win the game. Economist David Romer studied years worth of data and found that, contrary to conventional wisdom, teams seem to punt way too much. Going for a first down on fourth and short yardage in your end zone is likely to increase the chance your team wins (albeit slightly).

Here are the problems with Levitt using Romer's paper as justification for Belichik's move.

1. Romer only studied first quarters of football games not fourth quarters.

2. Second Romer used third down data not fourth down data!

Romer writes:

Decisions to go for it on fourth down (i.e., not to kick) are sufficiently rare, however, that they cannot be used to estimate the value of trying for a first down or touchdown. I therefore use the outcomes of third-down plays instead. I then compare the values of kicking and going for it to determine which decision is better on average as a function of where the team is on the field and the number of yards it needs for a first down or touchdown.

This demonstrates one of the problems with econometricians. They are often not even working with the data that will supposedly yield answers to their questions. I have to wonder if Romer has ever watched a professional football game from the view of anything but a college cheerleader. Does he really think third down, first quarter data is going to tell him anything about fourth down, fourth quarter success rates? Does he realize that any football team in the first quarter is playing with a much wider field that the defense must guard against and a much greater number of possible plays? Is he aware that an offense may try plays, or a defense might try a coverage, in the first quarter just to test an opponent? Is he aware that a significant increase in knowledge may be assembled about the play of the game by either the defense or offense or both, by the fourth quarter?

Romer claims to make some adjustments from some of these deficiencies, which means throwing more mathematical assumptions on top of his current assumptions.

Further, Romer makes no adjustment for the quality of the offensive team that will take the field if the first down attempt is not made. My guess is that you test giving Indianapolis Colts' Peyton Manning the ball with only 28 yards to go for a touchdown, even the stats are going to show that it is idiotic to give him even a small chance of that happening.

Bottom line: Romer's study is pretty much useless for fourth down plays (Especially against Peyton Manning). He basically has taken the old joke about economists making assumptions and taken it to a new level of Dali-like absurdity. Yes dear, assume a fourth down and assume a fourth quarter, while your at it. If Belichick was using this study as a reference in making his decision, well then, he got what he deserved, a big fat loss. And Levitt in his post about this continues to do what he does best, jump up and down about half baked data that make for interesting stories, but just like in his book Freakonomics are dangerous if implemented.

UPDATE: Not surprisingly, Greg Mankiw is a big fan of the third down, first quarter, is fourth down, fourth quarter club. He writes:

Chapter 2 of my favorite textbook [his own] has a box on David Romer's work on 4th down strategies in football. One fan of this work is Patriots' coach Bill Belichick, who recently applied Romer's analysis...It did not work out well in this particular case, and Belichick is coming under some heat for his call. This does not mean Romer and Belichick are wrong. Some strategies that fail ex post might be optimal ex ante. Randomness is a fact of life, even if Patriots' fans do not fully appreciate it.

I am beginning to understand how these guys can be Keynesians.

David Warsh with Big Questions and Big Stories

David Warsh has just published the silver anniversary issue of Economic Principals. I have been reading him from the start, when his column was at the Boston Globe. Twenty-five years ago, his column was the first item I turned to, in the big, thick Sunday Globe. The memories come back, as if it were only yesterday. He is not an Austrian, but he is a damned honest, sincere, interesting and informative writer. We definitely need more people like him (or at least hope, he carries on for another 25 years!)

His silver anniversary issues comes out with rockets blaring and asks these important questions:

How deep has been the opposition between the Federal Reserve Board and the US Treasury Department these last fifteen months? Fed chairman Ben Bernanke and Treasury Secretary Henry Paulson have presented a generally united front. But what goes on behind the scenes? What of their staffs? The sheer opacity of Paulson’s initial plan to buy and hold troubled securities, and the clumsiness with which it was presented, has yet to be explained. What was the process by which it was developed and internally reviewed?

Warsh is on to something here. In early Congressional testimony, Bernanke's view of how the mortgage bail out would proceed was decidedly different from Paulson's. Bernanke testified that the bailout would result in mortgages being bought at "value at maturity". Paulson said they would be bought at discounted market value. The next day in further testimony, Bernanke fell in line with Paulson. Paulson's plan proved a non-starter. Indeed, WSJ reported that it is a dirty little secret that Paulson's Plan to buy up mortgages would not work and indeed would cause more problems for banks, and that is why Treasury shifted to infusing capital directly into banks. And, then, of course, there is the fact that Goldman Sachs and Morgan Stanley become bank holding companies, and each receive a $10 billion infusion from Treasury. What Just Happened? is the title to Warsh's column, yes indeed.


Warsh also broke wide open Harvard University’s Russia scandal of the 1990s.


Warsh writes:

No column I ever wrote cost more than “The Thing’s a Mess,” the first installment, in 2002, of many columns over the last six years about the collapse in 1997 amid charges of corruption of Harvard University’s USAID-sponsored mission to advise the government of Boris Yeltsin. I knew I was damaging several longstanding relationships with economists whom I admired by calling attention to the details of the US Justice Department’s ultimately successful attempt to recover damages in Boston’s Federal District Court.

Since then I have gotten used to it, and in more than twenty pieces, I have given a pretty good account of how Harvard professor Andrei Shleifer was found to be investing in Russia, along with his wife, deputy, and deputy’s family, in violation of his contractual obligation to provide disinterested advice, and how his close friend and mentor Lawrence Summers sought unsuccessfully to distance himself from the lawsuit, but not from Shleifer, first as Treasury Secretary and then as president of Harvard, as the matter plowed on to its ignominious conclusion. The episode was widely covered in Russia, and became part of the rich lore of Russian resentment
of US policy in the aftermath of the Cold War.

Warsh also writes of the reception he received from the usual suspects about his breakthrough story:

But you would never have a clue that any of this [the Russia episode] had happened from three of the most widely-read economists’ blogs, the Freakonomics site, J. Bradford Delong’s Semi-Daily Journal, or N. Gregory Mankiw’s blog. Why? Because they are economists, and not committed to “without fear or favor” news, though they deliver plenty of interesting tidbits over the course of a week. Besides, Shleifer is on the board of directors of the Becker Center on Price, where Freakonomics’ Steven Levitt teaches. DeLong, who worked under Summers at the Treasury Department, has been Shleifer’s friend since the two were college roommates. Mankiw regularly touts his colleague for a Nobel Prize.

The online edition is free, but the $50 Bulldog edition puts bread, not likely steak, on Warsh's table, and is available by subscripton.

The Second Highest Paid Position In The NFL

Steven Levitt at Freakonomics has some interesting facts:

I just recently got around to reading Michael Lewis’s immensely entertaining book The Blind Side...

...there is brief mention in the book that left tackle is the second highest-paid position after quarterback, it is a popular book and thus light on formal statistics.

Curious about the numbers, I put one of my loyal assistants Trevor Gallen on the problem, and here is what he reports back after crunching the numbers for every player in the starting lineup for the first game of the 2007-8 season:

1) As Michael Lewis argued, starting left tackles are indeed paid more on average than any other position on the field except for quarterbacks. The average starting quarterback makes about $5 million a year. The average starting left tackle gets $4 million. Defensive linemen and wide receivers also do pretty well.
here is brief mention in the book that left tackle is the second highest-paid position after quarterback, it is a popular book and thus light on formal statistics.

2) There aren’t enough left-handed quarterbacks to do a rigorous analysis, but the blind-side theory is supported by the sparse data that exist on lefty quarterbacks. On those teams, right tackles tend to get paid much more than left tackles. Overall, the median blind-side tackles get paid over twice as much as the median non-blind-side tackles.

3) Punters and kickers make the least money — then safeties.

4) Perhaps the biggest surprise to me is how little running backs make. They are down toward the bottom of the list; I would guess that the wear and tear they face tends to shorten their careers, so they have fewer years over which to earn.

-Robert Wenzel

If You Are Ever On A Jury Trial Remember These Stats On DNA Matches

Steven Levitt posts a fascinating column:

Are the F.B.I.’s Probabilities About DNA Matches Crazy?

On Cheap Wine: The Best Post Steven Levitt Has Ever Made

At his Freakonomics blog, Steven Levitt explains a small test at Harvard he conducted regarding the difference in cheap wines versus expensive wines. Interesting results. In addition to the insight on wines, and I don't want to give away the ending, but I think it also punches a bit of a hole into the usefulness of polling and questionnaires in empirical studies.

Levitt's story and results are here.

Freakonomics Author on Lying Statistics

Steven D. Levitt, co-author of the best selling book, Freakonomics, pulled some doozy statistical magic tricks to reach some of the conclusions in his book.

Turns out, though, he doesn’t trust statistics either, when it comes to his health:

I never trust statistics I get from people in the field of medicine, ever.

We anxiously await Dr. Levitt’s paper explaining why statistics don’t work in medicine, but work in economics, given that there are a lot more variables to contend with in economics, much more difficulty in measuring and observing in economics and, the piece de resistance, there are no constants in economics–--making a lot of economic equations look rather silly. (Levitt’s included).

Freakonomics Author On Lying Statistics

Steven D. Levitt, co-author of the best selling book, Freakonomics, pulled some doozy statistical magic tricks to reach some of the conclusions in his book.

Turns out, though, he doesn't trust statistics either, when it comes to his health:

I never trust statistics I get from people in the field of medicine, ever.

We anxiously await Dr. Levitt's paper explaining why statistics don't work in medicine, but work in economics, given that there are a lot more variables to contend with in economics, much more difficulty in measuring and observing in economics and, the piece de resistance, there are no constants in economics--making a lot of economic equations look rather silly. (Levitt's included.)

Inside The Mind Of Steven D. Levitt : A Review of Freakonomics

Steven D. Levitt (with Stephen J. Dubner) has a hot new bestseller Freakonomics: A rogue economist explores the hidden side of everything.

In the introduction, Levitt makes abundantly clear that his book has no central theme. I can almost agree with this assessment. The book is indeed much more a blog type compendium of different topics, rather than an exposition on one theme. But I did find one theme that runs through out the book. Levitt poses interesting questions, reports interesting facts and occasionally makes clever arguments, but these questions, facts and arguments are surrounded by misleading statements, hazy statements, inaccuracies, poor logic, sloppiness and outright errors.

These flaws run from the minor to the grand scale. Indeed,one must begin by considering that Levitt clearly believes that through out the book he is "doing economics." In fact, although he does tend to include some type of cost benefit analysis in most chapters, his analysis tends to be much more that of a sociologist than that of an economist. Consider the titles of some of his chapters: "What Makes a Perfect Parent?", "Where Have All the Criminals Gone?" and "What Do School Teachers and Sumo Wrestlers have in Common?"

Further although there is an implication by Levitt that he is writing theory, he is in fact more of a historian reporting on past data.(For the important distinctions between theory and history, see Ludwig von Mises' Theory and History.)So instead of a book of economic theory, we have a flawed book on sociological history.

On a minor scale, Levitt tends to use misleading chapter titles. His chapter "What do School Teachers and Sumo Wrestlers have in Common?" comes up with the answer: some in both groups cheat to get ahead. Since there are some in almost any group that will cheat to get ahead, there is nothing distinctive about this supposed "link"

It is the same as saying "What does Steven Levitt and the members of the offensive front line of the New York Giants football team have in common?" Answer: They all use cell phones. The facts in both cases are true but they result in no new insight, but the questions themselves tend to mislead one into thinking that there is some type of distinctive link in the answers when there is not. At most the chapter title is a sloppy effort at being cute.

Levitt goes from bad to worse in the title of his next chapter: "How is the Ku Klux Klan like a Group of Real Estate Agents?" His answer: they both use privileged information to their advantage. Again, nothing remarkable about this chapter, since everyone uses privileged information to their advantage (Indeed that is pretty close to the definition of an entrepreneur!)

Levitt's presentation is so sloppy that it almost fails to get across the point that privileged information is used by the Ku Klux Klan and real estate agents. In short, it is a pretty bad example used to get in a little, quite interesting, history about the Ku Klux Klan.

As for sloppy and hazy arguments, in one chapter, Levitt relates the story of how a pre-school attempted to solve a problem of children being left late after school. The pre-school instituted a fine for parents who left their children late. With the new stated policy (It was only a $3.00 fine), more parents left their children late.

Levitt in the next chapter on page 45 calls this cheating: "So if...day-care parents...cheat are we to assume that mankind is innately and universally corrupt?"

Given all the cheating going on in the world, it is quite odd that Levitt uses this as one of the examples, which only by the wildest stretch could be called "cheating." In fact, it really is a story about the limits of knowledge, and how people will change their actions when more knowledge becomes available.

In Levitt's next chapter sloppiness is coupled with an implied wrong conclusion. Levitt does the math and shows that on a per hour basis "The per hour death rate of driving versus flying...is about equal." He then concludes "The two contraptions are equally likely... to lead to death." He ends his analysis of flying versus driving with this conclusion, which tends to imply that it doesn't matter whether you fly or drive. But, in fact, using Levitt's own data the clear conclusion is the exact opposite of what Levitt implies. The clear conclusion is to fly whenever possible. Why? Because you get there faster, which means you are traveling a shorter amount of time at the per hour death rate. If it takes five hours to fly from New York to Los Angeles and three days or 72 hours to drive, then if the death rate per hour is the same, the risk by driving is close to 15 times greater. If you do nothing else after reading his book other than take Levitt's implied conclusion on driving versus flying and drive instead of fly, Levitt has increased your chances of dying when traveling by nearly 15 times!

Levitt's chapter on names continues the trend of sloppiness, haziness, illogic and poor conclusions.


He tells us that "...it isn't famous people who drive the name game." He uses the fact that no parents are naming their daughters Madonna as part of his argument that this is proof that parents don't name their children after famous people, but this is just sloppy logic. Just because parents don't name their children after Madonna doesn't mean many parents aren't naming their children after famous people. In fact only a page away from where Levitt tells us that famous people don't drive the name game, Levitt lists the most popular black names in California in the year 2000. Number 4 was Michael and Number 2 was Jordan. Hmmm, it seems to coincide with a period when there was a pretty famous basketball player on the court, named Michael Jordan.

In short, I could literally write a book (maybe many books) detailing the errors, sloppiness, inaccuracies, haziness and poor theory going on in this book. And I haven't even touched on the errors in his chapter on Roe v.Wade. (Levitt is probably most well known for his theory that abortions reduce crime, see Steve Sailer for a critique of this theory.) Nor have I discussed his love affair with regression analysis, which is a questionable method to prove theory in the social sciences (See Leoni and Frola)

In a sense though, this book is a great mystery book. The great mystery being just what marketing plan was implemented to drive this disaster onto the best seller list?