Did President Obama's 2009 stimulus package work? We know how to measure success. As Obama said many times, the purpose of the stimulus package was to create or save jobs. Success or failure, then, is to be determined by the unemployment rate.
But unfortunately, it isn't that simple. It's not enough just to look at the raw totals or changes in the unemployment rate before and after the stimulus. That's because with or without a stimulus package, some number of jobs (positive, negative, or zero) would have been created anyway over the same period of time -- and we can't just subtract those ones out because we don't know the number. It's the problem of the counterfactual -- the basis upon which Obama offers his "it could be worse" excuse.
There is no easy way to know what effect the stimulus package had because we don't have an alternative universe to look at in which it never passed into law.
But there is actually a way to compare universes in which the economy was stimulated more with those stimulated less.
The universes I'm referring to are called states. We have 50 of them, and each received a varied amount of stimulus cash per capita. A few years back, when stimulus money was still being been spent, an assistant director at the San Francisco Federal Reserve wrote a very complex and technical paper in which he tried to compare the various states' levels of job creation based on how much stimulus money had been spent. He concluded that although some jobs were temporarily created early on in the process, they were short-lived, and the net effect of the stimulus was already zero jobs created or saved by August 2010.
I'm not clever enough to follow all of his equations, but I thought it might be worth trying the concept on a layman's level. If the stimulus package was a roaring success, we'd expect to see something indicating that by looking at how relatively un-stimulated states did versus their more stimulated peers. The results need not be dramatic, but it would be reasonable to expect some correlation between stimulus dollars spent in each state per capita and changes in state unemployment rates.
I was reminded of this today when I saw this blog post and scatter chart. It looks only at whether stimulus cash was targeted toward states that had high unemployment rates in February 2009 (it wasn't). But then I began to wonder -- what do you get if you plot the change in each state's unemployment rate against the amount of stimulus money spent in each state per capita?
So I did it, using unemployment data from the Labor Department and the stimulus expenditures as tracked by ProPublica through October 2012. As it turns out, you get something that looks like this, with each dot representing a state:
Note that I excluded D.C. and Alaska, but only because they got so much money per capita that they make the rest of the graph look like a tiny cluster at the bottom. This way, you can see more clearly the random distribution of dots, without any correlation that is obvious to the naked eye. The farther you go to the right, the more unemployment has risen since early 2009 (and to the left, has declined), and the higher up you go, the more stimulus money was spent per person.
I am not a statistician or an economist, and this may be a bit of a crude method, but it seems like a pretty reasonable test (which could be conducted more elegantly by others) of the theory that the stimulus package made a big difference in saving or creating jobs. The trend line on this graph is essentially flat, and in fact tilts ever so slightly the "wrong way." That is to say, on the whole, states that did better with their unemployment rates were just slightly more likely to be states that received less stimulus money per capita.
There could be many explanations for this that elude my very limited knowledge of statistics (for example, perhaps only certain kinds of stimulus spending did any good; or perhaps there's a minimum threshold or a point of diminishing returns for per capita stimulus spending, and we'd see more of a trend if we excluded the 10 states that received the most or the least money). But at least we can say that this chart presents no evidence that the stimulus was much help creating or saving jobs in the states on aggregate, and it's one of the few places one could look to for clear evidence that it did help.