Mark Zandi's Meaningless Growth Forecast

where orders emerge

by Russ Roberts on January 10, 2011

in Scientism

There was some confusion about my recent post on what economics is good for. I think it’s good for a lot of things. When I said it isn’t like physics I agree that that isn’t the end of the world. In one sense, who cares if it isn’t like physics? The problem is that too many economists and others treat it as if it were like physics. And most of these applications are silly, non-scientific and meaningless. Today’s Washington Post provides a good example:

According to Zandi’s analysis, the payroll tax cut will boost economic growth by 0.5 percent for the year, with most of the growth occurring in the first half. Low- and middle- income households are expected to benefit the most, as the payroll tax is only assessed on the first $106,800 in earnings. The tax cut is likely to create 1 million new jobs, he said.

That sounds scientific. A forecast of growth (using a decimal point!) and an estimate softened by the word “likely.” But it is a meaningless statement. It is scientism, the use of the language and tools of science to reach a conclusion that is not merited. The cut in the payroll tax could actually lead to more concerns about the deficit, discourage investment and risk-taking, and reduce employment in the future. Which view is right? This pessimistic view or Zandi’s? My argument is that there is no way to know. This is an abuse of economics. Here is a more accurate statement that Zandi might have made:

My model of the macroeconomy which has proved to be very inaccurate in the past, predicts a million jobs will be created. I am confident that I have made enough corrections to the model to insure that this forecast is accurate even though there is no way after the fact to assess the accuracy of my forecast.

I am sure Zandi’s forecast of job creation has some confidence interval around it. That is why he adds the work “likely.” That changes nothing. In a way it makes it worse–more scientific-sounding lingo like “confidence interval” when in fact there is no more science and no confidence either.

View Comments    Share var addthis_options = 'facebook, twitter, digg, buzz, delicious, reddit, stumbleupon, friendfeed, google, linkedin, yahoobkm, technorati, wordpress, blogger, typepad, more'; var addthis_exclude = 'email, print';     Print     Email

http://www.facebook.com/BoscoH Brad Hutchings

It would be more accurate if he used the word “ensure”, unless he’s taking out a policy with AIG to protect the economy from any errors in his prediction.

Anonymous

I seem to have the opposite response of a lot of people you’re hearing back from. I liked your long post precisely because of how it stepped back from earlier posts, but now this sounds like your earlier posts!

How would you provide estimates of impacts? It seems like everything economists already do to frame their estimates is inadequate “scientism” to you.

How would you propose to do it or do you just think economists shouldn’t estimate these things and stick to qualitative assessments?

And if they should just stick to qualitative assesments, doesn’t that still introduce the same problem? You’ll still say “I think it would have a negative effect for this reason” and he’ll still say “I think it would have a positive effect for this reason”.

Krishnan

I agree that no one can make accurate predictions and to appear to do so, is misleading at best. The problem as always is the difficulty of trying to convey the idea that, cutting taxes will lead to economic activity that will result in job growth (under some conditions). What would help is for the commentators to remind readers as to how what they have predicted MAY NOT come about – For example, Zandi could have said “Our models predict that a cut in the tax rate will result in the creation of 1 million jobs. The model we used for this prediction has successfully predicted (95% of the time) the impact of tax rate changes on economic activity. For some recent predictions, please see what we said on January, August and October of 2010 and how the predictions stacked up against the data” (But yea, I know – Zandi will not remind us of how good or bad the predictions were – like some climate researchers who cannot predict weather behavior a few days out, but are sure, ABSOLUTELY sure what WILL happen 50 or 100 years hence.

http://twitter.com/cygnegris Grey Swan

Ultimately, the difference between economics and physics is that economics is fundamentally concerned with predicting human behavior. And that is a notoriously difficult thing to do.

DG Lesvic

From The Joy of Paying Taxes

Taxes are painful by definition, for, if they were really joyful, you wouldn't have to force them on anyone, and they wouldn't be taxes, just rent. Rent is joyful, for, like any voluntary payment, you expect to gain by it. It is precisely because you expect to lose by a payment that it must be forced on you, is a tax rather than a price, painful rather than joyful, and must retard incentive and growth.

DG Lesvic

Daniel,

Quantitative assessments are the essence of economic or business history and forecasting, but simply irrelevant to economics proper. So the question is not whether “we” should make them or not, but who should. The historian or forecaster must, the economist, as such, must not.

http://pulse.yahoo.com/_GJWAVRVJ6LLWBJ5QO4DI4EHZ5Y Kyle8

I think it is possible to make some predictions. The way you have to state it is, “In similar circumstances when taxes were raised from X to Y, then this happened, therefore it will probably happen again. ”

You are not making a definite nor quantitative statement, so it is of limited usefulness. However, it is just incorrect to say that you can make no predictions of the future based upon the past.

Darren

A large part is yes, human beings cannot be predicted with the same accuracy as we can, electrons. But I think that’s lack of understanding on our part. Our brain is just a very complex biochemical machine and given enough scientific knowledge, we can have a solid understanding of how it works based on first principles. The bigger issue in my opinion is the fact that economics is not falsifiable. You cannot create controlled and robust experiments in economics. So given a set of predictions, reasonable people can disagree on whether it came true.

I do think however that econometric predictions are useful because these models are backtested and rigorous, even if they have large confidence intervals (which gets far less coverage than the topline number even though it’s just as important). But you can’t expect newspapers to report on the statistical significance of projections. This is hardly a problem with economics per se, but with trying to communicate complex topics to laymen. You get horrible “science” articles too. Ultimately, even though econometrics should be taken with a healthy dose of scepticism, I’d rather have people making decisions based on these than on things like value judgements which are just as likely, if not moreso, to be wrong.

Anonymous

Russ, I have appreciated your series on this topic, especially the last 2 posts. The discussion of the first few had devolved into a semantic debate over the meaning of “science.” I think you’ve effectively cleared that up and now the discussion is more on the point you’re making.

I think Khrishnan and Kyle* make good points above – I’d like to hear your thoughts on them.

DG Lesvic

It’s beginning to look like not even a week can go by without having to repost this.

The Chicago School of Superstition

Milton Friedman's empiricism is like that of ballplayers not changing their socks so long as they keep getting hits. While, to Friedman, the test of a theory is how it works out in practice, to the man he read out of the science, Ludwig von Mises, such tests can never be conclusive, for there is always the question of whether concurring events are cause and effect or coincidence.

"The question whether there is any connection between them can only be answered by" a theory "established beforehand on the ground of aprioristic reasoning"¦If there were no economic theory"¦economic facts would be nothing more than"¦unconnected data open to any arbitrary interpretation."

Theories of human action "are, like"¦logic and mathematics, a priori"¦not subject to verification or falsification on the ground of experience and facts"¦both logically and temporally antecedent to any comprehension of historical facts"¦a necessary requirement of any intellectual grasp of historical events. Without them we should not be able to see in the course of events anything else than kaleidoscopic change and chaotic muddle."

"There is no means of studying the complex phenomena of action other than first to abstract from change altogether, then to introduce an isolated factor provoking change, and ultimately to analyze its effects under the assumption that other things remain equal."

"Action and reason are congeneric and homogenous"¦two different aspects of the same thing. That reason has the power to make clear through pure ratiocination the essential features of action is a consequence of the fact that action is an offshoot of reason"¦Logical thinking and real life are not two separate orbits. Logic is for man the only means to master the problems of reality. What is contradictory in theory is no less contradictory in reality."

Since they are all complex, "Every historical experience is open to various interpretations and is in fact interpreted in different ways"¦History can neither prove nor disprove any general statement."

That is not to rule out empiricism altogether. For the basic premises of econ-omic theory, such as the disutility of labor and variety of resources, are derived from observation. But the theory itself is antecedent to all other historical facts.

Dg Lesvic

What all of this really comes down to is whether you’ve read Mises’ Human Action or not. There’s just no substitute for it. If you’ve read it, you’ll understand these things, and, if you haven’t, you won’t.

Darren

I’m not sure I see the point of posting all that. What you said sounds perfectly reasonable, which is why I said that any predictions should be taken with scepticism. And I also believe that people working in econometrics tend to think too highly of their own models. But what’s the alternative? However flawed it is trying to apply mathematical models to economics, isn’t whatever we try always going to be an approximation of the “truth”? Dismissing econometrics altogether is as foolish as thinking it’s the end all and be all of economics, it’s simply another tool.

Krishnan

I agree – about using the past data … Even though creating models using events that have already occurred may have limited abilities to predict the future, one can try – and see how the predictions bear out – and (more importantly) go back and change the models – and try again – Trying to fit models (mathematical OR otherwise) without such models being able to predict is quite useless – So, it explained something – So what? “Fitting” (equations) to data is NOT modeling (no matter how high the correlation coefficients may be to 1.0)

The fact is that policy makers rely on modeling/predictions all the time – “If we raise taxes by 5% we will increase revenue by 5%” or “If we provide unemployment income, we will stimulate the economy by a factor of 1.6″ (or some such) – Ofcourse it is entirely possible that those that make policy are not interested in determining if what they propose will actually happen – All they are interested in is demagoguing and getting elected so they can wield power -

SweetLiberty

Those who predicted the housing bubble did so not because they picked one of a thousand potential bubbles and just got lucky, but rather used their fundamental understanding of economics and realized that the writing was on the wall for the housing collapse. They made accurate forecasts based upon their economic worldview. Those who rejected the idea of a housing bubble made inaccurate forecasts based upon their (flawed) economic worldview. To say no predictions can be made based upon economics is to ignore those who – for sound reasons – identified and vocalized their misgivings concerning a housing bubble.

The goal of an economist shouldn’t be to identify the precise timing or intensity of a future event, but rather evaluate the current inputs against economic assumptions (founded upon their interpretation of past events), and test these assumptions by predicting general trends. If economists cannot make any forecasts whatsoever, then they should never advocate one policy over another – such as free trade over protectionism, higher vs. lower taxes, bigger government vs. smaller, etc.. I agree with Krishnan when he says economists should provide ci

It would be more accurate if he used the word “ensure”, unless he’s taking out a policy with AIG to protect the economy from any errors in his prediction.

I seem to have the opposite response of a lot of people you’re hearing back from. I liked your long post precisely because of how it stepped back from earlier posts, but now this sounds like your earlier posts!

How would you provide estimates of impacts? It seems like everything economists already do to frame their estimates is inadequate “scientism” to you.

How would you propose to do it or do you just think economists shouldn’t estimate these things and stick to qualitative assessments?

And if they should just stick to qualitative assesments, doesn’t that still introduce the same problem? You’ll still say “I think it would have a negative effect for this reason” and he’ll still say “I think it would have a positive effect for this reason”.

I agree that no one can make accurate predictions and to appear to do so, is misleading at best. The problem as always is the difficulty of trying to convey the idea that, cutting taxes will lead to economic activity that will result in job growth (under some conditions). What would help is for the commentators to remind readers as to how what they have predicted MAY NOT come about – For example, Zandi could have said “Our models predict that a cut in the tax rate will result in the creation of 1 million jobs. The model we used for this prediction has successfully predicted (95% of the time) the impact of tax rate changes on economic activity. For some recent predictions, please see what we said on January, August and October of 2010 and how the predictions stacked up against the data” (But yea, I know – Zandi will not remind us of how good or bad the predictions were – like some climate researchers who cannot predict weather behavior a few days out, but are sure, ABSOLUTELY sure what WILL happen 50 or 100 years hence.

Ultimately, the difference between economics and physics is that economics is fundamentally concerned with predicting human behavior. And that is a notoriously difficult thing to do.

From The Joy of Paying Taxes

Taxes are painful by definition, for, if they were really joyful, you wouldn't have to force them on anyone, and they wouldn't be taxes, just rent. Rent is joyful, for, like any voluntary payment, you expect to gain by it. It is precisely because you expect to lose by a payment that it must be forced on you, is a tax rather than a price, painful rather than joyful, and must retard incentive and growth.

Daniel,

Quantitative assessments are the essence of economic or business history and forecasting, but simply irrelevant to economics proper. So the question is not whether “we” should make them or not, but who should. The historian or forecaster must, the economist, as such, must not.

I think it is possible to make some predictions. The way you have to state it is, “In similar circumstances when taxes were raised from X to Y, then this happened, therefore it will probably happen again. ”

You are not making a definite nor quantitative statement, so it is of limited usefulness. However, it is just incorrect to say that you can make no predictions of the future based upon the past.

A large part is yes, human beings cannot be predicted with the same accuracy as we can, electrons. But I think that’s lack of understanding on our part. Our brain is just a very complex biochemical machine and given enough scientific knowledge, we can have a solid understanding of how it works based on first principles. The bigger issue in my opinion is the fact that economics is not falsifiable. You cannot create controlled and robust experiments in economics. So given a set of predictions, reasonable people can disagree on whether it came true.

I do think however that econometric predictions are useful because these models are backtested and rigorous, even if they have large confidence intervals (which gets far less coverage than the topline number even though it’s just as important). But you can’t expect newspapers to report on the statistical significance of projections. This is hardly a problem with economics per se, but with trying to communicate complex topics to laymen. You get horrible “science” articles too. Ultimately, even though econometrics should be taken with a healthy dose of scepticism, I’d rather have people making decisions based on these than on things like value judgements which are just as likely, if not moreso, to be wrong.

Russ, I have appreciated your series on this topic, especially the last 2 posts. The discussion of the first few had devolved into a semantic debate over the meaning of “science.” I think you’ve effectively cleared that up and now the discussion is more on the point you’re making.

I think Khrishnan and Kyle* make good points above – I’d like to hear your thoughts on them.

It’s beginning to look like not even a week can go by without having to repost this.

The Chicago School of Superstition

Milton Friedman's empiricism is like that of ballplayers not changing their socks so long as they keep getting hits. While, to Friedman, the test of a theory is how it works out in practice, to the man he read out of the science, Ludwig von Mises, such tests can never be conclusive, for there is always the question of whether concurring events are cause and effect or coincidence.

"The question whether there is any connection between them can only be answered by" a theory "established beforehand on the ground of aprioristic reasoning"¦If there were no economic theory"¦economic facts would be nothing more than"¦unconnected data open to any arbitrary interpretation."

Theories of human action "are, like"¦logic and mathematics, a priori"¦not subject to verification or falsification on the ground of experience and facts"¦both logically and temporally antecedent to any comprehension of historical facts"¦a necessary requirement of any intellectual grasp of historical events. Without them we should not be able to see in the course of events anything else than kaleidoscopic change and chaotic muddle."

"There is no means of studying the complex phenomena of action other than first to abstract from change altogether, then to introduce an isolated factor provoking change, and ultimately to analyze its effects under the assumption that other things remain equal."

"Action and reason are congeneric and homogenous"¦two different aspects of the same thing. That reason has the power to make clear through pure ratiocination the essential features of action is a consequence of the fact that action is an offshoot of reason"¦Logical thinking and real life are not two separate orbits. Logic is for man the only means to master the problems of reality. What is contradictory in theory is no less contradictory in reality."

Since they are all complex, "Every historical experience is open to various interpretations and is in fact interpreted in different ways"¦History can neither prove nor disprove any general statement."

That is not to rule out empiricism altogether. For the basic premises of econ-omic theory, such as the disutility of labor and variety of resources, are derived from observation. But the theory itself is antecedent to all other historical facts.

What all of this really comes down to is whether you’ve read Mises’ Human Action or not. There’s just no substitute for it. If you’ve read it, you’ll understand these things, and, if you haven’t, you won’t.

I’m not sure I see the point of posting all that. What you said sounds perfectly reasonable, which is why I said that any predictions should be taken with scepticism. And I also believe that people working in econometrics tend to think too highly of their own models. But what’s the alternative? However flawed it is trying to apply mathematical models to economics, isn’t whatever we try always going to be an approximation of the “truth”? Dismissing econometrics altogether is as foolish as thinking it’s the end all and be all of economics, it’s simply another tool.

I agree – about using the past data … Even though creating models using events that have already occurred may have limited abilities to predict the future, one can try – and see how the predictions bear out – and (more importantly) go back and change the models – and try again – Trying to fit models (mathematical OR otherwise) without such models being able to predict is quite useless – So, it explained something – So what? “Fitting” (equations) to data is NOT modeling (no matter how high the correlation coefficients may be to 1.0)

The fact is that policy makers rely on modeling/predictions all the time – “If we raise taxes by 5% we will increase revenue by 5%” or “If we provide unemployment income, we will stimulate the economy by a factor of 1.6″ (or some such) – Ofcourse it is entirely possible that those that make policy are not interested in determining if what they propose will actually happen – All they are interested in is demagoguing and getting elected so they can wield power -

Those who predicted the housing bubble did so not because they picked one of a thousand potential bubbles and just got lucky, but rather used their fundamental understanding of economics and realized that the writing was on the wall for the housing collapse. They made accurate forecasts based upon their economic worldview. Those who rejected the idea of a housing bubble made inaccurate forecasts based upon their (flawed) economic worldview. To say no predictions can be made based upon economics is to ignore those who – for sound reasons – identified and vocalized their misgivings concerning a housing bubble.

The goal of an economist shouldn’t be to identify the precise timing or intensity of a future event, but rather evaluate the current inputs against economic assumptions (founded upon their interpretation of past events), and test these assumptions by predicting general trends. If economists cannot make any forecasts whatsoever, then they should never advocate one policy over another – such as free trade over protectionism, higher vs. lower taxes, bigger government vs. smaller, etc.. I agree with Krishnan when he says economists should provide circumstances which could invalidate or significantly affect their forecast. But this is not to say that successful predictions – given the current general trends – cannot be made. They have been – and they will continue to be made by economists who put themselves on record. Economists who never test their assumptions are like football players who analyze previous games and go to practice every day, but refuse to leave the bench when it’s game time.

It looks like Zandi didn’t take the changes in U.S. tax withholding into account in estimating the amount of “stimulus”.

The only people seeing an extra 2% more on their paychecks will be those with annual incomes below $2100 (if filing single with no deductions), or those with annual incomes between $85,000 and $115,000. There are people whose paychecks in 2011 will be less than last year (for single filers again, those with incomes between $2,650 and $19,000), while everyone else will see a small increase, but less than 2% of their annual income. The so-called “stimulus” just isn’t as big as he thinks it is….

The stimulus isn’t a stimulus, it’s a depressant.

First, let me say that I am biased in that I am sympathetic toward the Austrian view of the business cycle. So I tend to agree with those same people you mention who “predicted the housing bubble”.

But I think you are getting dangerously close to confusing correlation with causation. Just because the predictions made by those people were “based upon their economic worldview” and turned out to be fairly accurate predictions, does not necessarily mean their worldview is correct.

I don’t think Russ is saying that economists should never make predictions. He is saying that economists shouldn’t mislead others into thinking those predictions are scientific. He is saying that economists have a responsibility to recognize themselves, as well as inform the public, that their predictions about the economy are no more scientifically valid than a sportscasters’ predictions about who will win the world series.

Darren,

A few excerpts from A Declaration of Economics.

Since we live in a world of scarcity, we must all economize. But that doesn't make us all economists. Economics implies something beyond Home Economics, and Ma Kettle in her kitchen. It implies Political Economy, the problems of nations and social classes in the market, and not its passing data, but eternal truths.

Read Full Article »


Comment
Show comments Hide Comments


Related Articles

Market Overview
Search Stock Quotes