The Expert Predicted It

In August 1990, Alan Greenspan told his colleagues at the Federal Reserve that those arguing the economy was already in a recession were "reasonably certain to be wrong."¹ The recession had begun in July. One month earlier.

In March 2007, Ben Bernanke testified before Congress that the problems in the subprime mortgage market seemed "likely to be contained."² The Great Recession began eight months later. It wiped out $11 trillion in household wealth, pushed unemployment past ten percent, and produced the deepest contraction since the 1930s.

In October 2007, three weeks before the recession officially started, the New York Federal Reserve staff forecast 2.6 percent GDP growth for 2008. They assessed roughly a 90 percent probability that no recession would occur.³

The Federal Reserve controls the most powerful lever in the American economy. It sets the interest rate. It regulates the banks. It employs hundreds of PhD economists and has more real-time economic data than any institution on earth. It could not predict what would happen to the economy it manages. The economists were not careless. Prediction, at the scale and complexity of a national economy, does not work. The evidence for this is one of the most thoroughly documented findings in the social sciences.

The study

In 1984, a political psychologist at the University of Pennsylvania named Philip Tetlock began collecting predictions from experts. Not casual opinions. Formal, structured predictions with defined outcomes and time horizons. He asked political scientists, economists, national security analysts, and government advisors to assign probabilities to specific questions. Will the Soviet Union collapse? Will GDP grow above three percent? Will this leader hold power? Three possible outcomes per question. Better, same, or worse. The expert assigns a probability to each.⁴

He did this for twenty years. Between 1984 and 2003, 284 experts generated roughly 28,000 predictions across political, economic, and geopolitical domains. The experts averaged twelve years of experience in their fields. Most held doctorates. Their political philosophies ranged from Marxist to libertarian. They made their living commenting on or advising about these exact topics.⁵

The benchmark was simple. Three outcomes per question. If you assign equal probability to each, you get 33.3 percent on everything. A dart-throwing chimpanzee. That was Tetlock's phrase. Imagine a chimp throwing darts at a board labeled with the three outcomes. That is the baseline against which the experts were measured.⁶

The experts, on average, barely outperformed the chimp. They also usually lost to simple algorithms that did nothing more sophisticated than predict the status quo would continue. The algorithm that said "things will stay roughly the same" beat the people who studied these questions for a living.⁷

That alone would be notable. But Tetlock found something more specific. He divided the experts into two cognitive styles, borrowing from the philosopher Isaiah Berlin, who borrowed from the Greek poet Archilochus. "The fox knows many things. The hedgehog knows one big thing."⁸

Hedgehogs organize the world through a single powerful framework. They are ideological. They extend one big idea into every domain. They project certainty. They tell a compelling story. When the evidence contradicts their framework, they explain it away. Foxes are different. They draw from multiple models. They are skeptical of grand theories. They are comfortable saying "I don't know." They adjust when the evidence changes.

Foxes outperformed hedgehogs on every measure Tetlock tracked. Calibration. Discrimination. Accuracy within their domain and outside it. This was the single most consistent pattern in the entire study. Not political orientation. Not years of experience. Not whether they were predicting within their specialty. The only variable that reliably separated better forecasters from worse ones was cognitive style.⁹

The gap showed up most clearly in how each group responded to being wrong. When confronted with an outcome that contradicted their forecast, foxes updated their beliefs by approximately 59 percent of the amount prescribed by Bayesian probability theory. Hedgehogs updated by 19 percent. Some hedgehogs, when presented with disconfirming evidence, moved their beliefs in the wrong direction. They became more confident in the position the evidence had just undermined.¹⁰

Here is the part that matters for the rest of this piece. Hedgehogs are what the media wants. They speak in clean narratives. They project conviction. They are quotable. Tetlock found that forecasters with the biggest media profiles tended to lose to their lower-profile colleagues. He described a "rather perverse inverse relationship between fame and accuracy."¹¹ The traits that make an expert attractive to a television producer are the exact traits that make them a worse forecaster. The confidence. The single narrative. The bold prediction. The refusal to hedge. Every one of those traits correlated with worse performance. Foxes, as Tetlock put it, "bore you with a cloud of howevers." Nobody books them.

The record

Tetlock's study measured individuals. The institutional record is worse.

In 2018, three researchers at the International Monetary Fund published the definitive analysis of recession forecasting. Zidong An, Joao Tovar Jalles, and Prakash Loungani examined GDP forecasts across 63 countries from 1992 to 2014. During that period, 153 recessions occurred. They asked a simple question: how many were predicted by April of the preceding year?¹²

Five. Out of 153. A 3.3 percent detection rate.

Loungani himself, an economist at the IMF, had studied this earlier. In 2001, examining consensus forecasts from 1989 to 1998, he found that only 2 of 60 recessions were predicted a year in advance. His conclusion: "The record of failure to predict recessions is virtually unblemished."¹³

The 2009 recession was the clearest test. Forty-nine countries entered recession that year. By September 2008, after the credit crunch had been front-page news for over a year, after Northern Rock had been nationalized, after Bear Stearns had collapsed, the consensus forecast predicted zero of them.¹⁴ Not one. The average growth forecast in April before a recession was approximately positive three percent. The average actual growth was approximately negative three percent. A six-percentage-point swing, in the wrong direction, every time.¹²

Private sector forecasts were no better. Loungani found them "virtually identical" to official forecasts. It was, as he put it, "a statistical photo finish." The problem was not that the IMF was uniquely bad. The problem was that recession forecasting itself does not work, regardless of who does it.¹³

The 2016 United States presidential election produced the same pattern in a different domain. Two things "everyone knew." Hillary Clinton would be elected. If somehow Donald Trump won, markets would tank. Trump won. Markets soared. No forecast that took a conventional view of the 2016 election got the period since then correct.¹⁵

Paul Krugman, a Nobel laureate, published a column in The New York Times in July 2022 titled "I Was Wrong About Inflation." He had been on the side less concerned about the inflationary consequences of the American Rescue Plan. He admitted the call was wrong. He described the experience as "a lesson in humility." He did not mention abstaining from future forecasts.¹⁶

Tetlock's study ended in 2003. The record since then has not improved. In January 2020, the IMF projected 3.3 percent global growth for the year. By April, they revised it to negative 3.0 percent. Actual global GDP fell 3.5 percent. A 6.8 percentage point miss. No model predicted the pandemic. No institution predicted the economic contraction it would produce. The largest single-year decline in global output since the Second World War arrived without a forecast.

The following year produced a different kind of failure. Throughout 2021, Federal Reserve Chair Jerome Powell described rising inflation as "transitory." In April, the month the Fed adopted the word, CPI was 4.2 percent. By December it was 7.0 percent. By June 2022 it reached 9.1 percent. In November, Powell retired the word. The Fed did not raise rates until March 2022. By July 2023, the target had gone from near zero to 5.25 percent, the fastest tightening in four decades. The Fed's own projections in December 2021 had called for a rate of approximately 0.9 percent by the end of 2022. The actual rate was 4.5 percent. The institution that sets the interest rate could not predict where it would set the interest rate twelve months later.¹⁷

The earnings report was one visible number nobody checked against the register. The forecast is another.

The hedge fund industry, which employs thousands of macro forecasters managing $4.5 trillion, produced annualized returns of 2.8 percent over ten years. The S&P 500 returned 13.8 percent over the same period. An index fund with no forecasters, no analysts, and no opinions outperformed the entire industry by a factor of five.¹⁸

The arithmetic

The record is bad. But the record alone is not the argument. Forecasts could fail for fixable reasons: bad data, insufficient computing power, not enough expertise. If the problem were technical, better tools would solve it. The problem is mathematical.

Howard Marks, the co-chairman of Oaktree Capital Management and one of the most respected investors alive, laid out the arithmetic in a 2022 memo titled "The Illusion of Knowledge."¹⁹ Most forecasting follows a chain of dependencies. The forecast does not predict one thing. It predicts a sequence.

Being right two-thirds of the time at each individual step, which would be an extraordinary track record, leaves only a 13 percent probability that the final forecast materializes. The chain does not forgive. Every additional dependency multiplies the chance of failure. Five dependencies at 67 percent each produce the same odds as a single coin flip that lands wrong seven times out of eight.

Marks cited the historian Niall Ferguson, who posed a question that appears simple: Has inflation peaked? To answer it, Ferguson wrote, you must predict the supply and demand for 94,000 commodities, manufactures, and services. The future path of interest rates. How long the dollar remains strong. The duration and outcome of the war in Ukraine. Saudi Arabia's response to Western oil demands. China's lockdown policies. The impact of Covid variants on labor markets.²⁰

Each of those is itself a prediction requiring further predictions. And each of those depends on decisions that have not been made by people who have not made them yet. Marks's observation was sharper: predicting inflation is not as impossible as predicting the war in Ukraine. It is more impossible, because it requires being right about the war, the pandemic, and a thousand other things, simultaneously.

The arithmetic guarantees it. The chain of dependencies is a mathematical feature of prediction in complex systems. You cannot fix it with better models, more data, or smarter people. The structure of the problem guarantees the failure of the forecast.

The mechanism

If the arithmetic explains why forecasts fail, the mechanism explains why they cannot improve.

Marks identified three structural problems. The first is complexity. The American economy has 330 million participants. Consumers, workers, producers, investors. Each interacts with suppliers, customers, regulators, and global markets. The number of nodes and interactions runs into the billions. Economic models simplify by assuming consumers behave rationally, producers optimize efficiently, and prices reflect information. Every one of those assumptions is false in specific, measurable ways that change over time.²¹

The second problem is inputs. A forecast requires inputs. Each input is itself a forecast. To predict where the economy will be in twelve months, you need to predict interest rate decisions, fiscal policy, commodity prices, consumer sentiment, geopolitical stability, and technological change. Each of those predictions requires further predictions. The model does not sit on solid ground. It sits on a stack of other models, each of which is guessing.²¹

The third problem is what makes economics permanently different from physics. Richard Feynman, the Nobel laureate in physics, put it this way:

Imagine how much harder physics would be if electrons had feelings. Richard Feynman

Electrons do not rebel, forget, innovate, or change their behavior because they read about what electrons are supposed to do next quarter. Economic participants do all of these things. They foresee, react, and adapt. They read the forecast and change the behavior that the forecast was predicting. A model of an economy is a model of a system that is aware of the model and responds to it.

Sowell observed the same problem from a different angle. People are not chess pieces.²² You cannot move them to a square and expect them to stay. They see the move. They adapt. And their adaptation produces the opposite of the intended outcome. Maryland raised taxes on millionaires expecting $100 million in additional revenue. Two thousand high earners left the state. Revenue dropped $200 million. The forecast assumed static behavior. The targets were not static.²³

Sowell's observation about economic forecasting was specific. "Economists are often asked to predict what the economy is going to do," he wrote. "But economic predictions require predicting what politicians are going to do. And nothing is more unpredictable."²⁴

Then there is the problem of stationarity: the assumption that patterns observed in the past will persist into the future. In most physical systems, this assumption holds. One hundred years of flood data is a reasonable guide for building a levee. In economics, the assumption fails because the relationships between variables change over time.

The Phillips Curve, the most influential relationship in macroeconomics for more than half a century, demonstrated this. For sixty years, the curve held: lower unemployment meant higher wage inflation. It was the bedrock of central bank policy. Unemployment fell below 5.5 percent in March 2015. It reached a fifty-year low of 3.5 percent in September 2019. No significant inflation appeared until 2021. The relationship that generations of economists relied on simply stopped working. The measurement was fine. The economy underneath it had changed in ways no model captured.²⁵

Scott Sagan, the political scientist, offered the shortest summary of the stationarity problem: "Things that have never happened before happen all the time."²⁶

The cost

Thomas Sowell asked three questions of every claim. Compared to what? At what cost? What are the hard facts?²⁷

Applied to forecasting, the answers are precise. Compared to what? Compared to chance. Two hundred and eighty-four experts across twenty years, and the result was barely distinguishable from a random number generator. Compared to simple algorithms. An extrapolation that says "things stay the same" beat most of the people paid to say otherwise. The hard facts? Twenty-eight thousand data points. One hundred and fifty-three recessions. Five predicted. 3.3 percent.

The remaining question is the one that carries the weight: at what cost?

The cost is borne by the people who trust the forecast. The homeowner who bought at the peak because no recession was predicted. The retiree whose portfolio followed the macro call. The policymaker who designed a stimulus, an austerity program, or a budget based on a growth forecast that turned out to be six percentage points wrong. The IMF's own Independent Evaluation Office examined its surveillance in the years before the financial crisis and found "a high degree of groupthink, intellectual capture, a general mindset that a major financial crisis in large advanced economies was unlikely, and inadequate analytical approaches."²⁸ The institution that exists to warn about crises was captured by the belief that crises of that scale could not occur.

John Kenneth Galbraith identified the structural reason the industry persists: "There are two kinds of forecasters: those who don't know, and those who don't know they don't know."²⁹ The person who issues the forecast does not bear the cost of being wrong. They issue another forecast. Their career does not end. Their reputation, in many cases, grows. Tetlock found that the most famous forecasters were the least accurate. Fame did not correct the error. It rewarded it.

This is the same structural finding that operates in the Boeing story. Knowledge in one place. Power in another. Consequences in a third. The forecaster has the platform. The institution has the authority. The person who followed the forecast bears the loss. Three groups. The same separation. The feedback loop that would force correction does not exist because the cost of being wrong lands on someone who had no voice in the prediction.

Sowell's framework for institutional failure predicts this. Residual claimants, people who bear the consequences of their own decisions, self-correct. They must. Their errors cost them something, and the cost produces change. Insulated decision-makers do not self-correct because the feedback never reaches them in a form that forces change.³⁰ The forecasting industry is structurally insulated. Its practitioners are not residual claimants. They are commentators. And commentators, as a class, face no penalty for being wrong. They face a penalty for being boring. Hedgehogs dominate because they are interesting, not because they are right.

The alternative

The constructive version of this argument comes from Marks. In a memo written in November 2001, weeks after the September 11 attacks and in the wreckage of the dot-com collapse, he drew a line between two questions that most people treat as one.³¹

The first question: Where are we going? This is prediction. What will GDP be next year? Where will the market be in December? When will the recession start? Marks argued, and the evidence supports him, that this question cannot be answered reliably by anyone. The chain of dependencies is too long, the inputs are unknowable, the participants adapt, and the math does not work.

The second question: Where are we now? This is assessment. Are we early or late in the economic cycle? Is credit expanding or contracting? Is investor sentiment euphoric or depressed? Are valuations above or below historical norms? These are questions about the present, not the future. They can be answered with observable data. They do not require a chain of predictions that multiplies toward zero.

We may never know where we're going, or when the tide will turn, but we had better have a good idea where we are. Howard Marks, "You Can't Predict. You Can Prepare," November 2001

The distinction is operational. A prediction says: the market will fall 20 percent next year. An assessment says: credit is expanding rapidly, risk appetite is high, valuations are stretched, and these conditions have historically preceded corrections. The prediction requires being right about timing, magnitude, and trigger. The assessment requires being right about where you are in a pattern that has repeated for centuries.

Marks's phrase for the actionable version is cycle awareness. Every market, every economy, every business moves in cycles. The pendulum swings between euphoria and depression, overpriced and underpriced. The midpoint of the arc best describes the average position, but the pendulum spends almost no time there. It is always moving toward an extreme or away from one. You cannot predict when the pendulum will reverse. But you can observe, with reasonable precision, how far from the center it has traveled.³²

The value is precision about which questions can be answered. Expertise, assessment, and diagnosis are real. The doctor who examines a patient and identifies a condition is exercising expertise. The epidemiologist who says a specific disease will kill a specific number of people next year is forecasting. The first has a track record. The second has a 3.3 percent detection rate.

The argument is not against expertise itself. Expertise operates in specific, bounded domains where feedback is immediate and repetitive. The surgeon who has performed a thousand procedures is better than the one who has performed ten. The pilot with twenty thousand hours reads an instrument panel differently than the one with two hundred. The chess master sees patterns the novice cannot. These are real skills developed through real feedback, where the expert acts, the outcome arrives, and the expert adjusts, repeatedly.

Forecasting does not have this structure. The prediction resolves months or years later, under conditions that have changed in ways nobody anticipated. The feedback is delayed, ambiguous, and contaminated by a thousand other variables. The forecaster can always explain why they were right in principle but wrong in timing. Or right in direction but wrong in magnitude. The escape routes are infinite. The feedback loop that would force improvement does not close.

Daniel Boorstin, the former Librarian of Congress, put it in twelve words: "The greatest enemy of knowledge is not ignorance. It is the illusion of knowledge."³³

The expert predicted it. The prediction was the problem.