In 1964, the man revered as the godfather of analysis at the CIA, Sherman Kent, wrote an essay, “Words of Estimative Probability.” In it, Kent derided analysts, whom he called “Poets,” who were unwilling to commit to the supposed rigor of quantitative estimates. Their opposite numbers, of course, were “Quants.” Quants were analysts unafraid to adopt different orders of probability for 11 specific words or phrases. For example: a 93% likelihood of an event (give or take about 6%) would translate into it being “almost certain”; in contrast, an event being “probable,” represented a 75% likelihood of occurrence (give or take about 12%). As we documented in our book Constructing Cassandra, with such efforts Kent anchored a quest for pseudo-scientific false precision that hobbles the CIA to this day, making it poorly armed to make sense of a world made up in large part of what we have called elsewhere “non-spreadsheet risks” and fuzzy (but nevertheless potent) ideas. Recall that bin Ladin sent a declaration of war to the United States in 1996, and that this declaration was in the form of... a poem addressed to then US Secretary of Defense William Perry. In the Muslim world and in many other cultures, poetic verse can carry weighty messages.
Why should 21st century finance care about such historical details? Because in many respects, Kent’s quest to quantify the unquantifiable is spreading like cancer in business, economics and public policy. Numbers are increasingly demanded as the only possible clothing predictive forecasts can wear, and are routinely employed to add a patina of neutrality or science to subjective predictions and ideas. That’s how Harvey Schwartz, the CFO of Goldman Sachs, could be taken seriously when he stated that the Swiss National Bank's decision last January to unpeg the franc - a political decision - was a "20-plus standard deviation move". Either Schwartz’s quantified model of the world simply couldn’t cope with reality, or he felt that this sounded smarter than saying, “Three guys met in Zurich and made a decision we sure didn’t forecast.” Alternatively, maybe Schwartz was just a Quant trying to be poetic?
Three Practical Reasons To Insist On Forecasts Without Numbers
But let’s be specific: why exactly should you sometimes insist on qualitative, i.e. non-numerical forecasts about the future? We can think of three practical reasons.
First of all, qualitative forecasting grants partial immunity to certain forms of self-deception. In other words, through their uncertainty, words may actually lend more, not less, accuracy when forecasts address situations in which the nature of your knowledge of the future is uncertain. Insisting on clear prose analysis rather than hard numbers keeps you from sliding down the slope that Paul Romer, an economist at NYU, recently named ‘Mathiness,’ i.e. economics masquerading as science through math. Avoiding the “mathiness trap” will thereby make you more likely to examine the logic and assumptions underpinning the drivers and constraints that are driving the situation. In the face of genuine uncertainty, it is the quality of your questions not the precision of your answers that count, and prose keeps you focused on that central fact. After all, as Sherman Kent was pondering his probabilities, the Soviets were quietly installing nuclear missiles in Cuba, an operation that took the CIA completely by surprise and that brought the world to the brink of a nuclear war.
The second reason it is often prudent to banish numbers from forecasts flows from the first: purely qualitative forecasts encourage debate the character of possible events and the circumstances in which they might occur. In contrast, quantitative forecasts more naturally lead to debates about probabilities even in situations in which probabilities are simply unknowable. Instead of asking, for example, “What is the probability of a Chinese hard landing?”, it might be better to discuss what people mean when they say “Hard landing”, and what a slowdown in Chinese GDP would mean in terms of exports, currency adjustment, and internal market development. For all the quantitative data they generate, even financial markets represent values that are ultimately social, not natural, facts.
Let’s say you were trying to think through how the VW emissions scandal affects Europeans’ preference for diesel engines. Stick with words, and you will more easily concentrate on what matters: rigorously framing logical arguments about the nature of risks or opportunities that might arise from a particular set of possible events. Later you may want to generate numbers, but numbers should not be where you begin – especially when the future is extremely open-ended. The easiest person to “Lie to with Statistics” in other words, can be you, so don’t start with them, start with the rigorous logical framing of definitions and ideas.
The third reason to favor prose over numbers in forecasts is that in our experience doing so helps encourage debate about possible consequences. In their absence, the results of our actions, our inactions or the countermeasures that a competitor might take in the future more easily become part of the discussion. Complex situations are rarely characterized by one event, the probability of which we should be concerned with, but by chains of events, and it is on the logic of this possible cascading of actions events that we should base strategic reasoning.
Go back to the VW scandal, for instance: one might expect consumers to trust car manufacturers less, which could enable independent pollution testing agencies to emerge and to undermine preferences for diesel, enabling electric car to get a boost. One can disagree with that chain of events, but the debate will focus quite properly on the logic of the cascade, not on the false precision of unknowable probabilities.
In ending this ode to words, it is perhaps fitting to end with a poem that captures our core message:
Roses are red
Violets are blue
Quants need Poets
And so do You.
