Stories triumph Statistics

My sun sign is Scorpio. One of the astrology website gave me the following prediction.


I wanted to find out if this prediction has any truth in it. I did a simple thought experiment. The world has 7.2 billion people and there are 12 sun signs. Let us assume that the sun signs are evenly distributed among all the human beings. This means there are 600 million scorpios (7.2 billion / 12). Of the 600 million, this prediction will be true for some of them by random chance. I broke down the prediction into several parts and for each part I assumed it to be true for 50% of the scorpio population. The result is shown below. As you can see, the prediction will come true for 18.75 million people just by random chance. This is just 3.125% (18.75 / 600) of the total scorpio population.

scorpiothoughexperimentFrom this simple thought experiment I concluded that the prediction is a baloney. Then why do so many of us believe in these predictions when they are not true? Some day this prediction will come true to each on of us as we might fall into the 18.75 million bucket. Remember this is a random chance event and it can happen to anyone. When this happens to us or to those we know, the specific incidence will make us believe that horoscope is true.

In the book The Success Equation: Untangling Skill and Luck – Mauboussin writes

I had a conversation with a doctor that illustrates these three types of information. He mentioned that he had a treatment for improving a specific ailment that succeeded about 50 percent of the time (the base rate). But he added that he could induce almost any patient to undergo the treatment if he simply told them, “The last patient who was treated this way is doing great!” (specific evidence about an individual case). For the patients who were evaluating the treatment, the story of success swamped the statistics..

Due to evolutionary reasons our brains do not think in terms of statistics. It looks for specific evidence. Stories triumph statistics. No wonder why astrology is a multi-billion dollar industry.

Good to Great is one of the best selling business book with sales exceeding over four million copies. In this book, Collins and his team analyzed thousands of companies and isolated eleven whose performance went from good to great. They then identified the concepts that they believed had caused those companies to improve – these include leadership, people, a fact-based approach, focus, discipline, and the use of technology – and suggested other companies adopt the same concepts to achieve the same sort of results. There is one problem. It does not work. From the book The Success Equation: Untangling Skill and Luck

No one questions that Collins has good intentions. He really is trying to figure out how to help executives. And if causality were clear, this approach would work. The trouble is that the performance of a company always depends on both skill and luck, which means that a given strategy will succeed only part of the time. So attributing success to any strategy may be wrong simply because you’re sampling only the winners. The more important question is: How many of the companies that tried that strategy actually succeeded?

Once again the statistics of how many companies failed by following the strategy is ignored and only the success stories are considered. Stories triumph statistics.

Our inability to understand statistics is exploited by stock market pundits. Imagine that you meet a stock market expert, who claims that he can predict the direction of NASDAQ index. Here are his predictions to you for six consecutive weeks. If all of his predictions came out to be true would you pay for receiving his seventh prediction.

Week 1: Index will rise
Week 2: Index will fall
Week 3: Index will fall
Week 4: Index will fall
Week 5: Index will rise
Week 6: Index will rise

If you pay then you are a patsy. It is a scam. Here is the explanation of this scam from the book Innumeracy

Some would-be adviser puts a logo on some fancy stationery and sends out 32,000 letters to potential investors in a stock index. The letters tell of his company’s elaborate computer model, his financial expertise and inside contacts. In 16,000 of these letters he predicts the index will rise, and in the other 16,000 he predicts a decline. No matter whether the index rises or falls, a follow-up letter is sent, but only to the 16,000 people who initially received a correct “prediction.” To 8,000 of them, a rise is predicted for the next week; to the other 8,000, a decline. Whatever happens now, 8,000 people will have received two correct predictions. Again, to these 8,000 people only, letters are sent concerning the index’s performance the following week: 4,000 predicting a rise; 4,000, a decline. Whatever the outcome, 4,000 people have now received three straight correct predictions. This is iterated a few more times, until 500 people have received six straight correct “predictions.”

These 500 people are now reminded of this and told that in order to continue to receive this valuable information for the seventh week they must each contribute $500. If they all pay, that’s $250,000 for our adviser. If this is done knowingly and with intent to defraud, this is an illegal con game. Yet it’s considered acceptable if it’s done unknowingly by earnest but ignorant publishers of stock newsletters, or by practitioners of quack medicine, or by television evangelists. There’s always enough random success to justify almost anything to someone who wants to believe.

Once again stories triumphed statistics.

One thought on “Stories triumph Statistics

Comments are closed.