Probably the most common and prominent example of ‘publication bias’ is ‘predicting the future’. Seers who make a living from spouting their forecasts and prophesies make no hesitation to issue a press release when one of their predictions has actually come true. But they are notably silent when a prediction does not come true. In fact, unscrupulous scammers in the investment business used to make a methodical practice of this. They would quietly publish two pieces…one that said the stock market would go up and one that said it would go down. After waiting a bit to see what actually happened, they would bury the faulty one and trumpet the correct prediction as a marketing tool to get people to buy their newsletter.
The Boston Globe featured a piece (thanks Mom) by Joe Keohane entitled “That Guy Who Called The Big One…Don’t Listen to Him.” He’s referring to Nouriel Roubini, who predicted the recession and housing bubble burst with a good deal of accuracy. But his article is more about how the biggest prognosticators of big events actually have very bad track records. Even more interesting is the last couple of paragraphs where he quotes Oxford economist Jerker Denrell: “By studying what successful ventures have in common (persistence, for instance), people miss the invaluable lessons contained in the far more common experience of failure. They ignore the high likelihood that a company will flop – the base rate – and wind up wildly overestimating the chances of success.” Keohane continues: “To look at Denrell’s work is to realize the extent to which our judgment can be warped by our bias toward success, even when failure is statistically the default setting for human behavior.”