Forecasting The Future is Sporty Business, So Knowledge, Skill and Experience is Needed

One of the posts on Tweeter's #NoEstimates is a presentation of bad forecasts. This is one of the analogies like many analogies that is interesting but fatally flawed.

First, forecasting political, technical, economic outcomes is not the same as forecasting the completion date or cost of a software development effort. The former are usually based on chaotic systems. If the latter are based on chaotic system, quit the project, run away, and go find a better project. 

So let's see what the facts are behind each of the pages.

1. Fooling Around with Alternating Current is just a waste of time - Edison and Tesla were in a piched battle using J. P. Morgan's money to capture the eletriclitghting market. This is a marketing pitch, not a suggestion that AC was wrong. In fact, when Morgan dropped funding and forces Westinghouse to absorb the technology, AC took off. The Current Wars provides some background, with lots of references.
2. The coming of the wirless era will make war impossible, because it will make war ridiculous - Marconi would not be considered a leading authority on statesmenship or world affairs. The fact that he belived this shows the naviety of his understanding of the politics of the era.
3. There is no liklihood man can ever tap the power of the atom - 
4. You will be before the leaves have fallen from the trees - start with some history - As German troops crossed the Belgian frontier on 4 August 1914, most people in Europe believed that the "boys will be home by Christmas." If they meant Christmas 1918, they were right. But of course, no one believed the war could possibly drag on so long. Previously, various authors had opined that, due to the massive expense of modern war, any future European hostilities would be short. Many people believed that assessment, but they forgot about one important thing: credit. No, there wasn't enough gold in the world to pay for a long war with modern weapons; there was, however, enough credit to pay for nearly anything.  This was a disruptive event, with underlying chaotic processes. Forecasting in the presence of chaos is usually disappointing. 
5. A rocket will never be able to leave the eath's atmosphere - The New York Times writer got a F in the physics class. Robert Goddard of the Goddard Space Flight Center, Greenbelt, MD, where we worked on the Hubble Robotic Service Mission rendevous and dock software described how to fly to the moon, in the late 1800's Be careful of the source of the forecast. That source may actually be incompetent.
6. It will be years - not in my time - before a woman becomes Prime Minister - will she was in the right place at the right time. Another disruptive event enabled by chaotic processes.
7. We can close the books in infectious diseases - the Surgeon General was asleep in the microbiology class. This was a political statement not a scientific statement. 
8. Rail travel at high speeds is not possible, because passengers, unable to breath, would die of asphyxia - Dr. Dionysys Larder would have gotten a D in the physics class. Like some stating you can't forecast the future would get a D in the statistics class.
9. Democracy will be dead in 1950 - anyone forecasting political outcomes is a fool at best. Nate Silver has showed us how to do this in modern times. His Signal and the Noise should be mandatory reading for any #NE self-proclaimed pundit on how to use probabilistic forecasting so read about the math behind for forecasting.
10. Stock Prices have reached what looks like a permanently high plateau - anyone believing that stock prices can be forecast as individual trading instruments needs to get connected with reality. First buy the index, second never chase the tape. Black Swans are disruptive events, so always hedge your bets. But at the same time Big Data and time series analysis is the basis of all portfolio trading strategies. But the quote is naive at best looking back at 1929, when statistical forecasting of the market was nowhere to be found. John Maynard Keynes A Treatise on Probability was the introduction to this discipline. So again for those conjecturing that the future cannot be forecast, go buy Keynes book, read it, and come back with the counter arguments that are mathematically sound to refute his thesis.
11. I see no good reasons why the views given in this volume should shock the religious sensibilities of anyone - well Darwin wasn't connected with the reality of the times was he?
12. The Beatles have no future in show business - this is why there are more than one agents and record companies. This has nothing to do with the forecasting of the future, just personal opinion. Uninformed in this instance, like many opinions on #NoEstimates, without any factual backup or external references.
13. Remote shopping, while entirely feasible, will flop - Time magazine is probably not the best authority on what will or won't happen with technology in 1968. Circa 1968 was the IBM 360 with batch processing. CICS had not yet been developed and the telecommunications links were SDLC. If you read James Martin'sThe Wired Society, circa 1978, you'll see how that uninformed opinion was replaced but those actually capable of forecasting the future. The technology was not present, so no wonder they - writers at Time - would think that. Yet another example of not actually a mathematical forecast, but opinion of one source.
14. Ours has been the first [expedition], and doubtless be the last, to visit this profitless place - so much for environmental awareness. When the Powell Expedition arrived 8 years later the opinion was much different.

So what's the point?

• Forecasting the future is sporty business. Not for the uninformed, inexperienced, or unskilled.
• You need to be qualified, not opinionated. If you have an opion, bring some facts along with it. It makes the conversation go a lot better if we can talk about facts not just about you.
• You need to be able to separate the sources of uncertainty, before starting the forecasting and estimating process. The categories of uncertainty start with two major divisions:
  • Reducible - epistemic - systematic uncertainty, which is due to things we could in principle know but don't in practice.
  • Irreducible - aleatory - statistical uncertainty, which is unknowns that differ each time we run the same experiment.
  • If you can't tell the difference, go find out the different before forming your opinion.

So back to the #NoEstimates discussion of estimating and forecasting future outcomes. Start here. read the books and papers. Take pause and rethink the blanket, unqualified statements like you can't forcast the future.

Related articles

Deterministic versus Stochastic Trends in Earned Value Management Data