One of the suggestions in the #NoEstimates community is that you can't forecast the future. They confuse estimating with forecasting at times, but estimating a variable is a single point forecast. Forecasting can be both single point or time series forecasting - that is a series of value in the future generating by the forecasting engine.
There are links below to some of the original work on time series forecasting. But before that, the notion of a time series is very powerful for project management work. And developing software for money is project management work. The concept that the past is a indicator of the future is at the heart of all estimating and forecasting. If you believe you can't forecast the future using the past and needed adjustments - this is Bayesian statistics - then stop reading here. And BTW that Bayesian statistical forecasting is baked into all you do, consume, interact with in daily life. From the prescription drugs, to air traffic control, to the emissions controller on your car.
Poor Mr. Tzu
Had Mr. Tzu been able to read these works, it is unlikley he would have stated...
Those with knowledge don't predict and those who predict don't have knowledge
Instead, he would have likely understood that in a non-chaotic system, future behaviour is derived from past behaviour. And if we observe the past behaviour, we can construct a productive model of the future, once we have categorized the drivers of that behaviour. The is the Box Jenkins contribution.
If however the underlying system is chaotic, forecasting the future is sporty at best. But if we working in a chaotic environment, while spending other peoples money, we're doomed to fail from the start. This is the basis - weak false basis - of Taleb's Black Swans. If they are applied to writing software for money, you're in trouble from day one.
This post is not going show you how to do this. Mainly because a Blog post is too small by several orders of magnitude to address the principles and practices of statistical forecasting. Instead here is a list of books, some free in PDF, that show how to do forecasting and most importantly how to do it using tools.
- First Course in Statistical Programming with R, W. John Braua and Duncan Murdoch
- Introduction to Time Series and Forecasting, 2nd Edition, Peter Brockwell and Richard Davis
- A First Course on Time Series Analysis, Using SAS, Chair of Statistics, University of Wurzburg
- An Introduction to Applied Multivariate Analysis with R, Brian Everitt and TorstenHothon
- New Introduction to Multiple Time Series Analysis, HelmutLutkepohl, Springer, 2005
and the final one all those failing you can forecast the future...
- Introduction to Time Series and Forecasting, 2nd Edition, Peter J. Brockwell and Richard A. Davis, Springe Texts in Statistics, 2002
Here's some of the original papers on time series forecasting as well.
- Distribution of Residual Autocorrelations in Autoregressive Integrated Moving Average Time Series Models, G. E. P. BOX and David A. Pierce, Journal of the American Statistical Association, December 1970, Volume 65, Number 332, Theory and Methods Section
- On a measure of lack of fit in time series models, G. M. Ljung, College of Business Administration, University of Denver, Colorado and G. E. P. Box
- Science and Statistics, George E. P. Box, Journal of the American Statistical Association, December 1976, Volume 71, Number 356, pp. 761-799
- Forecasting by Extrapolation: Conclusions from Twenty-five Years of Research, J. Scott Armstrong University of Pennsylvania. 11-1-1984
Here's the Point - Again
When someone makes a statement like you can't forecast the future or you can't estimate without looking into the future, you'll now know - after reading at least one of these books or papers - that the statement is false in principle and most likely false in practice.
In principle it's simple. Forecasting the future is done every single day in a wide variety of domains, including software development. In practice, there are some assumptions that need to be addressed, but those are addressed in the books. The primary one has to do with disruptive events in the future. Taleb has made a living speaking about Black Swans and they can't be seen coming and therefore we can't forecast the future.
In practice, we need to keep track of our performance, as Vasco describes. With this time series of past performance, we can start to make some statement about the future. These statements are probabilistic in nature, with statistical bounds - confidence intervals.
This Black Swan analogy has one problem first. Australia has Black Swans everywhere. We don't here, neither did England when that phrase was coined. That aside, the next problem is with software projects, any Black Swans come from not looking for actual problems. Or if the project is truley chaotic, run away as fast as you can, you're going to fail no matter how hard you try or how many #NoEstimates you don't produce.
