When I hear a post like:
If you use deterministic estimates you must ask the team. If you use probabilistic estimates you must not. #NoEstimates
Two things come to mind:
- All project work is probabilistic. There is no such thing as a deterministic estimate. OK, there is. But those estimates a wrong, dead wrong, willfully ignorant wrong. All project work is probabilistic. If you're making deterministic estimates, you've chosen to ignore the basic processes of probability and statistics.
- There is an important difference between Statistics and Probability. Both are needed when making decisions in the presence of uncertainty.
All projects have uncertainty.
And there are two kinds of uncertainty on all projects. Reducible and Irreducible.
Reducible uncertainty (on the right) is described by the probability of some outcome. There is an 82% probability that we'll be complete on or before the second week in November, 2016. Irreducible uncertainty (on the left) is described by the Probability Distribution Function (PDF) for the underlying statistical processes.
In both cases estimating is required. There is no deterministic way to produce an assessment of an outcome in the presence of uncertainty without making estimates. This is simple math. In the presence of uncertainty, the project variables are random variables, not deterministic variables. If there is no uncertainty, not need to estimate, just measure.
Empiricism
When we hear that #NoEstimates is about empirical data used to forecast the future, let’s look deeper into the term and the processes of empiricism.
First, an empiricist rejects the logical necessity for scientific principles and bases processes on observations. [1]
While managing other people’s money in the production of value in exchange for that money, there are principles by which that activity is guided. For empiricist principles are not immediately evident. But principles are called principles because they are indemonstrable and cannot be deduced form other premises nor be proved by any formal procedure. They are accepted they have been observed to be true in many instances and to be false in none.
Second, with empirical data comes two critical assumptions that must be tested before that data has any value in decision making.
- The variances in the sampled data is sufficiently narrow to allow sufficient confidence in forecasting the future. A ±45% variance is of little use. Next is the killer problem.
- With an acceptable variance, the assumption that the future is like the past must be confirmed. If this is not the case, that acceptably sampled data with the acceptable variance is not representative of the future behavior of the project.
Understanding this basis of empiricism is critical to understanding the notion of making predictions in the presence of uncertainty about the future.
Next let’s address the issue of what is an estimate. It seems obvious to all working in engineering, science, and financial domain that an estimate is a numeric value or range of values for some measure that may occur at sometime in the future.Making up definitions for estimate or selecting definition outside of engineering, science, and finance is disingenuous. There is no need to redefine anything.
Estimation consists of finding appropriate values (the estimate) for the parameters of the system of concern in such a way that some criterion is optimized. [2]
The estimate has several elements:
- The quantity for the estimate – a numeric value we seek to learn about.
- The range of possible values for that quantity
- For estimates that have a range of values, the probability distribution of the values in the range of values. The Probability distribution function for the estimated values. The range of values is described by the PDF, with a Most Likely, Median, Mode, and other cummulants – that is what’s the variance of the variance?
- For an estimates that has a probability of occurrence, the single numeric value for that probability and the confidence on that value. There is an 80% confidence of completing the project on or before the second week in November, 2005
Now when those wanting to redefine what an estimate is to support their quest to have No Estimates, like redefining forecasting as Not Estimating, it becomes clear they are not using any terms found in engineering, science, mathematics, or finance. When they suggest there are many definitions of an estimate and don’t provide any definition, with the appropriate references to that definition, it’s the same approach as saying we’re exploring for better ways to …. It’s a simple and simple minded approach to a well established discipline and making decisions and fundamentally disingenuous. And should not be tolerated.
The purpose of a cost estimate is determined by its intended use, and its intended use determines its scope and detail.
Cost estimates have two general purposes:
- To help managers evaluate affordability and performance against plans, as well as the selection of alternative systems and solutions,
- To support the budget process by providing estimates of the funding required to efficiently execute a program.
- The notion of defining the budget leaves open the other two random variables of all project work – productivity and performance of the produced product or service.
- So suggesting that estimating is no needed when the budget of provided, ignores these two are variables.
Specific applications for estimates include providing data for trade studies, independent reviews, and baseline changes. Regardless of why the cost estimate is being developed, it is important that the project’s purpose link to the missions, goals, and strategic objectives and connect the statistical and probabilistic aspects of the project to the assessment of progress to plan and the production of value in exchange for the cost to produce that value.
The Need to Estimate
The picture below, with apologies for Scott Adams, is typical of the No Estimates advocates who contend estimates are evil and need to be stopped. Estimates can’t be done. Not estimating results in a ten-fold increase in project productivity or some vague unit of measure.
[1] Dictionary of Scientific Biography, ed. Charles Coulston Gillespie, Scribner, 1073, Volume 2, pp. 604-5
[2] Forecasting Methods and Applications, Third Edition, Spyros Makridakis, Steven C. Whellwright, and Rob J. Hayndman
Some More Background
- Introduction to Probability Models, 4th Edition, Sheldon M. Ross
- Random Data: Analysis and Measurement Procedures, Julius S. Bendat and Allan G. Piersol
- Advanced Theory of Statistics, Volume 1: Distribution Theory, Sir Maurice Kendall and Alan Stuart
- Estimating Software Intensive Systems: Projects, Productsm and Processes, Richard D. Stutzke
- Probability Methods for Cost Uncertainty Analysis: A Systems Engineering Perspective, Paul R. Garvey
- Software Metrics: A Rigorous and Practical Approach, Third Edition, Norman Fenton and James Bieman
