Ester Derby posted a thought provoking topic on estimating and estimates. Ester is a well known and respected speaker, writer, and consultant. But Ester seems to have fallen into a common trap of tossing around ideas about probability, statistics, estimating, and confidence on those numbers without an underlying basis of those ideas.
From Handbook of Cost Risk Analysis Methods, Philip M. Lurie, Mathew Goldberg, and Mitchell Robinson, IDA Paper P-2734, April 1993.
Here's some content that needs to be address...
- People turn estimates into targets. Meeting the target becomes the de facto goal and the de facto method. Meeting needs fades in priority.
- This is a failure mode of the people performing the estimating process. Not estimating. The estimate is a probabistic number.
- The probability states an on or before or an at or below number with a confidence level.
- That confidence level also has an error band.
- We have an 80% confidence of launching to the space station on or before the 3rd week in November 2017 with an error band of -10% and +7%.
- If management fails to understand this, then they have violated the first tenant of estimating - all estimates are probabilistic based in the underlying statistical processes.
- People construe estimates as promises. No one can predict the future, but many people treat estimates as guarantees. Failed predictions fan blame. Trust and openness suffer.
- This is because people got a D in the High School statistics class.
- Anyone can predict the future, the statement above also would get you a D in the High School statistics class. This is one of those nonsense statements by those not familiar with probability and statistics.
- The question is what is the confidence in that production estimate? 50/50 that when I toss a quarter a single time it will be heads.
- People construe estimates as bids. Bidding usually involves some calculation of profit. That implies a margin. However, managers discourage margins in estimates. Managers view “padding” as a moral failing–but it’s really a contingency for the unknown (or compensation for bosses who are known to cut estimates to fit wishes. See below).
- Estimates can certaintly be used as bids. The Estimate to Complete for an incremental contract is found in all construction contracts.
- Bidding may or may not involve profit, this is a contracting issue not an estimating issue.
- Those making the basis of estimate for a software contract are not involved in the profit calculation. That calculation involves, profit, fringe, benefits, indirect costs. These are added to the Direct Costs produced by those estimating labor, materials and duration.
- People need to learn the differences. If they don't, don't blame the estimating process, blame management.
- Inappropriate precision in estimates implies that people know more than they do. When expectations and reality meet, people may feel disappointed. More likely, they feel deceived. Trust and openness suffer.
- The units of measure of inappropriate need to be defined for this statement to have any meaning.
- Calculating the total cost of a project to the nearest penny in the initial Basis of Estimate is of little value. Awarding a contract to the next penny may well be.
- Management Reserve and Contingency are terms used when awarding contracts.
- Accounting considerations are not the same as project management considerations. The Accounting department will be interested in more precision that possibly the project manager - depending on the Domain and Context. Without a domain and context, there is really no way to assess this statement.
- In the accounting business this is called diminimis meaning the differences don't matter.
- People game estimates. How many of you developers out there have thought long and hard about an estimate, only to have a managers say, “That estimate X is too high. The estimate should be X – Y.” Me, too. Fudging estimates to fit wishes sets off another round of deceit and disappointed expectations. Trust and openness suffer.
- If people are gaming estimates the estimating processes are failing. This happens all the time on our large defense and space programs. It's become part of the root cause failure mode.
- But this does not mean estimating is not something to do.
- The anchoring and adjustment processes must be read and understood before proceeding with the gaming discussion. This work goes back to the estimating bias that is built into our brains. But there are specific remedies for these biases and those can be found in all the estimating books and papers.
These are failures of the "people" performing the estimating. Ester's paradigm is people management. And people management is absoutley necesssary for project success. But the people must be using the proper processes, applying the proper principles, and performing the proper practices is there is any hope of improving the probability of success for the project.
It's a poor workman that blames his (her) tools - that platitude is applicable here
So In The End
There are numerous problems in making credible estimates. And lots of ways to make bad estimates, including making No Estimates because you think they are a waste. The estimating process needs to be credible from the start. Then the estimates themselves need to be credible, produced by the credible process. One place to start is Here. Start also with a survey of software estimating processes. A little Google work will find you all you need to see that the approaches described in the post can stand some improvement.
Bad Estimates Come from Bad Management Processes - Fix Management Not Estimating
