Esther Derby has a post on estimating. The thread seems to go that estimating is helpful but estimates are not.
Let's start with estimating in a domain where we are spending other peoples money. Better yet an environment where we are spending the money of a sovereign - the United States of America.
But first let's talk about the first principle of estimating anything.
Any single point estimate, without a variance and a confidence interval on that variance is a nonsense number - a number that has no meaning.
So, when someone gives you an estimate, you MUST ask and they MUST answer the question What is the confidence on that estimate? The conversation would go something like this...
What is you're planned completion date for the XYZ feature?
We'll be completing on or before the 3rd week in November of 2014 with a 80% confidence and a ±5% error on that confidence level.
It's the variance that is missing in the discussion Esther is having.
Regarding Other Ideas in the post ...
- People turn estimates into targets - don't let them do that. Estimates are estimates. Targets must have buffers. Those buffers must be developed from the probabilistic assessment of the risk in the estimates.
- People construe estimates as bids - separate those when appropriate. Estimates can certainly be a bid. A defense contractor provides a proposal with a cost volume in the proposal. That is a submittal for the proposed cost. It is a contractual number at submittal, and a definitized number at IBR (Integrated Baseline Review) and a Baseline numbers after "snapping the baseline" at the IBR. Changes to the Baseline happen all the time for all the right reasons.
- Inappropriate precision in estimates implies people know more than they do - this is a common problem when those making the estimates don't know how to make estimates - get people who can make credible estimates or train those you have. In the estimating world there are 5 levels of estimating confidence:
- Low Risk: Historical Data, Vendor Quotes
- Low to Medium Risk: Comparative Analysis
- Medium Risk: Level of Effort (sizing by labor hours)
- Medium to High Risk: Parametric estimates, Cost modeling
- High Risk: Engineering estimates
- People game estimates - don't let them do that. Have an estimating process, follow it, oversee it, vet it, risk adjust it, learn how to do estimates, look to sources that can show you how to do estimates - the Software Engineering Institute is one place, the Software Technology Support Center and their journal Crosstalk is another.
Space flight contractors can guess the cost of a Low Earth Orbit spacecraft within 15% by weighing the vehicle. Strange as it may seem, the developers of flight avionics can guess the cost of a Guidance, Navigation, and Control (GN&C) box to within 10% by counting the lines of code. Yes those dreaded SLOC estimates, that have been calibrated over 3 decades. For those looking to see how software is estimated in domains where software dominates, is mission critical, and developed on tight budgets, and tight schedules, with emerging requirements look to:
Credible software estimates are done all the time, every month for example on large weapons and space systems.
If you want to move toward similar capabilities, become knowledgeable of how this works, adopt what works to your environment, and DO NOT let anyone get away with sloppy estimating. If you do that, you get what you deserve.
