Martin Fowler has a nice post about estimating software development projects in an agile paradigm.
It starts like this
When asked to produce or handed an estimate, respond with the following:
- What confidence level are you needing for the estimate you're asking us to do?
- What confidence level have you assign to those estimates you handed me?
- What is the shape of the Probability Distribution for those estimates you want me to use?
- No answers to these questions from the High School Statistics class? These are pure guesses and we're going to be late and over budget before we start.
Here's how to fix this
- Look to similar past performance work to see if we can extract classes of work that can be used as the basis of estimating the new work.
- Don't have any past performance, ask if this project is inventing new physics, because that's the only reason it hasn't been done before. Go ask someone, go on the web forums and search around, go down to the cafeteria and ask, go across the street to Star Bucks and ask.
- Better yet, go cobble together a walking skeleton to see if we can build our own reference class.
- DO NOT GUESS, use agile to find out before we proceed.
- Assign a confidence interval to all the estimates so we can determine where the problem children are and record that confidence for later.
- Establish agreement - this is an agile team and full and open communication with the customer and management is part of any functioning agile team - on the credibility of the estimates and their proper use.
- The Straw man of management misuse in an agile paradigm is just that a straw man, don't let it happen, stop whining, stop being Dilbert. Insist on behaving in the agreed manner as members of the agile team, otherwise we are just AINO (Agile in Name Only).
Using estimates during development
Here's how to fix this
- When those statistically acceptable set of estimates are arrived at, make them part of the release plan. And make sure their confidence intervals are publicly visible and can't be ignored - the BIG VISIBLE CHART is one way.
- Color coding can work. Standard Deviations from the Mean can work. Anything can work as long as it is recorded in the plan, agreed to by the team - remember those agreement processes we did when we formed the agile team.
- Revisit those confidence intervals and the Most Likely † value for the estimate every time we release something, start to work on the next release, or do anything that impacts our confidence in the estimates.
Misuse of a statistical estimator is common in communities that don'y understand the nature of project work - all variables on projects are probabilistic.
Here's How to Fix This
- Actuals HAVE to be different than the estimate. That's why it's call an ESTIMATE
- Go back to the High School Statistics book and reread the definition of an estimate. Don't let anyone tell us estimating is guessing. If they do, they didn';t take that High School Stats class
An estimate is an indication of the value of an unknown quantity based on observed data or a model of the possible values that would be observed.
- Adjust those estimates for actual work. Adjust them for anything and everything that impacts the work.
Estimating the furture is not about controlling the future, it's about being prepared to respond to the emerging possibilities of the future
When we fail to recognize that all project work is probabilistic, and we have a need to know about about what might probabilistically happen we will always be responding to changes, rather than anticipating the changes and being prepared to respond.
But always remember, every number of a project, including our confidence that we've got bullet proof code ready to put into production on a moments notice is a probabilistic number. Speaking about any number in the absence of the confidence interval of the number is Guessing.
And as that building sized safety poster at Rocky Flats Environmental Technology Site used to say
Don't Do Stupid Things On Purpose
† Most Likely is NO the average. Never use average or allow anyone to speak to us about the average. The most likely number is the Mode. That's the number that will occur most often in all probabilistically driven work. Learn to think like a developers working in a statistically driven world - that's called the Real World, where uncertainty reigns. Remember the difference between probability and statistics in the picture above. We are subject to both in everything we do.