When we hear you can't forecast the future or we can't estimate the future without knowing about the past, we need to take a pause and consider what is being said and how to respond.
The first response should be what do you mean by "estimate?" This sounds like a simple question but in fact without a shared understanding of the estimating process, it's going to be a bumpy road.
Let's start with a quote - expert judgement is usually the crux of cost and schedule estimates, but in the spectrum of risk management processes (this is what estimating is all about - reducing the risk), quantification of expert judgement is the weakest area. Fred Raymond in Quantifying Risk to Manage Cost and Schedule, Acquisition Quarterly Review, Spring 1999.
This is not a new problem. The literature is full of estimating failures provided by experts. It is easy to criticize how estimating has failed to meet the need of business and government to know what something will cost and when it will be done. This need is not a false need, it is how business and government works. To say otherwise, ignores the fundamental premise of a business - income must exceed cost to remain in business. Decision making is founded on microeconomics of Opportunity Cost. Both these require estimates to be made in the presence of uncertainty. To ignore this framework is to willfully ignore how business works.
The picture above is from a blog post on estimating the hiking distance along the coast of California. The map distance for a car is not the same as the foot path. It is much more on foot, if you follow trails, then if you drive on Highway 1 - Pacific Coast Highway. What this analogy says is we must know the domain, the context of that domain, the units of measure (the metric of the measurement), the known variances, the knowable variances, and possibly the unknowable variable variances.
These last variances are the Black Swans in some domains. Finance being one. Projects, not so much. If you're spending other peoples money building something, you'd better have some understanding of the unknowns and a plan to deal with them, otherwise you're going to get your project canceled and your reputation handed to you on a platter.
So Let's Looks At The Spectrum of Estimating
When we say we need an estimate when and where can we push back and when and where had we better be able to produce the estimate with a credible confidence.
- We don't need estimates when the work is already planned out - when there is a list of work with a statistically similar size sitting in a queue. And the service performing the work can do so with statistically constant throughput, estimating the work up front is a waste. This is called Kanban in the manufacturing world. We've assembled raw materials - a story in software development - assured the raw material has a statistically small variance, so the server can process this raw material into a finished product in approximately the same duration. This is can be modeled by Little's Law, which says Little's Law says that under steady state conditions, the average number of objects in a queuing system equals the average number of objects leaving the queuing system multiplied by the average time each object spends in the system. No problem, it's actual a waste, no need to estimate.
- But of course someone, somewhere has to decompose the work and assure it is approximately the same size,
- And someone, somewhere has to install the server that processes the work and assure it can handle the capacity demands of the arriving work,
- And someone, somewhere has to assure that the arrival rate of the work does not exceed the departure rate from the server, otherwise the queue of work will start to grow and those waiting for work in the queue will experience a delay from the planned arrival of the finished products,
- This approach can be found in nearly domain in manufacturing, from making products to filling cans.
- We need a quick estimate so we can assure ourselves this project is feasible. You can approximate the cost of launching a satellite to Low Earth Orbit satellite without any special sensor by knowing the weight of the vehicle. It ranges from $1,700/lb on a Proton (Russian) to $5,500/lb on a Lockheed Atlas 5. With this quick estimate we can determine if our little idea is feasible or not.
- In the Enterprise IT world we can do similar things. How much will it cost to replace 30 legacy systems with SAP in 2 years? Good question. We can play 20 questions. >$500M - no way. < $5M - no way. How about >$200M - that seems high. <$20M - that seems low. You get the idea.
- We have to be careful here not to fall into the trap of believing our project has never been done before. Reference Class Forecasting can help. Reference Class Forecasting is used in many domains and can be applied in this quick estimate example.
- Next we need an estimate to provide to the customer so they can determine if we're a viable supplier of a solution. This is part of any good sales process. Convincing the customer through a proposal or even a face-to-face call that we can provide a solution has to include the question what is your budget. That most often is responded with what do you think this will cost? At least any good buyer will ask that for a fundamentally simple reason - the value and anything depends on its cost.
- So to answer that question what will this cost we need to somehow come up with a basis of estimate. We need to construct a proposal of the approximate cost to some confidence level of the solution
- This can be a SWAG cost or a confidence level cost. You soluton you're looking for will cost $275,000 or less with an 80% confidence.
- How did this cost come about, well you built an estimate bottom up from past performance, some parametric model - 10 databased table cost 2 weeks to build, 50 database tables - all things being the same - will cost 10 weeks to build, plus a buffer of 1 week for 11 weeks. Something like that.
- There are many tools in the software business for doing these types of estimates. Just Google "software cost estimating" tools and look at their processes, white papers and case studies.
- Which by the way will show you that this is done all the time. The notion that you can't estimate software systems is simply bogus. If you want to you can and you can do it with acceptable levels of confidence.
- Which by the way, if you don't want to, nothing is going to convince you otherwise, just move on.
- Now comes the hard part. You need an estimate of the total cost so someone, somewhere can go get a budget. This is common in government or governance based IT. Staying on budget has many dimensions.
- Funding must be established before authorization to proceed can be obtained.
- Board approval is needed
- Departments need to approve the budget
- Congress needs to approve the budget
- These types of estimates ae built bottom up, usually parametrically
- But the critical success factor for this type of estimating is the update the Estimate To Complete and the Estimate At Completion periodically. In our domain, this is done monthly. And the ETC / EAC have confidence levels on them as well.
So In The End
- Anyone can learn to estimate. But you have to want to do it.
- If you've convinced yourself or been convinced by someone else that estimates are of no value, you may want to check with the people who gave you money in exchange for a product or a service to see if they share your view of the value of estimates.
- When you see Scott Adam's cartons of the pointy haired boss doing stupid things on purpose arounf estimating, just remember it's a carton. If you work in such an environment, I'd start looking for more fulfilling work. I've been there. It pollutes your mind.
- All business, at least successful business, is based on managing cost. Don't manage cost, you won't be around for long.
- Managing cost means knowing what the future costs well be. You wouldn't manage your household without some forecast of what expenses are coming due. Mortgage, car payment, unexpected repairs. There recurring costs and nonrecurring costs. You have some sense of the costs in the future. If you aren't doing this, some time soon, you'll wish you had.
- When you're spending other peoples money, you really shold consider your obligation to inform them what the expected cost will be. If you have a customer that doesn't want to know, I want to work there to.
And the Final Close
When you hear about someones dedication to improving Return on Investment (ROI), think carefully about the formula for ROI.
Cost is in the denominator and numerator.
No estimate of cost, no way to calculate ROI. It's bogus to say we're always focused on ROI, but we don't want to estimate the cost. That's a laughable statement.
