There are several fundamental principles of writing software for money. Let's start with the prime directive.
It's not you money, behave appropriately.
When asked to develop software in exchange for money - this is legally called work for hire - we have an obligation to have some notion of the final cost. If not, we're likely working a staff augmentation and not doing work for hire. So let's assume we are on a WFH engagement. Knowing something about the final cost starts with estimating that cost. This estimate says its name - it's an estimate. It's not a firm price. It's not even a firm anything. It's an estimate.
This estimate can come in several forms:
- This customer has a budget and would like to know what can be produced for that budget. This is common in governance based domains. The owner Hubble Space Telescope had a budget to fix HST robotically. He got that budget from the NASA administrator. He released a BAA (Broad Area Announcement) for solicitations of what could be done for his budget. Some for a CIO of a health insurance company. She got a rough budget for upgrading the provider network ERP system and asked vendors to tell her what could be provided for that budget.
- The customer has a set of needed capabilities and needs to know the approximate cost of providing those capabilities and needs to know the cost to provide them on a needed date.
The answer to both of these approaches comes from making estimates. Estimates of cost and schedule. Estimates of capacity for work. The process for making these estimates is called Basis of Estimate and usually starts with an anchoring nbsp;process.
By anchoring it means making the estimate from something that can be used as a reference. It's been done before, there is a model of what could be done. Some reference class. This anchoring process is then followed by an adjustment process. Adjustments can come in a short time, or over a longer time. But the anchoring and adjustment paradigm is well developed.
One research study has shown:
One way to make judgments under uncertainty is to anchor on information that comes to mind and adjust until a plausible estimate is reached. This anchoring-and-adjustment heuristic is assumed to underlie many intuitive judgments, and insufficient adjustment is commonly invoked to explain judgmental biases. However, despite extensive research on anchoring effects, evidence for adjustment-based anchoring biases has only recently been provided, and the causes of insufficient adjustment remain unclear.
Anchoring and adjustment is well studied in behavioral finance - why people make financial decisions that they do. Anchoring and adjustment is also well studied in estimating starting with Oil & Gas estimates of reserves to estimating public works projects. A complete literature search can be found from Google of course.
But estimates needed for making business and technical decisions must take this research into consideration is they are to be successful. For cost and schedule estimates the best place to start is past performance. Here's an orginal drawing of Flybjerg's Reference Class Forecasting process flow. Google will find you all his work as well. Many about infrastructure and some about IT.
When it is mentioned we haven't done this before so we have no reference classes, we have to pause and think. Is this a trueGreen Field project. Technology that has actually never beed done before is rare in the IT world. If this project is truly without precedent, it's likely an R&D project and need to be planned much differently.
So assuming this is not an R&D project, what can we do about creating estimates when we have little past performance? There are many tools available, some free, to start the estimating process. To produce an anchor estimate, we can start to refine before the project starts, or even after the project starts.
There are three types of activities that participate in the estimating process
- Stable activities are understood and predictable. Estimating these is straightforward.
- Dependent Activities where the time or effort is dependent on project attributes or characteristics not yet known but they are knowable.
- Uncertain Activities where little data is available to support the estimate with any confidence.
- Analogous Estimating using the experience from previous projects and extrapolating that onto the current project.
- Parametric Model Estimating is an accurate and easy estimating technique developed for estimating the time or resources needed to perform a project activity.
- Expert Judgment when there is an expert on the project.
- Published Data Estimating for those activities for which there is published data. The activity is compared to the activities for which data exists and the actual cost or durations of the closest comparable activity is selected from the data and used as the estimate.
- Reserve Analysis is fundamental for all estimating and considers the level of uncertainty and risk in the project and establishes a reserve pool of time, resources, or possibly performance that can be drawn upon to offset the unestimated issues that arise.
- Bottom Up estimates improves the accuracy of the overall project estimate requiring the project team to decompose the work into very small work packages.
- Total project budget is many time set even before the scope and schedule are defined. (construction projects funded by bank loans.) A high-level allocation of the budget is created between the likely project deliverables. Each major activity is estimated and if this estimate is greater than the allocated cost, the timing of resources or scope and deliverables are varied until the project is able to meet the budget goals. This is usually an iterative process.
So when we hear estimating is the smell of dysfunction and no dysfunctions are listed let alone any credible and in use estimating processes, it's time to question the wisdom is assuming estimates are the problems with software projects.
So let's invert the conversation for a moment.
- Drip funding is suggested alternative to estimating. But drip funding just pushes the project estimate back to the project owner, who in turn releases budget. But the capacity for work the productivity of that work capacity still needs to be estimated to determine if there is sufficient budget to complete the needed capabilities on the needed date.
- Fixed Price funding move the estimating process to the providers of the product. They now need to estimate their capacity for work, risks, margin, and productivity.
- Decomposing work into same sized chunks - assuming this is possible - may be useful in some domain. But needs to be tested outside of the anecdotes of those using it.
- In the end the Return on Investment calculation [ROI=(Value - Cost / Cost)] used by all viable businesses needs to know both cost and value. These are both probabilistic, and the underlying statistical processes driving both these needs to be understood.
In the end it's the business that needs the estimates of the developers have decided they are not useful. It's not the developers money - if it is no one cares how they spend it. The business has a obligation to the shareholders, investors, the funding agency, to have some understanding of what the cost for the products or services will be when they are done.
