All software development projects operate in the presence of uncertainty.
This uncertainty is unavoidable. One might actually make the case in the Agile paradigm that uncertainty is desirable, otherwise, how can emergence add value to the deliverables.
In the presence of this uncertainty, the design, development, and deployment of software must rely on estimations, forecasts, and predictions based on an idealized understanding of what is an unknown (but knowable) future desired outcome. Guiding the work efforts toward a future is based on a Product Roadmap, no matter the development method - be it Agile or Traditional. Without such a Roadmap the developers and managers of the development are just wandering around looking for a solution for an illdefined problem.
If this future reality is unknowable you've got a much bigger problem and are headed for failure. Emergence plays a role in all development processes, but emergence without a goal is called Research not Development. And Development has specific business goals for breakeven, ROI, IRR, Cash Flow and other financal performance measures neeeded to run the business successfully.
So let's focus on the impacts of uncertainty in the development paradigm, and leave the research alone for now.
There are two broad types of uncertainty on all projects, and on software projects, these two types drive very different responses.
- There is uncertainty associated with the natural randomness of the underlying processes of writing software.
- There is uncertainty associated with the model of the real world the software operates in because of insufficient or imperfect knowledge of reality.
These two types have fancy names
- Aleatory uncertainty.
- Epistemic uncertainty.
The two types of uncertainty may be combined and analyzed as a total uncertainty or treated separately. In either case, the principles of probability and statistics apply equally.
The Alea in Aleatory is Latin for Dice.
This means there is an Inherent Randomness.
This is a data-based uncertainty associated with the inherent variability of the basic information of the real world processes of development. These uncertainties cannot be reduced, they are just part of the process of development. They are irreducible and the only approach to dealing with them is to have margin. Schedule margin, cost margin, performance margin.
By data-based, it means that the randomness is in the data generated by statistical processes. For example, the duration of a work activity is a statistical process. That duration can take on many values depending on the underlying model of the work. We can have a narrow range of values for the duration. Or a wide range of values, depending on the underlying processes.
Many software project phenomena or processes of concern to developers contain randomness. The expected outcomes are unpredictable (to some degree). Such phenomena can be characterized by field or experimental data that contain significant variability that represents the natural randomness of an underlying phenomenon. The observed measurements are different from one experiment (or one observation) to another, even if conducted or measured under identical conditions.
There is a range of measured or observed values in these experimental results; and, within this range, certain values may occur more frequently than others. The variability inherent in this data or information is statistical in nature, and the realization of a specific value (or range of values) involves probability.
This is why measures like velocity are very sporty since past performance is rarely like future performance in the presence of Aleatory Uncertainties (as well as Epistemic Uncertainties) of actual project work.
The term epistêmê in Greek means knowledge.
Epistemic uncertainty reflects our lack of knowledge.
This lack of knowledge is a probabilistic assessment of some outcome, usually an event based outcome.
There is a 40% chance of rain in the forecast area for tomorrow is an Epistemic uncertainty.
We assign probabilities to events, probabilities to the work activities that create the knowledge needed to assess the uncertainty, and probabilities of the residual uncertainties after our new knowledge has been acquired.
In practice, we can assign a mean or a median value to this uncertainty. That's what the weather forecast does. That 40% chance of rain is usually a mean value. Where we live, when we hear a 40% chance in Boulder County, we know we have a lower probability because of our micro-climate. That weather forecast is over the forecast area and may be much different depending on where you live in that area.
This forecast also includes inaccuracies and imprecisions in the prescribed forms of the probability distributions and all the parameters of the estimates. This is why forecasting the weather in some parts of the world is a very sporty business. In places like Los Angeles, it's easy - as shown in the movie LA Stories, where Steve Martin is the bored weatherman. Here in Colorado, with our mountain weather, making a forecast a few days from now is not likely to be very successful. As they say don't like Colorado weather? Wait a few hours, it'll change.
Some Challenges to Managing in the Presence of Uncertainty
The primary issue with all uncertainties is the communication of the accuracy and precision of the risk created by the aleatory and epistemic uncertainty.
- What is the scope of the uncertainty?
- What risks does it create to the success of the software development effort?
- Is the uncertainty time-dependent?
- At what level of decomposition of the project is the uncertainty applicable?
This is a Risk Communication issue. So let's restate the two forms of uncertainty
- Aleatory uncertainty: the uncertainty inherent in a nondeterministic (stochastic, random) phenomenon… is reflected by modeling the phenomenon in terms of a probabilistic model… Aleatory uncertainty cannot be reduced by the accumulation of more data or additional
information. - Epistemic uncertainty: the uncertainty attributable to the incomplete knowledge about a phenomenon that affects our ability to model
it… is reflected in ranges of values for parameters, a range of viable models, the level of model detail, multiple expert interpretations, and statistical confidence.uncertainty uncertainty can be reduced by the accumulation of additional information.
What Does This Mean for Software Development Working in the Presence of Uncertainty?
If you accept that all software development work operates in the presence of Aleatory and Epistemic uncertainty, then ...
No decisions can be made in the presence of these two types of uncertanties without estimating the impact of your decision on the project
This is a simple, clear, concise principle of managing in the presence of uncertainty. Anyone suggesting that decisions can be made without estimating has to willfully ignore this principle, OR the project is de minimus - meaning it's of no consequence to those paying if the project is late, over budget, or the delivered outcomes don't meet the needed performance level for the project to earn its Value in exchange for the Cost to produce that Value.
For Those Interested in the Underling Mathematics Here are Some Gory Details (click to load full sized poster)
