If you're on a project that has certainty, then you're wasting your time estimating. If you are certain things are going to turn out the way you think they are when you have choices to make about the future, then estimating the impact of that choice is a waste of time.
If your future is clear as day far enough ahead that you can see what's going to happen long before you get there, estimating is a waste. If you live in Fantasyland you really don't need to estimate the impact of decisions made today for outcomes in the future.
Peter Pan and Tinker Bell will look after you and make sure nothing comes as a surprise.
If how ever you live in the real world of projects - then uncertainty is the dominant force driving your project.
Let's say some things about uncertainty. First a tautology
Uncertainties are things we can not be certain about
Uncertainty is created by our incomplete knowledge of the future, present, or past. Uncertainty is not about our ignorance of the future, past, or present. When we say uncertain we speak about some state of knowledge of the system of interest that is not fixed or determined. If we are in fact ignorant of the future, then the only approach to is spend money of find things out before proceeding. Estimating is not needed in this case either. We can only estimate after we have acquired the needed knowledge. This knowledge will be probabilistic of course. But starting a project in the presence of ignorance of the future is itself a waste, unless those paying for the project are also willing to pay to discover things they should have know before starting. At that point it's not really a project - which has bounded time and scope.
So when you hear we're exploring, ask first who's paying for that exploration. Is the exploration part of a plan for the project? Can that cost be counted in the cost of the project and therefore be burdened on the ROI of the project? Maybe finding someone who actually knows about the project domain and can define the uncertainties would be faster, better, and cheaper, than hiring someone who knows little about what they're doing and is going to spend your money finding out.
This is one reason for a Past Performance section in every proposal we submit - tell me (the buyer) you guys actually know WTF you're doing and that you're not learning on my dime.
Back to Uncertainty
Uncertainty is related to three aspects of the project management domain:
- The external world - the activities of the project.
- Our knowledge of this world - the planned and actual behaviours, past, present, and future of the project.
- Our perception of this world - the data and information we receive about these behaviours.
There are many sources of uncertainty, here's a few:
- Lack of precision about the underlying uncertainty.
- Lack of accuracy about the possible values in the uncertainty probability distributions.
- Undiscovered Biases used in defining the range os possible outcomes in the project's processes, technologies, staff, and other participants
- Unknowability of the range of probability distributions.
- Absence of information about the probability distributions.
This project uncertainty creates Risk to the success of the project. Cost, Schedule, and Performance risk. Performance being the ability to deliver the needed capabilities in exchange for cost and schedule. There is a formal relationship between uncertainty and risk.
- Uncertainty is present when probabilities cannot be quantified in a rigorous or valid manner, but can described as intervals within a probability distribution function (PDF).
- Risk is present when the uncertainty of the outcome can be quantified in terms of probabilities or a range of possible values.
- This distinction is important for modeling the future performance of cost, schedule, and technical outcomes of a project.
There are two types of uncertainty on all projects:
- Aleatory - Pertaining to stochastic (non-deterministic) events, the outcome of which is described using probability.
- From the Latin alea.
- For example in a game of chance stochastic variabilities are the natural randomness of the process and are characterized by a probability density function (PDF) for their range and frequency.
- Since these variabilities are natural they are therefore irreducible.
- Epistemic (subjective or probabilistic) uncertainties are event based probabilities, are knowledge-based, and are reducible by further gathering of knowledge.
- Pertaining to the degree of knowledge about models and their parameters.
- From the Greek episteme (knowledge).
Separating these classes helps in design of assessment calculations and in presentation of results for the integrated program risk assessment.
Three Conditions for Aleatory Uncertainty
- An aleatory model contains a single unknown parameter.
- Duration
- Cost
- The prior information for this parameter is homogeneous and is known with certainty.
- Reference Classes
- Past Performance
- Parametric models
- The observed data are homogeneous and are known with certainty.
- A set of information that is made up of similar constituents.
- A homogeneous population is one in which each item is of the same type.
Aleatory Uncertainty can not be reduced - it is Irreducible
Epistemic Uncertainty
Epistemic Uncertainty is any lack of knowledge or information in any phase or activity of the project. This uncertainty and the resulting risks can be reduced, through testing, modeling, past performance assessments, research, comparable systems, and other processes to increase the knowledge needed to reduce the uncertainty in the knowledge of the project outcomes.
Epistemic uncertainty can be further classified into model, phenomenological, and behavioural uncertainty. (in "Risk-informed Decision-making In The Presence Of Epistemic Uncertainty," Didier Dubois, Dominique Guyonnet, International Journal of General Systems 40, 2 (2011) 145-167)
Epistemic Uncertainty can be reduced, it can be bought down by spending time and money
Dealing with Aleatory and Epistemic Uncertainty
- Epistemic uncertainty results from gaps in knowledge. For example, we can be uncertain of an outcome because we have never used a particular technology before. Such uncertainty is essentially a state of mind and hence subjective.
- Aleatory uncertainty results from variability that is intrinsic to the behavior of some systems. For example, we can be confident regarding the long term frequency of throwing sixes but remain uncertain of the outcome of any given throw of a dice. This uncertainty can be objectively determined.
So Now The Punch Line
To manage in the presence of these two uncertainties, reducible and irreducible, we need to know something about what will happen when we make decisions that are mitigating the risks that result from the uncertainties. The actions we need to take to reduce the risk or provide margin for the irreducible risks.
What is the probability that our actions will produce desirable outcomes in the presence of these uncertainties. What is the probabilities that the residual uncertainties, will be sufficiently low, so we will still have sufficient confidence of success - defined in any units of measure you want. Ours or on time, on budget, on specification. You can pick your own, but please write them down in some units of measure meaningful to the decision makers.
What is the probability we can show up on or before a need date with the needed capabilities the customer paid us from, at or below the expected cost of those capabilities, so the customer can accomplish their business goals - that is we can can provide the value in exchange for a cost that meets their business goals?
So Here it is, Wait for It
You can't make decisions in the presence of uncertainty unless you estimate the outcomes of these decisions, influenced by the probabilistic nature of the drivers of the uncertainties.
Let's make it clear - You can't Make Decisions For Uncertain Outcomes Without Estimating those outcomes. This means cost, schedule, performance impacts that result from your decisions. This is the basis of Microeconomics - the Opportunity Cost between choices.
If you hear you can make choices in the presence of uncertainty, ask to see exactly how. They can't show you. Move on, they don't know what they're talking about. ∴
