It is conjectured that uncertainty can be dealt with ordinary means with open conversation, identification of the uncertainties and their handling strategies. That quantitative methods are too elaborate and unnecessary for problems except the most technical and complicated ones.
When asked what is meant by uncertainty the answer many times is probably or very likely. But not any quantitative measure meaningful to the decision makers. Since the future is always uncertain in our project domain, making decisions in the presence of uncertainty is a critical success factor  for all project work.
Decision making is one of the hard things in life. True decision-making occurs not when we already know the outcome, but when we do not know what to do. When we have to balance conflicting values, costs, schedule, needed capabilities, sort through complex situations, and deal with real uncertainty. To make decisions in the presence of this uncertainty we need to know the possible outcomes of our decision, the possible alternatives and their costs - in the short term and in the long term. Making these types of decisions requires we make estimates of all the variables involved in the decision-making process.
What Are Probabilities?
There is a trend in the software development domain to redefine well established terms in mathematics, engineering, and science - it seems to suit the needs of those proffering that in the presence of uncertainty decisions can't be made.
Probabilities represent our state of knowledge. They are a statement of how likely we think an event might occur or the possible of a value being within a range of values.
These probabilities are based in uncertainty, and uncertainty comes in two forms. Aleatory and Epistemic.
- Aleatory uncertainty is the natural randomness in a process. For discrete variables, the randomness is parameterized by the probability of each possible value. For continuous variables, the randomness is parameterized by the probability density function (pdf).
- Epistemic uncertainty is the uncertainty in the model of the process. It is due to limited data and knowledge. The epistemic uncertainty is characterized by alternative models. For discrete random variables, the epistemic uncertainty is modeled by alternative probability distributions. For continuous random variables, the epistemic uncertainty is modeled by alternative probability density functions. In addition, there is epistemic uncertainty in parameters that are not random by have only a single correct (but unknown) value.
Both these uncertainties exist on projects. When making good decisions on projects we know something about these uncertainties and have handling plans for the resulting risk produced by the uncertainties.
- For Aleatory uncertainty (irreducible risk) we need margin. The margin protects the project deliverables from unfavorable cost, schedule, and technical performance that is part of the naturally occurring variances.
- For Epistemic uncertainty (reducible risk) can be addressed by buying down the uncertainty. Paying money to learn more.
This by the way is a primary benefit of Agile Software Development, where forced short term deliverables provide information to reduce risk. Agile is Not a risk management process, many other steps needed for that. But Agile is a means to reveal risk and take corrective action on much shorter time boundaries - reducing the accumulation of risk.
Some Background on Decision Making in the Presence of Uncertainty
One way to distinguish good decisions from bad decisions is to assess the outcomes of those decisions. The measurement critical for a good or bad decision needs some definition itself. There are issues of course. The results of the decision may not appear for some time in the future, but we need to know something about the possible results before we make the decision. As well we'd like to see the results of the alternatives of our decision for the choices that weren't made or rejected.
A fundamental purpose of quantitative decision making is to distinguish between good and bad decisions. And to provide criteria for assessing the goodness of the decision. To do this we need first to establish what the decision is about.
- When do you think we'll be ready to go live with the needed capabilities we're paying you develop?
- If we switch from our legacy systems to an ERP system, how much will we save over the next 5 years with the sunk cost of the entire project?
- On the list of desirable features, which ones can we get on the current need date if we reduce the budget by 15%?
Making decisions like these in the presence of uncertainty by estimating future outcomes is a normal, everyday, business process. Any suggestion these decisions can be made without estimates is utter nonsense.
Decision analysis starts with defining what a decision is - the commitment to resources that is irrevocable only at some cost. If there is not cost associated with making the decsion or changing your mind after the decision has been made - in the business domain - the decision was of little if any value. This is the value at risk discussion. How much are we willing to risk if we don't know to some level of confidence what the outcome of our decision is?
The elements of good decision analysis are . So for any good decision and its decision making process, we'll need answers to the questions on the left, some form of logic to make a decision, the defined actionable steps from that decision and then an assessment of the outcomes to inform future decisions - learning from our decisions
Decision support systems that implement the process above are based in part on the underlying uncertainties of the systems under management. Research into the cost and schedule behaviors of these systems is well developed. Here's one example.
In the end the decision making process will not meet the needs of the decision makes if we don't have alternatives defined, information at hand - and most times this information is probabilities information from condition in the future in the presence of uncertainty, and the value we assign to the outcomes - then making decisions is going to turn out BAD.
We're driving in the dark with the lights off, while spending other peoples money and our project will end up like this...
Reference Material for Further Understanding
- Strategic Planning with Critical Success Factors and Future Scenarios: An Integrated Strategic Planning Framework, Technical Report CMU/SEI-2010-TR-037 ESC-TR-2010-102.
- Decision Analysis for the Professional 4th Edition, Peter McNamee and John Celina,
Real Options: Managing Strategic Investment in an Uncertain World, Amran, Martha, and Nalin Kulatilaka,Harvard Business School Press, 1999.
- Making Hard Decisions: An Introduction to Decision Analysis, Robert Clemen,
Duxbury Press, 1996.
- Software Design as an Investment Activity: A Real Options Perspective, Kevin Sullivan and Prasad Chalasani
- Probabilistic Modeling as an Exploratory Decision Making Tool, Martin Pergler and Andrew Freeman, McKinsey & Company, Number 6, September 2008
- Value at Risk for IS/IT Project and Portfolio Appraisal and Risk Management, Stefan Koch, Department of Information Business, Vienna University of Economics and BA, Austria,