There was a tweet yesterday ...
In casinos, the betting rules ensure the house wins (Eliminates Risk) over time
This, of course, is not true, nor is it how casinos work. Let's start with a brief overview of Monte Carlo Simulation. Not because of its use for estimating cost, schedule, and technical performance outcomes on projects, but because this simulation started with a casino's need - in Monte Carlo - to have some level of confidence on how much money they needed every night for the roulette wheel.
But let's go back to the beginning. In Aristotle's De Moti Animalium, c 400 BC, the phrase to the left is found.
The rough translation is ...
But how does it happen that thinking is sometimes accompanied by action and sometimes not, sometimes by motion, and sometimes not?
It looks as if almost the same thing happens as in the case of reasoning and making inferences about unchanging objects.
But in that case, the end is a speculative proposition ... whereas here the conclusion which results from the two premises is an action. ... I need covering; a cloak is a covering. I need a cloak. What I need, I have to make; I need a cloak. I have to make a cloak. And the conclusion, the “I have to make a cloak,” is an action.
In the case of Aristotle, the casinos, and all project work, uncertainties exist.
Uncertainties are things we cannot be certain about - this is the standard state for all project work.
These uncertainties are created by our incomplete knowledge of the situation - not by our ignorance of the situation, just incomplete knowledge. These uncertainties are also created by the naturally occurring variances in the underlying processes of the situation.
For the roulette table and dice (craps) table, these underlying naturally occurring variances come from the spinning of the wheel and the tossing of the dice (assuming both are Fair). For card games, the uncertainties come from lack of knowledge of what cards are in the player's and dealer's hands.
So when we say Uncertainty we speak of a future state of a system (cards, dice, a ball bouncing on the wheel, an outcome from the development of a software management decision for cost, schedule, or technical performance of a Feature) that are not fixed or determined.
These uncertainties are related to three aspects of the world we live in:
- The external world - the activities of the project, the casino game, the externalities as an economist call them.
- The knowledge of this world - the planned and actual behaviors of the system.
- Our perception of this world - the data and information derived from the data we receive about these behaviors.
To increase our probability of success - managing SW projects or playing at the casino - we need to Manage in the Presence of Uncertainty and answer each of these success factor questions:
- What does Done Look Like? In units of measure meaningful to the decision-maker?
- What's our plan to reach Done?
- What resources do we need to reach Done?
- What are the impediments to reaching Done?
- How are we measuring progress toward Done?
If we're the gambler - at least a successful gambler † - we have these measures of the probability of success. So does the house. If not, the House would be out of business. and since there used to be a sign as you entered Las Vegas that said ...
Keep Las Vegas Green, Bring Money
Looking at the House side of the risk of losing too much money, there are two kinds of uncertainty, which have been mentioned here many times before ...
- Aleatory Pertaining to stochastic (non-deterministic) events, the outcome of which is described using statistical processes.
- From the Latin alea.
- For example in a game of chance stochastic variability are the natural randomness of the process and is characterized by a probability density function (PDF) for their range and frequency.
- Since these variabilities are natural they are therefore irreducible.
- Epistemic (subjective or probabilistic) uncertainties are event-based probabilities, are knowledge-based, and are reducible by a further gathering of knowledge.
- Pertaining to the degree of knowledge about models and their parameters.
- From the Greek episteme (knowledge).
Separating these classes uncertainty helps in the design of assessment calculations and in the presentation of results for the integrated risk assessment.
For Projects, there are 3 conditions of Aleatory uncertainty
- The aleatory model contains a single unknown parameter.
- Duration
- Cost
- The prior information for this parameter is homogeneous and is known with certainty.
- Reference Classes
- Past Performance
- The observed data are homogeneous and are known with certainty.
- A set of information that is made up of similar constituents.
- A homogeneous population is one in which each item is of the same type. This is the IID condition needed for the Central Limit Theorem, which is many times ignored in Kanban, and simple-minded approximations of Story Points, Stories, and generally unknown by Agile practitioners, since they didn't attend Mrs. Germains physics class in High School.
Any measurement, in the presence of uncertainty (Aleatory or Epistemic), has
- Precision - how small is the variance of the estimate?
- Accuracy - how close is the estimate to the actual value?
- Bias - what impacts on precision and accuracy come from human judgment or misjudgment while making the estimate?
These can be modeled as:
To make a credible estimate - for cost, schedule, technical performance OR how much cash we need tonight in the vault to cover the bets of the players in the casino requires that the estimate has precision and accuracy.
When we connect cost and schedule for our project and construct a model of how these interact, we get a picture that looks like this:
This picture is from the National Reconnaissance Office, Cost Group Risk Process, Tim Anderson, The Aerospace Corporation, 2003.
So How Do We Come Up with These Estimates?
One way, and there are several, is to use Monte Carlo Simulation.
There is an incorrect statement going around the #NoEstimates community that Monte Carlo Simulation cannot be used on a single project. This is a fallacy, based on the confusion between ensemble statistics and specific process statistics. Those making this claim have read a book on Macroeconomics and confuse the principles there with the Microeconomics of choices and model for projects.
Anyway, here's how the Monte Carlo Simulation came about. It came from the Casino in Monte Carlo when George Louis Leclerc, Comte de Buffon was asked how much money do we need to cover tonight's gambling. He created a model that asked and answered what was the probability that a needle would fall across one of the Green lines, shown below?
Turns out this occurs only if A < l sin 𝛳. The casino at Monte Carlo used this model to ...
- Examine all the possible states of a variable, not just the mean and variance, which that had been doing.
- Provide an accurate (sufficiently accurate) estimate to answer their question of how much money.
- Understand the sensitivity of the interacting variables.
- Determine the dependencies between all the variables in the casino.
This method - Monte Carlo Simulation - was used during World War II to model the development of the Atomic Bomb - see I got us back to Mrs. Germain's physics class at Western High School, Las Vegas, Nevada. There's a nice paper summing up this history Beginning of the Monte Carlo Method.
But Monte Carlo Simulation can be:
- Any method which solves a problem by generating suitable random numbers and observing that fraction of the numbers obeying some property.
- A method that provides approximate solutions to a variety of mathematical problems by performing statistical sampling experiments on a computer. The WWII Computer was a mechanical computer, where the term computer was a job description, mostly filled be women working at Los Alamos, New Mexico on the Bomb.
- A method applied to problems with no probabilistic content as well as to those with inherent probabilistic structure.
The Punch Line
I'll say this in the loudest voice possible so everyone can hear, especially those redefining terms in the software estimating world
You Can Not ELIMINATE Risk Unless You Eliminate ALL Uncertainty in your Knowledge of the System (Epistemic Uncertainty) and ALL the Naturally Occurring Variances in the Underlying Processes driving that system (Aleatory Uncertainty).
Here are Some Resources for Using Monte Carlo Simulation for Projects
† Full disclosure, I went to High School in Las Vegas, Nevada. My father worked at the Nevada Test Site where underground nuclear weapons testing occurred before the SALT II treaty. But we had neighbors who were Firemen, Accountants, and Pit Bosses in the casinos. My High School girlfriend's father was the Bar Manager at the Golden Nugget. So our High Physics teacher (who put on the path to graduate physics degree) taught about probability and statics using an example from the casinos, which were applicable to nuclear weapons, environmental cleanup and the probability of success of the amateur rockets we built in those days,
