Yesterday's Post Epistemic Uncertainty Creates Reducible Risk and What Is Risk? was a response to a tweeter post conjecturing
Risk is not there to be mitigated, it's there to be eliminated
This, of course, is not factually true. Risk is the result of uncertainty, which comes in two kinds for all projects, for everything actually. Aleatory uncertainty, from the naturally occurring variances in the process and Epistemic uncertainty from the probabilistic event-based processes that impact the project. Epistemic uncertainty and the risk it creates can be reduced with handling processes.
Aleatory uncertainty and the risk it creates comes not from the lack of information, but from the naturally occurring processes of the system. For aleatory uncertainty, we cannot buy more information nor take specific risk reduction actions to reduce the uncertainty and resulting risk. The objective of identifying and managing aleatory uncertainty to be preparing to handle the impacts when risk is realized.
The method for handling these impacts is to provide margin for this type of risk, including cost, schedule, and technical margin.
Schedule Margin should be used to cover the naturally occurring variances in how long it takes to do the work. Cost Margin is held to cover the naturally occurring variances in the price of something we are consuming in our project. Technical margin is intended to cover the naturally occurring variation of technical products.
Aleatory uncertainty and the resulting risk is modeled with a Probability Distribution Function (PDF) that describes the possible values the process can take and the probability of each value. The PDF for the possible durations for the work in the project can be determined. It turns out we can buy knowledge about aleatory uncertainty through Reference Class Forecasting and past performance modeling. This new information then allows us to update ‒ adjust ‒ our past performance on similar work will provide information about our future performance. But the underlying processes is still random, and our new information simply created a new aleatory uncertainty PDF.
The first step in handling Irreducible Uncertainty is the creation of Margin. Schedule margin, Cost margin, Technical Margin, to protect the program from the risk of irreducible uncertainty. Margin is defined as the allowance in budget, projected schedule … to account for uncertainties and risks.
Irreducible Schedule Risk
Projects are over budget and behind schedule, to some extent because uncertainties are not accounted for in schedule estimates. Research and practice is now addressing this problem, often by using Monte Carlo methods to simulate the effect of variances in work package costs and durations on total cost and date of completion. However, many such project risk approaches ignore the significant impact of probabilistic correlation on work package cost and duration predictions.
Irreducible schedule risk is handled with Schedule Margin which is defined as the amount of added time needed to achieve a significant event with an acceptable probability of success. 8 Significant events are major contractual milestones or deliverables.
With minimal or no margins in schedule, technical, or cost present to deal with unanticipated risks, successful projects are susceptible to cost growth and cost overruns.
This is the purpose of buffers to protect the deliverables from the naturally occurring - Aleatory - uncertainties that exist in All project work.
The notion that Estimates lead to buffers. Buffers lead to waste. Waste leads to ruin, is a fallacy and violates to principles of managing in the presence of uncertainty.
Irreducible Cost Risk
Irreducible cost risk is handled with Management Reserve and Cost Contingency are program cost elements related to program risks and are an integral part of the program's cost estimate. Cost Contingency addresses the Ontological Uncertainties of the program. The Confidence Levels for the Management Reserve and Cost Contingency are based on the program's risk assumptions, program complexity, program size, and program criticality.
When estimating the cost of work, that resulting cost number is a random variable. Point estimates of cost have little value in the presence of uncertainty. The planned unit cost of a deliverable is rarely the actual cost of that item. Covering the variance in the cost of goods may or may not be appropriate for Management Reserve.
Assigning Cost Reserves needs knowledge of where in the Integrated Master Schedule or Product Roadmap, with it's Release Plan, these cost reserves are needed. The resulting IMS, with schedule margin, provides locations in the schedule where costs reserves aligned with the planned work and provides the ability to layer cost reserves across the baseline to determine the funding requirements for the program. This allows the program management to determine realistic target dates for program deliverables and the cost reserves ‒ and schedule margin ‒ needed to protect those delivery dates.
Irreducible Technical Risk
If we use the definition A Margin as the difference between the maximum possible value and the maximum expected Value and separate that from contingency is the difference between the current best estimates and maximum expected estimate, then for the systems under development, the technical resources and the technical performance values carry both margin and contingency.
- Technical Margin and Contingency serve several purposes:
- Describe the need for and use of resource margins and contingency in system development.
- Define and distinguish between margins and contingency.
- Demonstrate that, historically, resource estimates grow as designs mature.
- Provide a representative margin depletion table showing prudent resource contingency as a function of project phase.
For any system, at any point in its development life, there is a maximum possible, maximum expected, and current best estimate for every technical resource. In general terms, the current best estimate of a resource changes as the development team improves the design; that is as the design matures.
For a system in development, most technical resources carry both margin and contingency. This is true historically and, independent of exactly why, development projects must plan for it to occur.
The goal of Technical Margin (unlike Cost and Schedule Margin) is to reduce the margins (for example Size Weight and Power) to as close to zero as possible, to maximize mission capabilities. The growth and its handling include:
- Expected growth ‒ contingency accounts for expected growth
- Recognize mass growth is historically inevitable.
- As systems mature through their development lifecycle, with better understanding of the design emerging from conceptual to actual designs.
- Requirements changes often increase resource use
- Unplanned growth ‒ margins account for unexpected growth
- Recognize space system development is challenging
- Projects encounter "unknown unknowns" with the use of new technology that is difficult to gauge, that develop into uncertainties in design and execution of the program, all the way to manufacturing variations.
Wrap Up
Risk does not standalone, Risk comes from Uncertainty - reducible uncertainty and irreducible uncertainty
You cannot remove the risk without first removing the uncertainty.
For those interested in future understanding how risk is managed on software intensive system of systems using agile development here's a bibliography from our several decades of doing just that for Enterprise, embedded control systems, and space and defense systems.