I was invited to participate in the Twitter #PMCHAT session today to speak about the principles of Increasing the Probability of Project Success. The #PPChat process is based on asking questions and engaging with those providing answers.
Here are the questions asked
- Q1: for the projects you manage, how do you know what Done looks like, and who defines it?
- Q2: When you state what Done looks like, what units of measure do you use to describe Done?
- Q3: What role does risk management play in the management of your projects?
- Q4: Do you make estimates of cost and schedule?
- Q5: How do you measure progress toward Done beyond the passage of time and consumption of money?
- Q6: How often do you assess progress to Plan?
- Q7: How often do you change your plan and what are the causes of those changes?
You can find the Twitter thread for the answers.
But here are the resources to support the answers found in our Software Intensive System of Systems domain
Q1: For the projects you manage, how do you know what Done looks like, and who defines it?
Defining Done starts with defining what Capabilities are needed from the project to accomplish the mission or fulfill the business strategy.
“Capabilities-Based Planning… starts with the functional analysis of operational requirements. Capabilities are identified based on the tasks required… Once the required capability inventory is defined, the most cost-effective and efficient options to satisfy the requirements are sought.” [1]
Q2: When you state what Done looks like, what units of measure do you use to assess progress toward Done?
Capabilities–based planning transforms the delivery of features and functions into the delivery of processes and technologies that support the business strategy or means to accomplish a mission.
Capabilities–based planning is planning, under the conditions of uncertainty, to provide capabilities suitable for a wide range of business or technical challenges and circumstances, while working within an economic framework. This approach emphasizes flexibility, adaptiveness, and robust capabilities, implying a modular building–block approach to the delivery of enterprise applications.
The units of measure toward Done are: [4]
- Measures of Effectiveness (MoE) of the delivered Capability - Operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions. MoE expresses the extent to which an MoP satisfies a stated user requirement.
- MoE's are stated in units meaningful to the buyer, focus on Capabilities independent of any technical implementation, and are connected to mission success.
- MoE's belong to the End-User
- Measures of Performance (MoP) of the delivered Capability - Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. An MoP is a measure of what a system achieves in operation.
- MoP's are attributes that assure the system has the capability to perform, are an assessment of the system to assure it meets design requirements necessary to satisfy the MoE.
Without these measures, the only assessment of Done is the Passage of Time and Consumption of Money or any other measure that cannot be traced directly to a Capability, with its MOE's and MOP's. [3]
Q3: What role does risk management play in the management of your projects?
As Tim Lister says Risk Management is How Adults Manage Projects.
As project managers, we can only manage in the presence of uncertainty that creates risk. Since all projects operate in the presence of uncertainty, we must find ways to handle the risk created by uncertainty. So our risk management approach is actually a risk handling approach since we can't make the uncertainty go away. [2]
Uncertainty comes to two forms on projects:
- Epistemic uncertainty, which comes from a lack of knowledge. Epistemology is the study of knowledge. The handling strategy for risk created by epistemic uncertainty is to buy down this uncertainty with new knowledge
- Aleatory Uncertainty, which comes from the natural variances on projects. The natural variances of cost, schedule, and technical performance activities. The handling strategy for risk created by aleatory uncertainty is the margin. Cost, schedule, and technical performance margin.
- Any project schedule without margin is late on day one
- Any project budget without Management Reserve is over budget on day one
- Any technical outcome without margin on the performance targets is going to be a disappointment.
The critical understanding here is
- The risk created by epistemic uncertainty is reducible can be handled by spending time and money to learn more about the uncertainty
- The risk created by aleatory uncertainty is irreducible and can only be handled with a margin
There is a third uncertainty, Ontological Uncertainty which represents the state of complete ignorance. If we have this uncertainty on the project, nothing can be done to address it.
Q4: Do you make estimates of cost and schedule?
Since all projects operate in the presence of uncertainty - reducible and irreducible - estimates are needed for any credible decision-making process
- An estimate as a noun is an approximate calculation or judgment of the value, number, quantity, or extent of something.
- An estimate as a verb is to roughly calculate or judge the value, number, quantity, or extent of.
But those estimates, nouns, and verbs themselves have other attributes. They have precision, accuracy, and bias
- Precision - how small is the variance of the estimate
- Accuracy - how close is the estimate to the actual value
- Bias - what impact on precision and accuracy come from human judgments or misjudgment?
The best starting point for determining the NEEDED precision and accuracy is to determine the Value at Risk.
Estimating and the resulting Estimates must be described by their accuracy and precision. If you hear any other description, like estimates can't be precise, or estimates are never accurate, those words are mathematically incorrect.
The accuracy and precision themselves have accuracy and precision. This is the error on the error - the confidence in the error - that is needed.
Q5: How do you measure progress toward Done beyond the passage of time and consumption of money?
All project success requires measuring Physical Percent Complete (P%C). That starts with defining what P%C means for the deliverables and how that P%C increases as a function of time. This is the principle of a control chart with an upper and lower control limit. This is a closed-loop control system.
- Define a target
- Measure the current state or status
- Determine the variance
- If outside the upper or lower control limits, take corrective or preventive actions to get the measured entities back inside those limits
- Do this at a periodic rate suitable for controlling the loop.
Q6: How often do you assess progress to Plan?
The first question is how long are you willing to wait before you are late? The answer is At a minimum, 1/2 the time before that date arrives.
Since the information created by the project - cost, schedule, and technical performance are signals to the project manager for managing in the presence of uncertainty, we can use a signal processing paradigm. In this signal processing paradigm, the Nyquist sampling principle states that the sampling rate must be at least twice the maximum bandwidth of the analog signal to allow the signal to be completely represented.
Q7: How often do you change your plan and what are the causes of those changes?
In the presence of uncertainty, plans change all the time. Otherwise, we would be operating in a deterministic world. So Change Management is a critical success factor for all project success.
The challenge is how to manage change in the presence of uncertainty?
The answer to this is based on the domain, context, and Value at Risk
- For an agile development software project at a shoe company near our home, changes can be made when needed to support business operations.
- For a flight safety development project subject to DO-178C Software Consideration in Airborne Systems and Equipment Certification changes can only be made through a strict formal risk assessment process.
- For a Nuclear facility subject to ASME NQA-1, changes can only be made after the evaluation of the effects of those changes on the overall design, and any analysis upon which the design was based.
Compendium's of Resources for each Topic
Bibliography for this Post
[1] "Analytic Architecture for Capabilities-Based Planning, Mission-System Analysis, and Transformation," Paul K. Davis, RAND Corporation
[2] "Risk-Informed Decision Making Handbook," NASA/SP-2010-576
[3] "A consistent multi-user, multi-goal framework for assessing system performance with application to a sonar system," Dr. Colin M. Reed and Dr. Alan J. Fenwick,
[4] "Technical Measurement," INCOSE- TP-2003-020-10
