Herding Cats

The World of Probability and Statistics

This is from an article about the application of Bayesian Statistics to a civil suit in the UK over the source of a building fire.

The idea that you can assign probabilities to events that have already occurred, but where we are ignorant of the result, forms the basis for the Bayesian view of probability. Put very broadly, the 'classical' view of probability is in terms of genuine unpredictability about future events, popularly known as 'chance' or 'aleatory uncertainty'.

The Bayesian interpretation allows probability also to be used to express our uncertainty due to our ignorance, known as 'epistemic uncertainty', and popularly expressed as betting odds. Of course there are all gradations, from pure chance (think radioactive decay) to processes assumed to be pure chance (lottery draws), to future events whose odds depend on a mixture of genuine unpredictability and ignorance of the facts (whether Oscar Pistorius will be convicted of murder), to pure epistemic uncertainty (whether Oscar Pistorius knowingly shot his girlfriend).

When we build probabilistic models of project performance - cost, schedule, and technical - we assume we understand the underlying statistical processes that drive these probabilistic generating functions. These are the aleatory uncertainties in duration, cost, and performance. We define the Probability Density Function in the Monte Carlo Simulator. Then we apply that to the network of work activities (the Integrated Master Schedule), to produce confidence outcomes for completing on or before a planned date and at or below a planned cost. This is all fine and dandy. But we really don't know the underlying drivers that create coupling, correlation, and cross correlations between the work activities, cost, and technical performance. These can be model by discovering the drivers in the network.

For the Epistemic uncertainties we need another modeling tool. The current tools don't actually use Bayesian statistics, rather they use Monte Carlo Simulation and treat the Probability of an Event as an aleatory process integrated with the other PDF's, ranges, and their shapes (Kurtosis and Skew).

We're missing the tools needed to construct a credible epistemic model of how the program works. Using the Integrated Master Schedule (IMS) as the topology for work, the probabilistic behaviours of the work elements at each node - cost, schedule, and technical performance compliance of the products - and the coupling and cohesion of the nodes. With this information - assuming it is credible, which is a HUGE assumption - we could model the behaviour of the program and ask what if questions. 

Probabilistic Cost and Schedule Processes

We hear all the time about how project managers can't make credible schedules or generate credible cost estimates for IT project or even other types of projects - mega project construction for example.

I'd have to say Flyvbjerg is a bit over the top calling estimaters fools and lairs, but he does have a point about the in-credibility of cost and schedule estimates.

We about how building such estimates is a waste of time and we should switch to doing everything in an agile manner and let the requirements emerge over time while spending the clients money to discover them.

This is an exaggeration I know and it literally doesn't work that way even for agile processes. But it brings out the point that developing credible estimates of cost and schedule is actually hard and requires careful consideration to the processes.

The first thing to recognize is there is no such thing as a credible estimate in the absence of the underlying statistical and resulting probabilistic processes being applied to that estimate. The notion that we can forecast the duration of a work activity or the cost of that activity without a probabilistic confidence to that forecast is näive at best and delusional at worse.

Maybe if Flyvbjerg had used the work "delusional" it would set better with me. Liar implies malicious intent. Fool implies no one else was looking after the work and a "fool" was left in charge of the estimiating process. 

What To Do About This?

Here's an example of the probability distribution of a cost estimate for a project.

This is from a model project that has a cost loaded IMS. The model will be only as good as the IMS is good and the resource loading is good. So this approach ONLY works when the model is right. That's a problem of course, but for now let's assume the model is right, since it came from a VERY credible source. David Hulett and I work the Risk and Opportiunity policy segment of the US DOD, along with others from government and industry.

The 50^th percentile cost is $2,296,898 (a small project in the A&D world BTW). That means there is a 50/50 chance the project will cost more than that or less than that. That number is the median - the middle of the range. If we want to know what the number is with an 80% confidence, than it is $2,333,153. That says there is an 80% confidence that the cost of the project will be $2.3M or less.

When we speak about schedule, we can use the same terminology. In this case there is an 80% confidence the project will complete on or before 11/06/2015. 

These graphs are generated from a Monte Carlo simulation tool applied to a resource loaded Integrated Master Schedule (IMS). If there is any doubt as to why you MUST create a resource loaded schedule, please please put those doubts away. These two graphs are the source for the Joint Confidence Level showing the over all probability distributions for both cost and schedule for the project.

This picture should convince anyone that the "joint" probability of completing on or before the target date (I didn't say what that was) and at or below that planned budget (didn't say what that was either) needs to be modeled in a way that would be considered credible. 

Outside of a small domain of aerospace and defense this is rarely if ever done. Most IT projects have some made up budget and schedule. Their strategy is based almost entirely on HOPE with no underlying assessment of the statistical and probabilistic drivers of the cost and schedule, let alone the technical aspects coupled with risk.

And we wonder why IT projects come in late and over budget.

Turns out of course large defense and space programs have the same problem - poor estimating, politically motivated estimating, naive estimating, etc. But they are required to produce data like this, so we know it is bogus sometimes early enough to fix it - sometimes not.

But the real root cause of most serious overages for A&D comes from other sources. Those can be found in the Nunn McCready breach reports at RAND in Volume 1 and Volume 2. IDA has there assessment as well. There are some professional provocateur voices that conjecture all this can be fixed with a good WBS, a scaled down Earned Value Management System, and "firing the program managers when they go over cost and schedule." That of course is probably not the real solution.  Read the RAND reports for the root causes of major overages. Read the Feb 18th issue of Aviation Week, "Slaughterhouse Five: If sequestration can rewrite U.S. strategy It's time to kill the sacred cows in industrial policy." F-35's current cracked first compress blade grounding and the world wide subcontracting effort that provides the program with nearly invincible support in congress.

We only wish it were as simple as many would have us believe. But sadly it is one of those wicked problems that we simply have to manage in the presence of. Many have suggested solution - credible ones, not the cocka mamy ones - but those solutions usually don't address the political forces in play when you are working with the public's money.

Why I've Followen in Love with R

In the programming language R, drawing the joint probability plot is done with a simple command

> pairs(X)

produces a scatter plot of the variables defined by the columns in X. Every column in X is plotted against everyother column of X and the resulting n(n-1) plots arranged in a matrix with constant plot scales over the rows and columns of the matrix.

Complex Problems Require Better Solutions

There is a class of responders on the forums that continue to seek simple and simple minded solutions to complex problems. Claiming the problems can be solved with simple solutions. Of course those solutions cannot be demonstrated with any credibility outside of a limited domain. Let's start with Mencken

For every complex problem there is an answer that is clear, simple, and wrong.  - H. L. Mencken

In the project domain most of the problems start with poor estimates of cost and schedule. Hollmann starts it off  http://lnkd.in/vjf4X4. Bent Flyvbjerg takes a more radical view, calling estimators liars , but I chalk that up to a language barrier.

The core problem starts with the politics of estimating. Rarely do the buyers of the project know what it should cost. So when they hear a number that is larger than their expectations, they reject the number. They signal to the sellers what they expect to pay for something. This expectation can come from poor knowledge or simply a naive understanding of the problem, or worse a public promise that simply can't be kept. 

But a second problem exists. That across domains, there is the wrong headed belief that one domain has the solution to another domains problems. We need to break the loop we're stuck thinking that problems are only found in other domains and making the ill-informed suggestions that "if you just did it my way," there would be no problem. There is a fundamental issues with managing complexity and managing in the presence of uncertainty. We want it to be simple, it is not, but since it is hard, people don't want to actually work hard on the solution. Instead seeking simple and may times "simple minded" solutions - buy this tool, use my excel spread sheet, dumb down the processes.

So What Is The Solution

First is to recognize that ALL domains have similar problems. Cost and Schedule estimates have been discussed for many decades. Lots of literature leads us back to the singular source - the political will to search for the credible estimate, even if that means not starting the project. Or better yet, to start the project knowing the estimates are flawed.

One provocateur voice in our community suggests that PMs that take on project with poor estimates should be held criminally liable for the results. This of course is laughable, but it demonstrates the complete understanding of the problem and the solution.

This approach reminds me of Pauli's quote when presented with a nonsensical submission from a student.

They were worse than wrong because they could not be proven wrong. Famously, he once said o: Das ist nicht nur nicht richtig, es ist nicht einmal falsch! "Not only is it not right, it's not even wrong!"

Thus is where we are now. Not much in the form of research and questioning, lots of pontificating about how the other guy is all messed up, and if you'd only do it my way, you'd have your solution. Reminds me of the current US political problems. Do it my way. Of course that doesn't work, because we have complex - wicked - problems, and there is no single way. Mencken was right, and he was right long before any of the current problems.

We may be doomed to repeat history with first recognizing the problem is us.

Measurement of Uncertainty

When we talk about measuring things in project management, there are four types of outcomes for two attributes of that measurement.

The measurement can be accurate and the measurement can be precision. We want it to be both. But without knowing if it is both, we're probably not going to get what we expected. 

This is translated into our estimating processes like this. We need to know the accuracy of the estiimate as well as the precision of that estimate.

It does no good to have a precise estimate, that is not accurate. Or to have an accurate estimate that is not precise. Both are needed, but we also must know if we have accuracy and precise in the presence of natural variability of the underlying processes.

So when someone says I estimate that this project or this part of the project will be $127,700 and take 6 months, you can ask them back accuracy and precision questions.

 

Unrealistic Cost and Schedule Estimates

There is a popular notion that project and program overruns come from some core unprofessional behaviour of those involved in constructing the estimate. That there is some basic malfeasance afoot in the cost and schedule business. This conjecture usually comes from those on the outside (far outside) looking in.

An April 18th, 2011 report discovered some interesting facts in the US DOD domain...

The source of these problems is not unprofessional or incomplete cost estimates. It is bad programmatic assumptions

One example is the current poster child for cost and schedule overrun - the F-35, Joint Strike Fighter

The bad assumptions (postulates) include:

• The state of the art was much more mature than it actually was. The Technology Readiness Level (TRL) of the critical components of the aircraft was not well understood. Therefore cost and schedule estimates based on TRL models were wrong.
• The vertical lift component of the aircraft was assumed to be model-able in CATIA with simulation tools. Turns out that was not the case.
• The multi-service requirements - the Joint part of Joint Strike Fighter - would be straight forward. The book Illusion of Choice: The F-111 and the Problem of Weapons Acquisition Reform, was not on the reading list. This should be mandatory reading for anyone working in new and innovative domains, government or private sector.

The core issue is there are deeply held programmatic assumptions about getting things done by specific Milestones. These Milestones are baked into the process - Government or Private. 

The source of these problems starts with the missing basics.

• Missing or poorly developed requirements -This includes immature requirements. Requirements committed to, too early in the life cycle of the program. We'll discover the requirements are we go, it'll be OK because we have an open architecture and can adapt to emergent needs.
• Undefined and possibly undefinable dependencies - between the programmatic elements and the increasing technical maturity of the deliverables. We didn't know there was deep connection between the behaviour of two critical components. This behaviour is not favorable, so neither component now works properly.
• Missing or underanalyzed interoperability assessments - you forgot to tell me we had to work with that other part of the system.
• Requirements instability - we keep discovering new needed capability and the resulting requirements. 
• Lack of stakeholder involvement - if we can't talk to the "real" stakeholders, let's just keep going and they'll catch up.
• Poorly developed Significant Accomplishments and Accomplishment Criteria - we don't know what done looks like, even if it walked through the door.
• Lack of a credible Integrated Master Schedule - schedules, we don't need no stinkin schedules. Without a credible, risk adjusted, resource loaded, outcome measurable schedule, the project is just level of effort, executing until you run out of time, money, or both
• Looking at program performance at too high a level - the overused, silly and un-ctionable statement of "the devil is in the details" doesn't really help. But reading Darrell Huff How To Lie with Statistics, shows you how to hide the variance by agragating them to the top.
• Poor integration testing - unit testing, subsystem testing, system testing are all necessary but not sufficient. Full up integration testing - "test as you fly" is the final proof that the result actually has chance of working.
• Poor or missing integrated planning - a plan is a strategy for success. The strategy is a hypothesis. The hypothesis needs constant testing - experiments - to confirm we are not fooling ourselves. This is basic high school chemistry class, but seems very hard in program management.

These are inspired by Mr. Bliss's "Summary of PARCA RCA Findings after One Year," Acquisition Insight Days, 2011.

Scheduling Guides

Yesterday's post on overusing and misusing activity relationships needs some follow on. In the domain we work - space, defense, energy, process industry - there are scheduling guides.

These guides provide the framing assumptions for constructing a credible Integrated Master Schedule.

Here's some of the guides that we work with

• Air Force Integrated Master Schedule (IMS) Assessment Process - a more detailed implementation of the PASEG
• NDIA Planning and Scheduling Excellence Guide - a draft of a scheduling guide, but missing the Integrated Master Plan
• GAO Cost Estimating and Assessment Guide  - has best practices for the IMS as well as other needed parts of the Performance Measurement Baseline
• DCMA Earned Value Management System - System Surveillance Guide - contains the 14 Point Assessment
• The Integrated Project Management Handbook - Dayton Aerospace's guide for the IMP/IMS
• NASA Scheduling Management Handbook - NASA's version of how to be a credible scheduler
• AACEI Planing and Scheduling Professional Study Guide - the brochure for the PSP certificate
• DOD/DAU Scheduling Guide for Program Managers - an aging guide for Government Program Managers
• Integrated Master Plan analysis - an overview of the concept of an IMP at Space and Missile Systems Command
• Air Force Material Command Pamphlet 63-5 - buidling the Integrated Master Plan in the US Airforce
• Marshall Space Flight Center Project Management and System Engineering Handbook - MSFC's guide to building credible schedules and manging the program with them.
• Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide - how to build the Integrated Master Plan and Integrated Master Schedule
• NAVAIR Integrated Master Schedule Guide - this is my favorite because it speaks directly to the problem of building a credible IMS. The building building them don't know what they're doing, get better IMS (and IMP) builders and your program's probability of success will increase. As the safety poster campaign at Rocky Flats (where I was one of many Program Managers) says Stop doing stupid things on purpose.

As well most programs we work have Command Media that describes how the Integrated Master Schedule is to be developed and used. These documents are driven by several DID's 

• DI-MGMT-81861 is the current

So when there is a discussion about how to develop schedules, what relationships to use, how to construct the Program Critical Path and get the Schedule Critical Path to match it. This BTW is an important issue. In the spacecraft business or any high technology business, the Program Critical Path is know to the subject matter experts. For example if the propulsion system is not on the Critical Path then your IMS is bogus, no matter what the scheduling tools says. Same in the BioPharma world for the Clinical Trial CRO contracting processes, etc. etc. etc.

So What Does This All Mean

• Scheduling is a profession
• Treating the schedule as a personal creation is a mistake, the Integrated Master Schedule is the document of record for how the project or program will be executed. Developing the schedule as a piece of arrt is a mistake
• Building a credible schedule is hard, takes effort, experience, and guidance. Go look at guideance and take with a grain of salt those making claims about schedules, scheduling, and the guidance it they have not actual worked hands in the specific domain. Have you built spacecraft for NASA Houston or Marshall SFC, using thie gudiance documents? No? Uhm. 

Finally here's a brief overview of how to Build the Perfect Schedule.

Related articles

Good Project Management Advice

Why You Need A Plan and a Schedule To Go With It

Requirements Elicitation

The Train Wreck of all Projects Starts With These Problems

The Purpose of Risk Management

NOOOO, Don't Do It

There are several posts on the web this week about relationships (predecessors and successors) in project schedules. Some suggest using all the relationship to create as Cleaver a schedule as needed to represent. Another asks the question, who uses all the relationships.

These relationships include Finish to Start (FS), Start to Finish (SF), Start to Start (SS), and Finish to Finsh (FF).

Plus there can be leads and lags for each of these.

Without drawing out the pictures for each these (you can find them here), there is a very simple answer to the questions

USE ONLY FINISH TO START

That's it, Full Stop as they say in England. Don't use anything else. Here's why. First the domain we work in doesn't like it. By doesn't like it, I mean the Defense Contract Management Agency will write a Discrepancy Report (DR) for your Integrated Master Schedule if you have cleaver things like SF - 20d. 

Their assessment is called the 14 Point Assessment. The guidance is found in the Earned Value Management System (EVMS) Program Analysis Pamphlet (PAP). Now every project uses Earned Value, even thoughthey should. But the advice in the PAP is good advice for all projects.

The real reason to use only FS, is all other relationship hide the natural flow of the path through the project. You can convert any of the other three releationships to a FS by simply adding a zero duration activity at the beginning or at the end of the other activity.

There are some in the community that contrive situations were SF is needed for example. This is course is exactly that - contrived. There is no actual work relationship that can be expressed with a FS.

Along with this FS advice is the use or mis-use of LEADS and LAGS. NAVAIR provides advice for this. You can have a 5 day lead or lag. On a 4 year program that is the same as ZERO days, so don't even waste your time.

Inside put a real activtiy to hold the LEAD or LAG. Here's how to do this. If you have work that has a lead or lag, this creates the opportunity for rework. Break up the work into two parts. One that send forward completed ready to use outcome and the second that carrys on.

It's Not Chaos It's Football

There is a popular myth - mostly by those Agile Management 3.0 thought leaders who don't play or even know much about team sports involving balls, grass - about "emergent" behavior. The myth is that the "rules" of the game and the boundaries of the field are the governing guidance in the game of Football.

The Michigan coach in the picture to the left is looking at his playbook. The playbook defines what plays can be, or are going to be, used during the game.

The playbook has to follow the rules of the game. That's the role of the referees and field judges. If the play is not in the playbook, it is not going to be used during this game. There may be more plays available to the team than are in the playbook for today's game, but they're not going to be used.

So here's the lesson for Project Management.

Where is your Playbook? And If you have a Playbook, are those Plays going to increase your chances of winning?

Don't have a playbook?, then it's going to be an ugly day. Have a playbook but don't execute the plays properly?, it's going to be an ugly day. Execute the proper plays in the proper sequence, your chances of winning will improve. No guarantee's, just be improved.

The Playbook for a project is the Integrated Master Plan. The application of the plays, like American Football, depends on the emerging situation that resulted from the last play. Not the score on the score board. That's the Integrated Master Schedule.

Does this mean the Master Plan doesn't change during the game or during the project? Of course not, that would nonsense. But you better have a really good reason for changing the plan. In football and baseball, that's actually a strategy - to get the other team to change their plan. To go off the playbook and try doing something that they haven't practiced well enough to properly execute. To disrupt the flow of play.

How about changes in the Master Schedule during the game or during the project? Happens all the time. Here's a thought to put to work in ball games as well as projects.

Planning is not “plan and forget” but an ongoing dynamic activity that peers into the future for indications as to where a solution may emerge and treat the plan as a complex situation, adapting to an emerging solution.

If you have a Plan B, you should have given thought to the possible Plan B's, before you execute Plan B. Otherwise your Plan B may be just what your opponent wants you to do - decrease your probability of success.

This idea comes from a colleague Mike Dwyer, IT Program Manager, used with permission.

So why the notion that a football team - or a project - that self-organizes within the boundaries of the playing field and the rules as they are laid down by the football association? Without the understanding of the Playbook, with most times the plays being called in by the coach. Can't say. Maybe because those notions come from observers of football, not players of football.

The Naviety of Solutions to Wicked Problems

In this silly season of national elections, the notion of simple solutions to wicked problem is rampant. The same is true for wicked problems in the project and program management space. Government procurement is a wicked problem.

There are always voices out there ranting on about how bad everything is - and there some real examples of how bad things are - without actually providing any credible solutions.

If we first look at what it means to have a Wicked Problem we see, Rittle and Webber's definitions

• There is no definitive formulation of a wicked problem (defining wicked problems is itself a wicked problem) - this makes some sense, but is not too useful for searching for solutions
• Wicked problems have no stopping rule - this means the problem is continuous. Air pollution, green house gases, health care, federal procurement as examples of the "no stopping rule."
  
• Solutions to wicked problems are not true-or-false, but better or worse - even the better or worse may not have units of measure. Better or worse for who, over what time?
• There is no immediate and no ultimate test of a solution to a wicked problem - if we knew that it wouldn't be wicked would it?
• Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly - this doesn't have to literally be the case. For weapons systems acquisition for example, the problem can be "wicked" but still be repeatable if the complexity is high enough. We know what we want in a stealth fighter, but the actual acquisition of a stealth fighter is a "wicked" problem.
• Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan - this is the basis of Capabilities Based Planning. We need to know a small number of capabilities. When there are 100's or 1,000's of top level requirements, the problem becomes "wicked."
• Every wicked problem is essentially unique. Every wicked problem can be considered to be a symptom of another problem - a colleague at a local defense contractor said “The problem is…that’s not the problem.”
• The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution - explaining the problem, becomes a "wicked" problem.
• The planner has no right to be wrong (planners are liable for the consequences of the actions they generate) - this is common in all "wicked problems" for cost, schedule and technical performance estimates.

Parallel and Series Tasks and Merge Bias

Had a conversation today with a colleague around probabilistic modeling of cost, schedule, and the impact on risk management. Lots of topics touched, but one was the concept of Merge Bias.

Merge Bias is the impact of parallel tasks on the probability of completing on or before a need date. I've talked in the past about Schedule Margin using Monte Carlo. The notion of a Deterministic versus Probabilistic Schedule is the starting point for Merge Bias.

In the management of projects, the important points along the path to completion are usually where several parallel paths come together. At these merge points, the paths must all be completed before a milestone can be achieved. An inspection is done, sub-assemblies integrated for testing, initial flight testing complete - activities like these.

The Fallacy of the Critical Path and PERT Methods

There are several critical factors in the use of standard scheduling techniques.

• The project duration calculated by the critical path method is accurate only if everything goes according to plan. This is rare in real projects.
• In many cases the completion dates the critical path method produces are unrealistically optimistic and highly likely to be overrun, even if the schedule logic and duration estimates are accurately implemented.
• The critical path method completion date is not even the most likely project completion date, in almost all cases.
• The path identified as the critical path using traditional critical path method techniques may not be the one that will be most likely to delay the project and which may need management attention. This mear critical path is the "killer" path not shown in static assessment methods.

Extreme Parallel and Serial Task Example

Here's an example where all the work is performed in parallel and all the work performed in series. This extreme example is used to show that when we encounter project schedules where merge points are used, there is a hidden effect - merge bias that will result in unpleasant outcomes if we don't understand and deal with the probabilistic aspects of this type of structure.

In the first figure all the work is in series. We plan to finish on 1/25/2013. The work is a series of 10 day activties that have a total of 100 days work.

When we run the Monte Carlo Simulation (MCS) using Risk+ and look at the final outcome, Series Tasks Complete, we get this curve.

This graph tells us there is a 55% probability of completing on or before our target date of 1/25/13. This is a cumulative probability curve for all the 1,000 samples taken in the model. That's OK, since there is a 90% probability of completing on or before 1/30/13, 5 calendar days after our target date. The schedule has no slack, so this is another topic about having margin in all schedules.

The extreme Parallel example looks like this. Here there are two sets of parallel activities, whose total durations to the same 100 days. How they do this is not an issue here, maybe less resources, maybe just different work periods, but the parallel aspects are the point of the model.

The MCS for this parallel case, for the same Period of Performance looks like this. The target date of 1/25/13 is not even in the realm of possibilities. 1/31/13 of only 5% probability. The 80% probability date - the common confidence level of a planned date is 2/13/13, 3 weeks late. This lateness is from the parallel aspects of the network of activities.

So What Does This Tell Us?

It doesn't mean don't have parallel work. It means that when there is parallel work, the standard PERT style estimating is no good. It means:

• The network should not be developed at too high a level of detail. This creates or may even mandate parallel flows.
• The schedule should show clearly the parallel paths that could cause the project to be late if not coordinated.
• The network should not be developed in such great detail that it requires too much information and create more opportunities for parallel flows.

Here's the killer concept that is little understood

When we think of statistical processes, say an estimate of cost, the variability of these numbers will tend to the Mean. That is, when there is an estimated cost with a higher than planned cost, that will be (or could be) off set by an estimate with a lower than planned cost. This is the notion of the Central Limit Theorem. Sets of random numbers tend to the mean when the sample size grows large.

In a schedule, when the work is in parallel, when the duration of a work activity along a path experiences a higher than planned result, and another work activity experience a lower than planned result, it is the higher than planned duration activities that impacts the schedule. If both are higher, than the highest one is the result. If both are lower, than the highest of the lower one is the result.

Without a Monte Carlo Simulation of the network any credible confidence in completing on or before at date with a budget at or below that planned value is not there.

Integrating Cost, Schedule, and Technical Performance Measures

In the previous post, the new IPMR DID instructs us to integrate cost, schedule, and technical performance.Here's some more advice.

Now the bigger question. Where is the similar advice for IT projects. Agile starts down this road, but needs a broader capabilities statement. PMBOK based PM doesn't have much to say if anything around this. Home grown PM processes are usually simple and obvious check lists.

It Can't Be Any Clearer Than This

If you're going to have any hope what so ever of being successful with your project, here is some advice. Only with this integration can you know the answers to the 5 Immutable principles of project management.

1. What does DONE look like? - define the needed capabilities in Measures of Effectiveness and Measures of Performance in units meaningful to the decision makers.
2. How are you going to get to DONE? - construct a plan showing how you are going to assess the increasing maturity of the deliverables and a schedule showing the order of the work needed to produce these deliverables, and the measures of Technical Performance needed to assure these deliverables meet the needs of the customer.
3. What Resources are you going to need along the way? - resource load the Integrated Master Schedule - or what ever artifact you are using to show the path to completion.Show the profile (demand and capacity) for each resource.
4. What Impediments will you encounter along the way? - without a risk management plan and a risk management process to support the plan, showing how you are going to handle each risk, you're going to be late, over budget, and your gadget probably isn't going to work.
5. How are you measuring progress to plan? - this measure needs to be in units of physical percent complete. This measure needs to be adjusted for compliance with the Techncial Performance Measures, and this measure must be probabilistically risk adjusted for the variances in performance.

This text in the picture is taken from the new Integrated Program Management Report (IPMR), DI-MGMT-81861.

Requirements Elicitation

Several commenters to Herding Cats asks about requirements elicitation. Here's some top level guidance. But first a fundamental change needs to take place in the IT world on how to capture requirements. This change is guided by the processes used in the defense and space business.

It's called Capabilities Based Planning. Identifying System Capabilities is the starting point for any successful program. Systems Capabilities are not direct requirements, but a statements of what the system should provide in terms of “abilities.”

For example, on the Hubble Space Telescope Robotic Service Mission, three capabilities were needed:

1. Do no harm to the telescope.
2. Change the Wide Field Camera.
3. Connect the battery umbilical cable.

Here's a brief presentation on the concepts of Capabilities and Requirements

Capabilities Based Planning

The critical reason for starting with capabilities is to establish a home for all the requirements. To answer the question “why is this requirement present?” “Why is this requirement needed?” “What business or mission value does fulfilling this requirement provide?”

Capabilities statements can then be used to define the units of measure for program progress. Measuring progress with physical percent complete at each level is mandatory for the techncial assessment of the project's progress. But measuring how the Capabilities are being fulfilled is most meaningful to the customer.

The “meaningful to the customer” unit of measure are critical to the success of any program. Without these measures the program may be cost, schedule, and technically successful but fail to fulfill the mission.

The process flow shown on Page 4 of the slideshow is the starting point for Identifying the Needed Capabilities and determining their priorities.

Starting with the Capabilities prevents the “Bottom Up” requirements gathering process from producing a “list” of requirements – all needed – that is missing from a well formed topology. This Requirements Architecture is no different than the Technical or Programmatic architecture of the system. Capabilities Based Planning (CBP) focuses on “outputs” rather than “inputs.” These “outputs” are the mission capabilities that are fulfilled by the program.

In order to fulfill these capabilities, requirements need to be met. But we need the capabilities first. Without the capabilities, it is never clear the mission will be a success, because there is no clear and concise description of what success means.

The concept of CBP recognizes the interdependence of systems, strategy, organization, and support in delivering the capability, and the need to examine options and trade‒offs in terms of performance, cost and risk to identify optimum development investments. CBP relies on scenarios to provide the context to measure the level of capability.

Requirements Elicitation

Requirements are the defined attributes for an item prior to the efforts to develop a design for that item. System requirements analysis is a structured, or organized, methodology for identifying an appropriate set of resources to satisfy a system need (the needed capabilities) and the requirements for those resources that provide a sound basis for the design or selection of those resources.

It acts as the transformation between the customer’s system needs and the design concept implemented by the organization’s engineering resources.

The requirements process decomposes a statement of the customer need through a systematic exposition of what that system must do to satisfy that need. This need is the ultimate system requirement which all other requirements and designs flow.

There are two fundamental classes of requirements:

• Process Performance Requirements.
• Product Performance Requirements.

The Process Performance Requirements define how the work processes are used to produce a beneficial outcome to the customer.

The Product Performance Requirements define the product specifications and how they are related to the process requirements. As well as the product and process requirements, there are functional and non‒functional requirements.

These non‒functional requirements play a significant role in the development of the system. Non‒functional requirements are spread across the entire system or within individual services and cannot be allocated to one specific product artifact (e.g., class, package, component).

This makes them more difficult to handle than functional requirements. The specifics of the system’s architecture, such as highly distributed services bring up additional difficulties.

The distinction between process and product requirements lays the foundation for the Work Breakdown Structure. The WBS, the related Integrated Master Plan (IMP) and Integrated Master Schedule (IMS), also focus on this separation.

The success of the project or program depends on defining both the product and the processes that support or implement the product.

When properly connected, the Requirements Taxonomy, the Work Breakdown Structure, the IMP and the IMS “foot and tied” to the Performance Measurement Baseline (PMB). This provides the traceability of the increasing maturity of the deliverables (vertical) and the physical percent complete of the work efforts (horizontal).

Step-by-Step for Capabilities

1. Determine what the system is supposed to do in terms of Scenarios or Use Cases. This is not a new approach at all. Alistair Cockburn introduced the notion of Use Cases long ago. But they went astray because doing good Use Cases requires understanding what capabilities the customer needs.
   
   • What is the business problem or mission to be accomplished?
   • How would you recognize that this problem has been solved or the mission was accomplished?
   • Measures of Effectiveness are the units needed to confirm this accomplishment.
2. Assemble these Capabilities into a functional architecture, showing how each capability supports the mission or business need. The process flow below is an example.
   
3.  Develop a maturity flow for each capability showing how the presence of this capability allows the business or the mission to do some work. 
   • This of course is simple agile - working software.
   • But agile is now coming to see that bottom up, response to the customer need requires a framework - a Programmatic Architecture to assure that the end is reached as planned.
   • This of course is Stephen Covey's Habit 2 Begin with the End in Mind, and is actually the Integrated Master Plan / Integrated Master Schedule paradigm of DOD 5000.02 procurement. Imagine that 5000.02 and agile on the same page. See Agile+EVM=Success for guidance here.

Here's a picture of the Capabilties Maturity Flow from a real project...

As each capability appears, the project can then start producing useful services - do something useful. But - and this is critical - the business or the mission MUST be capable of receiving this capability. It does no good to have capabilities that cannot be put to use. This is the purpose of this diagram. To show what capabilities are needed, in what order. This is a Top Down process, done by the business or mission owners.

Step-by-Step for Requirements

1. For each defined capability there needs to be a process and product requirements as shown on Page 7 of the slide deck.
2. Each requirement MUST flow from a needed capability. Each requirement needs a reason for being there. It needs a parent. It must fulfill some needed capability.
3. As an aside - all requirements are derived. This means all requirements are derived from the needed capabilities. In some circles this is not the paradigm, but in the complex, software intensive world of space flight and weapons systems - All Requirements Are Derived from the Mission Statement or the Concept of Operations (ConOps).
4. Thesetwo documents are usually missing in the IT world, with the resulting gap of not knowing WHY we're doing something.

Books PMs Should Read and Put To Use

There are several shelves of books about many topics in our house. Project Management and related topics are in the Office. The list of books below have specific attributes. They have not only been read, they have been marked up, reread, and used in practice. I have many books that are read, but not marked up. Other books that have been read, marked up, but not put into practice. And a hand full of books that have all three attributes plus one more. The contents of the book has changes how I do my job.

There are other books that I've read, marked up, and concluded the content was either not applicable to the problems I work on, or in a few cases complete nonsense. By this I mean the content is not only not appropriate to the problems I work, but if applied will result in disappointment.

So here's my must have list. This list is not populist in nature. By populist I mean a book or paper in which you CANNOT calculate some outcome. A populist book is something to read if you are just coming to the topic and want to discover the lay of the land.The books listed here are nit populist books

This list is in no particular order, other than they were taken off the shelf in this order. One other attribute is if I know or met the author.

General Project Management

These books are about the general concepts of managing projects, including agile projects. They are from authors who are or have recently benn project and program managers. This is important for the simple reason that who wants to read about something from someone that actually doesn't practice what is in the book. By the way, project management techniques that do not address agile approaches are not that useful anymore. As well, authors with 1996 style credentials have likely been passed by the current approaches as well.

• Making the Impossible Possible: Leading Extraordinary Performance, Kim Cameron and Marc Lavine - this is a case study about the management processes at Rocky Flats. Here's a quick summary of the concept of Positive Deviance and Heliotropic Abundance.
• Strategic Performance Management: Leveraging and Measuring your Intangible Value Drivers - Bernard Marr speaks to managing the business value from projects. There is lots of discussion around this, with lots of soft terms. This is a book that turns that populist view into a balance sheet view.
• The Handbook of Program Management, James T. Brown - I have met Dr. Brown. He is a former Program Manager at NASA. This is a book about program management. Program Management is about managing collections of projects. It is a hands on instructional advice type of book.
• The Radical Elements of Radical Success, Dan Ward - this is a book from a person practicing Program Management. Dan is an Air Force procurement officer, just back from theater. It's one of those books that causes me to think. Dan and I have had long discussions about the issues in government procurement. Here's a pre-defined search for Dan's work
• Project Management the Agile Way, John C, Goodpasture - I know John professionally, but have never been face to face. John and I share many common principles. This book is published by Management Concepts, which I do work for. MC publishes books, provides courses, and services for the Federal Government. John's book shows you how to deploy agile processes in the presence of the governance based environment. One where you are spending other peoples money and are accountable for success to external parties, like the Federal Government, a Board of Directors, and share holders.
• Flawless Execution, James D. Murphy - I cam across the After Burners at a PMI conference. The speakers are all ex-fighter pilots. There approach is logical, concise, and clear. The perfect paradigm for managing projects. This approach is not for everyone. It holds all members accountable for their actions, builds teams in ways not found in any other approach and is not one of those touchy feely styles.
• Catastrophe Disentanglement: Getting Software Projects Back on Track, E. M. Bennatan - there are many books about recovering projects. I'm reading another and writing a review. But this book is a check list book, with a step-by-step process of recovering projects.
• Beyond the Hype, Robert G. Eccles and Nitin Nohria - this is one of those must read for anyone interested in processes. Many if not most of the new ideas are actually old ideas in new cloths. Or worse, poor copies of old ideas promoted by self aggrandizing authors who know how to market. This book speaks to the problem of not knowing the past, so you
• Reinventing Project Management, Aaron J. Shenhar and Dav Dvir - this is a good starting point for PM in the IT environment. With field proven approaches, the books provides the step-by-step activities needed to manage projects in the Enterprise business domains.
• The 7 Habits of Highly Effective People, Stephen R. Covey - this is one of my populist books that has direct applicability to project management. Our method - Performance Based Management - contains most of the content of Covey's book is for Project Manager, including the Performance Based Management process we use on our programs.
• The Seven Secrets if How to Think Like A Rocket Scientist -Jim Longuski describes the processes needed to manage complex projects. It's a check list of items rarely thought about in the normal project management paradigm. This is done, because the domain Jim works in has projects that blow up and kill people of not properly managed.
• A Bee in a Cathedral - Joel Levy's book is here because using analogies in the project management world is very common. And most often the analogies are bogus. read this to see how to use analogies properly, since they are powerful tools to get points across, reveal understanding.

Leadership

There are so many books on leadership in the project management space. I'm simply burned out listening to all the voices on leadership from folks who speak and write about the topic, but who have not actually lead people under circumstances that require real leadership. Here's the books that you need from those people who have.

• How NASA Builds Teams - Pellerin led the Hubble Space Telescope program and speaks to how that worked and didn't work.
• Augustine's Laws - Norman was the leader of Martin Marietta here in Denver. These Laws are general business processes but for high tech firms.
• 8 Basic Habits if Exceptionally Powerful Lieutenants - this book and the next come from the Air University Press,  many in downloadable PDF.
• Commanding an Air Force Squadron in the Twenty-First Century - this is another survey books about how leaders learn to lead.

Systems Engineering

In the aerospace and defense business Program Management is a Systems Engineering paradigm. This is because the technical architecture of the program has to be matched with the programmatic architecture. The product that is being designed must also be built and shipped. This means the value stream flow of work needs an architecture as well. This is the purpose of the Integrated Master Plan (IMP) and the Systems Engineering Management Plan (SEMP)

These books are the basis for being a good project manager. Here's a list of specific topics, meaning more technical, needed for project success.

• System Engineering: Coping with Complexity, Richard Stevens - this is another handbook for designing, developing, and deploying systems.
• The Art of Systems Architecting, Second Editon, Mark W. Maier and Eberhardt Rechtin - everything is a system. No those silly descriptions of those populist systems where you can't design or calculate anything. But system we encounter in the real world. The air traffic control system, the banking system, the internet, the enterprise IT systems, manned space flight, building the 787. Systems like that. Rechtin is the father of the approaches used to build real systems. Yes, people are in these systems. And people are an important part, but in these types of systems, the components of the system are the basis of success.

Risk Management

There are lots of risk management books, articles, websites, and Bloggers out there. Here are the two you should own and read.

•  Risk Happens!: Managing Risk and Avoiding Failure in Business Projects, Mike Clayton - I've corresponded with Mike for several years. This is a great book for starting the process of risk management. It follows the steps defined in many locations - the best is the SEI Continuous Risk Management (CRM) model. There are simialr guides from DOD and DOE. PMI has a Risk Management Practice Guide, but like most things PMI they have their own way of doing things and rearrange the work processes.

As an aside determining risk drivers is a critical success factor for any Risk Management process. Typically risk management means making lists of risks, ranking them, and doing some kind of handling. This is not only naive, but it is wrong. You must determine which risks drive which undesirable outcomes of your project. Start with A Framework for Categorizing Key Drivers of Risk and Taxonomy-Based Risk Identification. I worked with one of the authors.

• Effective Risk Management: Some Keys to Success, Edmund Conrow - I've worked with Ed on a proposal and he turned me on to several concepts that I use daily. The first is the serious error in the PMBOK about multiplying the probability of occurrence with the ranked outcome to produce a risk score. This is not mathematically possible, since both are probability distirbution functions. The Method of Moments can help if you don't have a Monte Carlo Simulator. But if you don't buy Ed's book and read how to do Risk Management the right way.
• Effective Opportunity Management for Projects: Exploiting Positive Risk, David Hillson - I have met David, and spoken with him over the web. This is good book for addressing the opportunity concept. You should read Conrow's advice for mixing opportunity and risk before doing so yourself.   

Earned Value Management

There aren't really a lot of books about Earned Value Management. But there is a ton of misinformation about how EV is used, no used, applied, and misapplied. So these books and sources should be the starting point

• Performance Based Earned Value - Paul's book came out a few years ago, and the ideas have been with me ever since. The notion that Technical Performance Measures are part of Earned Value was present in the C/SCSC days, but was lost in 748-B and needs to come back. See Paul web site of the same name for good background.
• Earned Value Management Intent Guide - is the National Defense Industry Association's offical document for US Government Earned Value Management. This guide tells you how to comply with the 32 Guidelines of Earned Value Management. For most projects outside the government this is not applicable. But you should read this anyway, just to have an understanding of what the largest implementations look like.
• Earned Value Management Maturity Maturity Model - Ray's book provides a framework for assessing the processes needed to improve the maturity of your Earned Value Management implementation
• Earned Value Management site at the Office of Secretary of Defense, Performance Assessments and Root Cause Analysis.
• Earned Value Management at the NASA site has lots of background as well.

There are other sources of Earned Value, even in the agile world. But care is needed. To properly use Earned Value you must be able to do several things:

1. Maintain a baseline of the capabilties at least - the end outcomes of the project should not be changing
2. Know the total budget - the end budget and some notion of how this is spread over time.
3. Have some measure of physical percent complete and know how much physical percent complete you should be at when you measure it.

If you can't do these, you shoulded be considering Earned Value for your project

Statistics and Math

Project managers need to know somethings about probability and statistics. All the numbers in project management, cost, durations, head count, technical performance are random numbers. In the absence of knowing how to use statistics and probability, you're going to be late and over budget before you start.

• How to Lie With Statistics - every PM must have this book, must have read it, and must put it to use. When you have a conversation with anyone on the project, have this conversation in the context of this book.
• Then get Advanced Statistics Demystified or Statistics for Dummies and learn about sampling, variances, inferences, and other models of how your project is going to be working even if you don't know that is working that way.

 

Rings of Success

Olympic Rings featured in nearing everywhere these days. Those rings represent the 5  parts of the world which now are won over to Olympism and willing to accept healthy competition. While there is much talk about the projects of the Olympics, some difficulties and many successes, there are other rings that must be connected for project success:

1. Risk - all project elements have risk. Cost and Schedule risk are a starting point. These can be addressed through straight lists of risks, their probability of occurrence, impacts, costs to mitigation and residual impacts after mitigation. Most approaches to risk management assume an independent association between the risks. That is they are decoupled risks, existing on their own. Many times this is not the case. What is Program Controls? speaks to some of the complexity of integrating technical risk with programmatic risk and addressing the coupling issues that are ignored in the Monte Carlo Simulators of project duration and their resulting cost models.
2. Cost - the olde phrase time is money is tossed around without any real consideration for what that means. What is the relationship between time and money. Where is that relationship defined? What are the statistical attributes of that relationship and its impact on the probability of success for the project. This must be addressed through a cost modeling system. SCEA is one place to start.
3. Statement of Work - What Does DONE Look Like? The SOW is one place to look. Assuming it is written properly. But Statement of Objectives, Concept of Operations, Scenarios, Use Cases, and Performance Work Statement are also places to look. Without a clear and concise collection of deliverables
4. Technical Performance Measures - define what DONE looks like in units of measure meaningful to the decision makers. TPMs involve a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development based on current assessments of products lower in the system structure.
5. Work Breakdown Structure - describes the products or services to be delivered by the project, how they are decomposed and related. At the end of the tree of the WBS, is usually the Packages of Work that produce these outcomes. Without a properly formed WBS, you cannot determine what done looks like in any meaningful way. A typical mistake is to develop a WBS based on work types - design, code, test. This simply states the effort not the result.
6. Integrated Master Plan / Integrated Master Schedule - The IMP is the Plan for the delivery of a set of Capabilities from the project and describes the increasing maturity of these capabilities through Significant Accomplishments and their Accomplishment Criteria. The Integrated Master Schedule describes the sequence of work, usually in Work Packages needed to produce the deliverables for the Accomplishment Criteria.
7. Performance Measurement Baseline - brings the cost, schedule, and outcomes together in one place, puts this under change control, and measures progress to plan through assessments of Physical Percent Complete, and forecasts future performance (Estimate At Completion and Estimate to Complete) using Earned Value Management.

Putting This All Together Through the Tangible Evidence

Each of the Rings are represented by tangible evidence in the form of a document. I can hear the agilest now, no documents. Well sorry, in the domain we work, we need evidence that you do in fact have a SOW, a Risk Plan, an IMS, a cost basis of estimate, a Technical Performance Management plan and other items that show the project is planned properly, being managed properly, and measured properly.

Here's the top level connections...

What Is Program Controls?

There is a discussion on the LinkedIn #ProjectControls forum. It started out with a simple question of what does PERTMaster do? PERTMaster is one of many tools that provides results from a schedule, showing the probability of completing on or before a date and a confidence of that forecast, using Monte Carlo Simulation. The Monte Carlo method is used in assessing project schedules, but more importantly it is used to modeling a wide variety of physical processes. The term Monte Carlo was coined in 1940, but the real algorithmwas developed at the casino's in Monte Carlo to determine how much money to have on hand for a fair roulette wheel. This was the origin of 00 on the wheel.

Since then the notion of analyzing the performance of a schedule and its cost using random numbers for the duration and assembling a Histogram of the completion times to determine the probability of completing on or before a date has been baked into the government Integrated Master Schedule processes, through DI-MGMT-81650.

Let's start with a core problem statement

"Despite Using All the Metrics Commonly Employed to Measure Cost, Schedule, Performance and Program Risk, There are Still Too Many Surprises (Poorly Performing 
 or Failing Programs) Being Briefed “Real Time” to Army Senior Leadership"

Is a quote from John Higbee around the problem of using Monte Carlo and leaving it at that. There are several important understandings needed before the MCS tools can be used:

• Your schedule has to be credible. By credible I mean a schedule that is actually executable. By executable I mean a schedule that passes the smell test of how to Increase the Probability of Success.
• By credible it also means that the network of activities that make up the schedule have themselves credibility. One place to looks (actually several) is
  • GAO Schedule Assessment Guide: Best Practices for Project Schedules
  • Planning and Scheduling Excellence Guide
  • Integrated Master Plan / Integrated Master Schedule Preparation and Use Guide

Now with a credible schedule, comes the resource loading of that schedule and of course the Earned Value Management of the work to forecast future performance. This work because the probability distributions for the activity durations in the schedule - when connected in a well-formed network - can be modeled to produce a picture of the confidence of completing on or before a planned date. Along with comes cost models if the schedule is resource loaded and Other Direct Costs are added.

This approach is mandated for Earned Value Management programs through DI-MGMT-81650 and the latest integration of the Schedule Risk Assessment (SRA) in DI-MGMT-81466B. Both require Schedule Risk Assessment, which is usually done with Monte Carlo Simulation. Method of Moments can be used, but I don't know how to make that work. I have colleagues that do.

But Here's the Real Problem

Having a Monte Carlo Simulation of your schedule and let's say a possibly credible model of the costs - By The Way most MCS tools build models of cost from the resource loaded hours, not the material or ODCs.

First, the MCS is for duration (and maybe cost) "risk." The term "risk" means the variability of the durations of the work (and maybe their cost). But there are lots of other risks besides duration risks. Technical risks, operational risks, political risks, environmental risks. All of which can have unfavorable impacts on cost and schedule. So a credible model of the actual Probability of Success for a project needs to include not only the variances in the activity durations and their resulting costs - from labor hours and maybe ODCs, but also impacts on those hours from other risks.

This problem has several layers:

• What are the risks? Can you name them, analyze them, categorize them, come up with a handling plan for them, monitor them and essentially retire them.
• How are they related? Are they all independent? Meaning each risk lives alone in its own little world of probability of occurrence and probability of impact, with cost and schedule impacts?
• How do they drive the actual variances in cost and schedule?

BTW, there are many words flying around here, and no real independent dictionary. Many authors use the same words for different things, so care is needed, and I'll try to provide the definitions as I go.

An accessible book on the topic of risk management is Mike Clayton's Risk Happens! It's easy to read has all the facts right and can be put to work.

The next step is to determine where all this stuff lives on the program. From my experience it lives in the Systems Engineering domain. These types of discussions (other than the actually scheduling and cost) belong to the System Engineers. This is because the desgin and construction of the thing is directly connected to the ability to actually make the thing on-time, on-schedule. And to do that we need to have a credible model of not only the thing but how the thing is made, the cost, schedule, processes, tooling, facilities, testing, validation, etc. etc.

This is called PLANNING and the Plan is the Integrated Master Plan.

So What Does All This Mean and Who Actually Cares?

It means that simple Monte Carlo Simulation of the Integrated Master Schedule is absolutely necessary but it is far from sufficient to assess the Probability of Program Success (PoPS). What this means is we must discover how all these moving parts interact with each other, how they influence each other, how a small distruption in one place can cause big trouble some where else.

Risk Management is Systems Engineering and that means we a Managing a System. One of the best populist books (I know I'm not a recommender of populist books, but this one is a must read) is John Gall's Systems Bible. The best Gall quote is on the cover of my Master's Thesis

In a complex system, malfunction and even total nonfunction may not be detected for long periods, if ever.

This is the case for programmatic systems as well. Programmatic systems are everything about cost, schedule, technical performance and risk.

So the first thing to do in a Programmatic System is to determine the topology of the system. What are its pieces, how are they connected, what are the dependencies (or couplings), what is the strength of these couplings, and what are the dynamics of the resulting system.

These couplings can be  “interdependent” or “coupled” work activities and their probabilistic attributes, including risks associated with those “coupled” tasks. These form a model of the program – the probabilistic aspects of each task, cost, schedule, probability of technical performance; physical performance failure, as well as process failures and their impacts in the system or subsystem failure in any dimension. These couplings have several classes of dependency

1. A  binding dependency represents the dependency which is regarded as a constraint between two dependent tasks.
2. A  non-binding dependency represents the dependency which is  not regarded as a constraint between two dependent tasks.
3. A  critical binding dependency is a binding dependency between the tasks with  zero level slack.
4. A  non-critical binding dependency is a binding dependency which is not critical binding. Following critical binding dependencies along the tasks, a critical dependency path goes through the entire process hierarchies of a project from the first level to the last level. This path is different from a critical path because it is determined without considering time aspects. However, this path provides guidance for process improvements in the early stage of planning processes when detailed data for durations are not available or the probabilistic models fail to provide sufficient confidence intervals for decision making.

These are all assembled into a model of the program. A wonderful tool for modeling all these things that make up the Programmatic Architecture is a Design Struture Matrix (DSM). Here's a notion DSM of a project. This could be a real project, so let's use it. This is taken from the handbook we use for this approach.

I discovered DSM through Tyson Browning, while he was at Lockheed Martin Aero in DFW. There is a whole discipline for DSM. Search for Dr. Browning's work and example of how DSM is integrated into the Program Controls activities of Boeing, BMW, GE Aircraft Engines, and dozens of other firms who understand that Programmatic Architecture and the Program Controls discipline is Systems Engineering.

One irony is the 2012 DMS conference is titled "Managing Complexity by Modeling Dependencies

DSM is one tool that can be applied here. But DSM fits into a broader range of tools around System Dynamics. Since all the programmatic aspects are embedded in the broader system aspects of the program, we can start to see that simple cost and schedule modeling in the absence of dependencies, in the absence of risk events which can be called causal risks or casual events all need to be integrated with the Master Plan, the Master Schedule, the physical system taxonomy and topology, and of course all the categories of risk.

This let us start to answer the question - What is the Probability of Program Success?

Deadlines versus Must Finish On or Finish No Later Than

There are several (or at least two) ways to show that the schedule is pushing to the right of a planned completion date. FINISH NOT LATER THAN or MUST FINISH ON and a DEADLINE.

Here's the two ways in MSFT Project.

I prefer the DEADLINE for several reasons:

• The planned finish - the DEADLINE is always visible - and is described by the DEADLINE
• The actual finish is also always visible,in this case showing to the right of the planned finish.

With the DEADLINE approach the schedule is free (dynamic) to move during the Monte Carlo Simulation used for the Schedule Risk Assessment (DI-MGMT-81650).

The concept of a dynamic schedule, free to move left or right at all times is a critical success factor for any credible schedule.

How to be a Master Scheduler

The GAO Schedule Assessment Guide is out. This is a handbook for anyone looking for advice on how to construct credible schedules.

This may sound obvious, but it is not obvious, otherwise we'd have beautiful, credible, executable, assessable, and risk adjusted schedule for all the work we do.

There are ten best practices described in this document. All of them obvious in their titles, and much less obvious in their execution. While it may seem trite to say it, these ten best practices are about all there is to building good schedules. But for some reason we rarely see actually good schedules.

These 10 are (if you don't want to download and read the full 211 pages:

1. Capture all activities - this means for all the terminal nodes in the Work Breakdown Structure (WBS) there should be work activities that produce the products and services for that deliverable. These work activities should be contained in Work Packages, which are derived from the Accomplishment Criteria that support the Significant Accomplishments of the Program Event in the Integrated Master Plan (IMP). But that another topic
2. Sequence all the activities - time (and order) prevents everything from happening at once - Einstein. But the order of performance requires careful consideration, not just arranging the work to fit the availability of resources, funding, or whims of management.
3. Assign resources to all activities - this means a resource loaded schedule. A schedule, where the resources have been carefully considered and the work effort carefully considered as well.
4. Establish durations of all activities - start with the reference class approach.What can you learn from somewhere else on how long it will take to do this work. Use the 1, 2, 5 estimating technique. Use some technique, do not just make up the numbers - have a model of how these numbers came about. There is lots of materials for estimating, read them.
5. Verifying the schedule is traceable horizontally and vertically - it should start with the vertical traceability. As an IMP "jock" the vertical traceability defined the architecture of the program. Then the horizontal traceability come in.
6. Confirm the Critical Path is valid - any good scheduler knows what the critical path is, she doesn't need to have a software tool tell her. If you're flying to low earth orbit or beyond, the propulsion system had better be on the critical path. If not your schedule is not credible.
7. Ensure reasonable float - this number needs to be developed through a Monte Carlo Simulation IAW (in accordance with) DI-MGMT-81650.
8. Conduct a schedule risk analysis - is the inverse of the float analysis. Both are needed.
9. Update the schedule using actual progress and logic - answer the question how long are you willing to wait before you find out you're late? You'd better be updating - statusing - the schedule at half or less, because when you do discover you're late, you'll not have enough time to fix it.
10. Maintain a baseline - no baseline, no credibility to the forecasts of future performance. You can't manage a project without variances. Rubber banding the baseline is not program management, it's cooking the books.

Each chapter goes into details for each practice. Each chapter has a case study to show how to apply the principles of the practice.

Download this, read it, start to apply it. Even if you're schedules don't improve, now you'll know why?

Building the Perfect Schedule

I'm speaking at the Denver Professional Development Symposium in April of 2012. The topic is Building The Perfect Schedule. LinkedIn has a thread on schedule float and margin. There are several other threads around on scheduling, margin and risk.

The place to start for guidance in scheduling is NDIA (National Defense Industry Association) Planning and Scheduling Excellence Guide (PASEG).

The PMI briefing is based on the notion that a schedule can be built that is credible.

This of course with the description of DONE is units meaningful to the buyer. It's the buyer that says what done looks like. Capturing that description of DONE in sufficient fidelity is a critical success factor.

DONE is not described by the tasks performed in the execution of the  project. DONE is described by the tangible evidence from the Outcomes that fulfill the project's mission and vision in terms of a set of Capabilities that can be put to work by the buyer for their purposes.

When we see scheduling and the supporting planning as a mechanical process, then we'll end up with a typical schedule. A rats nest of tasks, linkages, dependencies. All failing to describe how we will recognize we're making progress to plan.

All successful projects must answer those 5 Immutable questions that must always be answered if the project has any probability of success.

No matter the project management or product development method, success comes from credible answers to those 5 Immutable Principles. In all cases the schedule support those answers. This means that the method used to schedule - from agile sticky notes to the 100,000 activity network - must also support those 5 immutable principles.

Deterministic Versus Probabilistic

The world is probabilistic. The world is driven by statistical - stochastic - processes. That does not mean the world is chaotic, uncontrolled, or even uncontrollable.

I drive my car on I-25 to Denver office when I'm in town. Out in the country side, the speed limit is 75MPH. No one of course is going 75. Here in Colorado, open road driving is 80 to 85. My car - a Fire  Engine Red Honda SI - moves right along at 85. I stay in the lane, but the steering inputs are small random movements. At the lowest level the inputs are stochastic. A random, non deterministic behavior. But I stay in the lane.

When we think of  a project's's cost and schedule, those are stochastic processes as well. They are random non-deterministic processes. By deterministic we mean "knowing the EXACT outcome. The underlying numbers are probabilistic, they are not deterministic. But that's doesn't mean they are chaotic. They are random wihtin the probability distribution driving their behaviour. The stochastic process driving their behaviour. This is lost many times in the conversation about "big anything up front." No credible project manager believes that the numbers - cost, schedule, and technical performance - are deteministic. To do so is basic nonsense. I know it happens, but that doesn't not make it nonsense.

So let's look at some definitions here.

Let's put this to work on a "notional" schedule." If we look at a schedule we SHOULD look at the probabilistic aspects not the deterministic aspects.

Each task durations are defined as "most likley" but have probabilitic durations as well. The shape of the probability distribution curve may vary for each "class" work, risk level, and other drivers of variance. The actual schedule looks like this.

If you consider your world deterministic without variance then you're late, over budget, and the thing you're building probably doesn't work - and most of all you probably don't even know it.