Herding Cats

Language of Program Success

When we speak of program management (or project management) "success" starts with speaking about DONE in units of measure meaningful to the program participants.

This language is a powerful source of clarity or confusion. Learning to speak clearly, concisely, and in words meaningful to all the participants is not only important it is critical. Here's the basis of starting to speak about the Probability of Program Success.

This means:

• What are the Planned Accomplishments for the program to keep it moving toward the end goal and at the same time measuring increasing maturity of the deliverables.
• Defining the criteria for each of these deliverables in units of "physical percent complete."

Define a set of nouns and verbs in phrases are used to speak about project success:

• Define Needed Capabilities
• Define Technical and Operational Requirements
• Performance Measurement Baseline
• Execute the Performance Measurement Baseline
• Continuous Risk Management

The conditions for success include:

• A good plan
• The ability to follow the plan
• A comprehensive, well documented, and accurate description of the program, its measures, and the plan

This last condition creates the needed credibility. It's having a plan and following the plan that is critical success factor.

    Stating that adapting to change rather than following the lan is the preferred approach simply reveals the navity of the planning process in any credible project management domain. The Plan is the strategy for success. As such it MUST adapt to change.

Can Earned Value Save Government Projects?

I'm attending, along with many 100's of others, the PMI EVM World 2009 conference in Naples Florida. (The proceedings should be available on the College of Performance Management's web site.) During a conversation with one of the attendees, he expressed his frustration with how government (I'll assume US Federal government) projects are poorly run at times.

The speaker works in international construction which certainly has a less than stellar track record of project success. Ignoring the built in inefficiencies of the largest organization on the planet for the moment, I asked "what is the alternative?" The answer I seemed to hear was "well I don't have one; my role is to point out the problem." His clearer answer was "earned value," but that of course is not the solution or the alternative. EV simply reports poor past performance in a timely manner and in units of measure of money, not time.

Pointing out the problem is the the easy part. We all recognize the problem; it's the answer that is elusive. One speaker at the conference stated government projects are different in many ways, some obvious some subtle. This struck a chord with me. They're different but in a way I had not thought of. They have do profit incentive (obvious actually) in the way a commercial IT project does (and ERP deployment), or a commercial constriction project (a high rise or shopping mall).

More important they're "emergent" in exactly the ways agile proponents suggest (but with less success). They are emergent on the defense side because the enemy is changing, the requirements are changing, the technology is changing. They are emerging on the civil side because of the political influences on the project. I've worked one city and one state project. If you think your commercial project has "politics," city projects have real politics!

It might be conjectured that in the absence of a profit motive the behavior of the system is inefficient. That is actually a tautology as well as a restatement of the obvious.

Leaving us with the unanswered question - "what's the alternative?"

What projects should the government - any government undertake? Can these projects be managed in ways different than commercial projects given they are spending the public's money? Is there any evidence that any alternative process than the ones being used today are more effective. By evidence I mean something beyond personal opinion?

I earn my living "working" government projects at NASA and DoD. Our answer is to make all attempts to improve the performance of those projects in what small ways we can.

Earned Value "can" and I emphasize "maybe can" provide actionable information to the Program Manager. But EV only is not a viable solution to increasing the Probability of Program Success (PoPS). http://www.afmc.af.mil/news/story.asp?id=123020393 is a short article on the broader tool set of approaches to increasing the PoPS. Earned value is one of those. But as my paper at EVM World suggests, CPI/SPI are driving in the rearview mirror using past months data with data measures in Dollars. The units of measure for the evidence produced by the actions and documents in this post are part of the attempts to increase the PoPS. Look on Google for the PoPS (Probability of Program Success Air Force Army Navy) concept to see if it adds light to the conversation. I'll post the EVM World paper next week.

The real issue at hand is how to turn all our opinions into some type of actionable outcome, with evidence that those outcomes are beneficial to the stakeholders in some unit of measure meaningful to them? Politics aside, For DoD programs the stakeholder is the war fighter. As a former combatant in Vietnam, I might suggest the units of measure are defined by them, not the non–combatants attending conferences in Florida.

Questions That Need Answers

The second day of the PMI Earned Value Management World 2009 conference produced some thoughts. There were many presentations, some good, some better. But during one presentation it came to me that all the "whining" about how poorly projects are managed rarely comes with a causal analysis. I'd call this the "Standish Report Syndrome.

Here's an approach I've started taking:

• If you come to the conversation with some "whining" observation, what's your alternative?
• What changes are suggested, and are those changes credible enough to improve the situation?
• Of you have no alternative, or don't have a credible improvement suggestion, you're just whining.

So here's an approach to the situation. Ask the following questions of the troubled project and look for the tangible evidence of the existence of the answers;

Question	Evidence of Answer
How will this system you're trying to build fulfill its mission?	Concept of Operations (ConOps)
What systems and subsystems need to be constructed to fulfill the mission?	Work Breakdown Structure (WBS)
What is each of the deliverables in the WBS going to cost when we're done?	BCWS spread against the WBS
When will you be finished with each of the deliverables?	Integrated Master Schedule (IMS)
What is the evidence that the program is maturity according to the plan?	Integrated Master Plan (IMP)
What are impediments to progress and how will they be retired or mitigated when they occur?	Risk Management Plan (RMP)
How will all the components of the system come together to produce the benefits to the mission?	Systems Engineering Management Plan (SEMP)

 

Without the answers to these questions, the project is likely headed for the ditch and you don't know it.

This doesn't mean you need a tome for each of these "documents." Sticky notes on the wall just as well for small self contained project as the 500 page ConOps for flying to Mars does for NASA. Same for all the other "evidentiary materials." But someone has the think through the answers to these questions, and convey these answers to the project participants, including the customer – be it NASA or your garden club president.

Self Inflicted Wounds

I'm speaking and attending the EVM World 2009 conference in Florida. This is a gathering of 500 or so Earned Value and Cost Estimating specialist discussing the theory and practice of Earned Value in a variety of domains. These domains are dominated by Department of Defense and large construction.
    There is not an IT manager within a 1,000 miles. Even though SOX, CobiT, and ITIL all call for "performance measurement" of projects.
    While listening to speakers - some from the Office of the Secretary of Defense (OSD), who buys and builds billions of dollars of software - some from other enterprises - it's clear that the core failure cause of IT projects starts with the following:

• No description of the product being built - No WBS
• No predefination of expected performance of the project team - No Performance Measurment Baseline
• No predefined units of measure of success - No meaningful measures for the customer to discover if progress is being made

One of the core principles of agile software development is the production of working software. This is a measure of physical percent complete - If and Only If (IIF) there is an understanding of what "done" looks like.
    In the absence of this information - physical percent complete and the Estimate At Complete - the project becomes a death march at worse and a "level of effort" at best.

Mind Mapping and the Work Breakdown Structure

Gantt Head has an article about the use of Mind Mapping (free login required) in project management. Mind Mapping is a powerful paradigm for capturing and organizing information that is hierarchical in nature. Projects are primarily hierarchical, starting with the Work Breakdown Structure (WBS).
    The WBS is a fundamental tool for projects. It describes the work to be done in terms of deliverables. Effective January 2009, the seminal WBS document, the MIL-HDBK-881A became a standard MIL-STD-881, mandating a WBS for Department of Defense programs.
    881A makes a clear and concise statement...

The Program WBS provides a framework for specifying program objectives. It defines the program in terms of hierarchically related, product-oriented elements

The word PRODUCT is critical here. No functions - design, code, test, no level of effort - manage design, code, or test. PRODUCT.
    The examples of the Mind Mapping all violate the mandatory principles of 881A and the suggestions in PMI's Practice Guide for Work Breakdown Structures.
    Now I know WBS's are the most exciting topic in the world of project management, especially IT project management. But in large construction and Aerospace and Defense, we live (and die) by the WBS. Not only is it mandated, but the WBS is a contract document. The WBS tells the buyer (the government), what the selling (the contractor) is going to do for the money.
    In both cases (DoD and PMI) the elements of the WBS are the products that will be produced by the project or program. These products start at the bottom of the WBS and roll up to the completed System. This system can be a spacecraft, a wheeled vehicle, a suite of avionics products, a working telecommunications systems. But they are always products, never the functions that create those products.
    There are some "thought leaders" in the PM Domain, with large resumes that will claim the WBS has more flexibility than that. Don't believe them for a second. The DCMA (Defense Contract Management Agency) would toss them out into the parking lot after the audit.
    Here's what the WBS does

It describes in clear and concise terms - through the WBS structure and the associated WBS dictionary - what DONE looks like.

    Done is not the design, the coding, or the testing. Done is the product. The subsystems and systems that make up the product. The customer did not buy "design." Design is absolutely necessary for the product to appear. But the customer bought the product, not the process that produces the product.
    This is annoying in many ways:

• You have to sit down and figure out what the deliverables actual are
• You have to structure these in some well formed "tree"

Mind Mapping does this. Use it to build your WBS. Ask for each node in the tree - "is this a product or a part of a product?" If not, keep trying to build the WBS, because you don't have one yet.

Using Earned Value to Manage the Program

There is a discussion on LinkedIn Earned Value Management group about how to use EV to manage a program in the enterprise IT domain. The domain of course is the Federal Government and all the reporting around the growing failure rate. This is one of those non-trivial topics that has been ongoing for decades. 

The concept that Earned Value is the means of managing a program has several issues, not the least the of which EV uses past performance, that is some time 45 days old. The second issue is that EV uses Money as its unit of measure. The Schedule Variance (SV) and the Schedule Performance Index (SPI) is stated in units of Money, and translated into Time in a 1:1 relationship. This is course is not the case in practice.

But one point of view is to look at why programs get in trouble from the Earned Value paradigm independent of these anomalies:

• Inattention to budgetary responsibilities
• Work authorizations that are not always followed
• Issues with Budget and data reconciliation
• Lack of an integrated management system
• Baseline fluctuations and frequent replanning
• Current period and retroactive changes
• Improper use of management reserve
• EV techniques that do not reflect actual performance
• Lack of predictive variance analysis
• Untimely and unrealistic Latest Revised Estimates (LRE)
• Progress not monitored in a regular and consistent manner
• Lack of vertical and horizontal traceability cost and schedule data for corrective action
• Lack of internal surveillance and controls
• Managerial actions not demonstrated using Earned Value

For any or all of these issues, the source of the program failure can be traced to Earned Value. Actually the failure of Program Planning and Controls processes.

We All Know Multitasking Is Bad - Now What?

 There's been a rash of post around the notion that Multi-Tasking is bad on projects. Nice restating the obvious. Now what can be done about this?

The first step is the acknowledge there is a problem, and search for a solution. There are actually very few solutions. The first solution is to staff the project with the right people at the right time so multi-tasking is no longer an issue.

This sounds naive, but it's not. If you don't have the right labor spreads right for the work to be done, you're in trouble before you start. But there are some activities that can be done to get the project off on the right foot:

• Resource allocate across work packages to minimize the multitasking assignments. Ruthlessly drive to this goal. Without a clean set of assignments, the project is an unaddressed risk. DO NOT, I repeat DO NOT let MSFT Project resource level your project. Once you do that, you're turned over the role of "manager" to a machine - an all round bad idea.

• Serialize work so multitasking does not take place. Serialization also creates on environment where 100% work is complete before new work starts along that work stream.
• This serialization has another benefit - the down stream work is always consuming 100% complete raw materials
• Use a resource allocation view in your project management tool to smooth the assignment of resources.

When reading the multitasking is bad, ask "and what is your suggestion for correcting it." No suggestion? - move on, the speaker hasn't had to solve the problem no a non-trivial project.

The Project Management Team

Craig Brown described his response to questions about technical teams and the management of technical teams. This got me thinking about how I would respond to the same questions from the point of view of Project or Program Management.
    First let me say that managing the people side of projects and programs is not my strong suit. There are people in our firm that are much better than me when it comes to the "softer side." I know this and surround myself with people who either share my limit and focus on the technical side of management or people who have strong people skills and enjoy sorting out all the "softer" issues.
    I converted the questions into Program Management and Project Management questions. Remember Program Management is the management of Project Managers, not projects.
Q1: How do you select the best possible candidate for the project management team?
Q2: How do you ensure each member of the project management team is motivated?
Q3: What do you do when the project is getting behind schedule or over budget?
Q4: How do you pick the methodology for managing the project?
Q5: What  is the best advice to give a project manager?

College of Performance Management

    The Project Management Institute's College of Performance Management has a periodic newsletter The Measurement News. This month's issue is has several articles about PMBOK and its Earned Value content.
    The To Complete Performance Index (TCPI) is very useful for IT type projects that are keeping track of the performance the work being done using Earned Value at the Work Package level. TCPI shows how much better you need to performance to come in on time.
    Remember the notion of Velocity in agile makes a huge assumption - the work is time boxed around a constant work effort - a fixed number of people and a fixed number of productive ours per iteration. The dependent variable is the outcomes from that time boxed, fixed effort.
    If you project is anything other than time boxed (fixed iteration durations) and fixed work effort (fixed number of people on the team doing fungible work) then velocity is pretty much meaningless. You'll need to have "real earned value" in some way, where Physical Percent Complete is measured against the Planned Percent Complete for a defined period of performance.

Specifications

    For all of us looking for specifications and documents that are usually buried in some web site or worse, available for a price, there's a source - EverySpec.
    Some things that were only available at IHS can be found there. The mythical ANSI/EIA-748-B is not.

Project Management Blogs

    I subscribe to a 100 or so blogs through Google Reader. There are a few I read all the time. One is Eight to Late, by Kailash Await. I too had a another life in physics - doing experimental work on deep inelastic scattering and writing Kalman filters to grab signals from a 100db below the noise, which turned into "going to the dark side, in the  radar / sonar target identification business."

    K as he calls himself has a great post titled, "The Role of Cognitive Biases in Project Failure." The post itself is a must read. But more importantly is the other content of the Blog. It's the example of what project management should be like in all domains, not just defense.
    There are some other Blogs in the same class:

• Paul Ritchie's Crossbery Blog
• Craig Brown's Better Projects
• Pat Weaver's MoasicProjects
• Ron Rosenhead's

There are many other Blog's worth the effort to follow. But these 4 provide top notch content, with thoughtful ideas, and absolutely credible opinions. When I figure out how to "share" my entire list from Google Reader, I post them here.

Why I Don't Like Stage Gates

The term "stage gate," is popular in manufacturing and many project management discussions. This term along with "Milestone" is one of those vague references meant to indicate progress.

• We made it through the Design Stage Gate
• We passed the first article tested milestone

The actual notion of a Milestone, is a rock on the side of the road showing how many miles the traveler is from Rome. This "rock" assures the traveler that they are moving along the right path. Distance traveled or they're on the right road. The problem of course is the units of measure of the Milestone is usually not explicitly stated on the Milestone - the Rock kind or the Project Management kind.
    The Stage Gate process is similar, with a "gate" being passed through without explicit description of what happens before the gate and after the gate.
    A much better approach is to define what "done" looks like in units of measure meaningful to the stakeholder at some specific point in time for the project:

• At the Preliminary Design Review we will have 80% of the Bill of Materials identified in the Supply Chain Management system from our top 90% qualified suppliers.
• With the completion of the first Engineering Unit, we have test the vibration and shock limits in an actual working environment.

These types of descriptions describe several things Milestones and Stage Gates can never describe:

• The planned maturity of the product or service
• The units of measure for measuring this maturity
• The technical performance measure boundaries for these values - Upper and Lower boundaries

    These measures speak to physical percent complete of the product or service. Milestones or Stage Gates do not. Try to avoid Milestones and Stage Gates if at all possible. Instead speak in "units of measurable increasing maturity, the boundaries of these measures, and the probabilistic behavior of these measures.

You Know You're Managing a Project When - V2

    When some say we're doing agile project management, or "our software development method includes project management," ask the following to determine if you're actually doing project management

• A project charter has been developed - do we know what done looks like, why we're doing this project, and the everyone on the project shares this understanding in units of measure meaningful to their particular point of view?
• Requirements are collected and managed - can we capture in some form, what are the deliverables of the project? Along with all the annoying little requirements to make these bigger pieces appear to the stakeholders?
• The scope of work is defined and controlled - is there some process for bounding the scope of the project? This means keeping the scope changes within the ability of the team to respond and forecast what the cost and schedule impacts will be?
• A Work Breakdown Structure (WBS) has been created in the spirit of MIL-STD-881A - 881A is the Gold Standard for building a WBS. The core concept is the have Product Deliverables ONLY, no functional activities. Things like Design, Code, Test are NOT in the WBS.
• A schedule has been developed and its contents controlled - some form of a sequence of work needed to produce the deliverables. Remember the purpose of time. It keeps everything from happening at once.
• There are estimates for cost and schedule and these estimates are managed - some way to bound the final cost and delivery dates.
• Quality of the outcomes is planned and managed - a description before the work starts of what "good" looks like.
• There is a risk management process applied to the project - Risk Management is how adults manage projects. No risk management, no project management. Hope is the only strategy at that point.
• There is a communications processes applied to the project - keeping people in the loop is the glue that holds the project together.
• The stakeholder's expectations are managed - if the stakeholders don't agree what done looks like, then the project is lost before it starts.
• The project team is managed - you don't always have to be happy. But have to be willing to continue to work until you are done. Making the team happy is not the primary role of the project. That self actualization process is outside the project. But the team members need to work for the stakeholders in a productive manner.
• The project's performance is monitored and reported, and this information is used to control the project - velocity is of no value if you don't know the distance.Earned Value is a core process in many domains. Velocity is not the same. It's close. It's useful. But it's not the same as Earned Value, not matter how many times someone in the agile community says it is.
• The project is closed out - getting people off the payroll is part of close out. They can move to the next project or to support. But projects have finite ends.

    The terms "control" and "manage" can be defined to meet the needs of the reader. For agilest these are terms that elicit knee jerk reactions. But in someway, somehow, the project needs management and control - pick your poison.
    If your project does not "do" all of these items in some recognizable way - then you're not managing the project, the project is managing you.

DoD Acquisition Reform Focused on Earned Value

Paul Solomon's web site is a wealth of "breaking news" on topics on interest to Program Planning and Controls and Earned Value. At the site is testimony by the Deputy Desecrate of Defense, Lynn on Acquisition Reform. While there is some conflicting ideas in the testimony, the concept of performance measurement using EV is starting to spread to domains outside DoD.
    The ARRA (American Recovery and Reinvestment Act) requires reporting processes starts with the White Memo. Earned Value for capital projects is obvious. For service contracts its probablematic, since there is usually no clear measure of physical percent complete. For money flowing to "projects" some form of "done," progress toward done, and the units of measure meanigful to the stakeholder makes Earned Value a natural method for measuring progress to plan.

Undercorrelated WBS Elements Means Understated Cost Risk

The current issue of the National Estimator, The Society of Cost Estimating and Analysis, monthly magazine has an eye opening article titled "Correlating Work Breakdown Structure Elements."

A Key aspect of cost risk is correlation between work breakdown structure (WBS) elements.
...

For example .. with a 30 element WBS has an assumed zero (0) correlation between all elements, but the actual correlation between WBS elements is 0.2, the total cost risk will be underestimated by 60%

This should be a frightening fact for all of us who assume no correlation, or worse have no recognition that such correlations are important.
    So when the question of "why are projects late and over budget," is asked this is one of the answers, along with a myriad of other answers:

• No understanding of the needed schedule margin, derived from Monte Carlo Simulations
• No understanding of how the increasing maturity of the products or services are being measured in units meaningful to the stakeholders
• No management of the inherent risks of the program
• Insufficient partitioning of work efforts to reduce the coupling between work durations - "How Long Are You Willing to Wait Before You Find Out You're Late?"
• The inability to determine the capacity for work, so all work estimates are underestimated.

• And the final killer result - multitasking creates hidden correlation between WBS elements.

The last element alone, should be reason enough to start to understand the correlations, and remove the multitasking. After that the real correlations, need to be ferreted out and placed in the cost model.

Increasing the Probability of Success

    Hal Macomber's blog Reforming Project Management - asked the question "does reliability matter?" Being reliable is necessary but not sufficient for project success.
    The defense industry learned to classify those "necessary but not sufficient" attributes into a larger model. The model is:

The Probability of Success P(s)

    The Probability of Success has several components formed into a Windshield chart, rather than a Dashboard. Cars are best driven by looking out the windshield, rather than looking at the Dashboard. P(s) has the following variables:

1. Requirements
2. Resources
3. Execution
4. Fit with Vision
5. Advocacy

    These variables can be arranged in a variety of ways to calculate (notionally) the P(s). This arrangement is not important here. These variables have sub-variables, but that too is not the point.
    What is important is to understand that the "reliability" of the work effort promises - showing up on time, on schedule, with the agreed upon outcome - lives in the Execution variable of the P(s) calculation, as one of many variables that increase the probability of success. Failing to recognize that, creates the illusion that being reliable is what it's all about.
    You can reliably deliver the wrong outcome. You can reliably produce products that don't fit the vision of the customer - the needed capabilities in the example of the DoD.

    A good detailed example can be found at https://acc.dau.mil/CommunityBrowser.aspx?id=245260

This link will ask you to proceed in the presence of a certificate error. You can do so, since this is a US DoD site.

    Reliability is a necessary but not sufficient attribute of project success.

Risk Management

There have been several posted recently about "management of risk." Many times risk gets mixed up with issues.
    The best place to start to understand how risk management is performed, is to look at how it is performed in the world where risk is always present and the consequences of not "handling" risk are severe.
    The US Department of Defense Acquisition Technology and Logistics Community of Practice site has a Risk Management group. There are many other communities at DAU, so browse around to see what else is interesting.
    This link may ask you to confirm the certification acceptance. Most DoD sites do this. So answer "yes" and proceed with confidence.