Herding Cats

Quote of the Day - Billion Here, Billion There

A billion here, a billion there, and pretty soon you're talking about real money. - Everett Dirksen

When we speak about "program controls," there are lots of domains and context where this is a mandatory role. Some are larger than others. Some are more formal than others. Some have special needs and others are just straight forward Program Planning and Controls using Earned Value straight out of ANSI-748B.

Here's a program we're working on getting on. Look carefully at the total award value for the engineering, technical, and programmatic support services. Programmatic Support means "program controls." That means "project" management, and the process around "managing" projects.

Yep, that number on the right is $19 Billion, as in a $Billion with a $B. In this domain there are all kinds of development and engineering services. Including IT development, including Agile development in IT. So when it is suggested by some thought leaders that EV is "hog wash," ini the IT world, ask yourself the question - who's looking after the $B's? A self organizing team of very qualified software developers?

Uh, probably not.

Part of that number on the right goes to

Yep, good old software development processes. This is where agile is most needed. When we get into serious money as Senator Dirkson mentions above, we need to best of both worlds. Rapid development to verify the requirements are right, put the software to work to verify it can earn it's keep. And control mechanisms that can forecast future performance in units meaningful to the decision makers. Those units are of course Dollars.

Team Accountability? Really How Does That Work?

We use a business management process from After Burner. This firm is lead by James Murphy, a former Air Force fighter pilot. This months newsletter speaks to Accountability and members of the team being accountable.

Personal Accountability is part of the team's success. The Afterburner method has 4 simple processes:

1. Plan
2. Brief
3. Execute
4. Debrief

This may sound familiar in the project management world, except for the "brief" and "debrief" parts.

The planning part is of special interest to me. The Afterburner^© Planning consists of:

1. Determine the mission objective
2. Identify the threats
3. Identify available and required resources
4. Evaulate Lessons Learned
5. Develop Course of Action
6. Plan for Contingencies

So What's the Point?

When I hear the notion of "collective" ownership in an agile team principles of "collective ownership." In principle this is a bad idea for all kinds of reasons, not the least of which there is no single point of accountability.

 

Plan First, Then Schedule

 

These two words should be tattooed on your wrist. If we don’t have a Plan, our schedule is not credible. Plans are not Schedules. And Schedules are not Plans.

A Plan is the Strategy for the successful delivery of the project. Plans state “what” is to be done (programmatically what, not technically what). Schedules state “how” it is to be done – programmatically how it is to be done.

While this may seem subtle or maybe not even useful, it is critically important for several reasons:

• The plan shows how the project produces increasing value and increasing maturity of the products.
• It’s is the “road map” from the beginning to end, INDEPENDENT from the actual durations of the work.
• The Plan speaks to What we are doing.
• The schedule is the “driving instructions” for the vehicles on the roads, following the map.
• The execution of the schedule is the actual “driving” of the vehicle by the driver along with the passengers.

All three are needed, no one can be missing, all three interact with each other.

Plans are strategies. They tell us the flow of the project in terms of deliverables and the increasing value and maturity of those deliverables. I’m going to introduce some new words here that we’ll use later. They are just words for now, so their exact meaning is not important.

This is similar to the Value Stream Map found in Lean Six Sigma.

The “map” is the flow of the Significant Accomplishments in the project or program. For these Accomplishments, there are a set of Criteria that define the “exit conditions” for the underlying work. Defining these criteria BEFORE defining the details of the work is the basis of the Planning process.

This is a top down-first approach . It is NOT a Top Down only approach, just the 1st step in the process. But with the Accomplishments and the Criteria defined, there is a notional view of what “done” looks like. Measureable in some units meaningful to the project management team and the stakeholders in the project.

This focus on the definition of “done” is important for several reasons:

1. “Done” is a measure of 100% done, not partially done, not almost done. But DONE. This is the concept of “starting with the end in mind,” popularized by Covey.
2. Along the way to DONE, there are measures of “getting to done” that are “mini-dones.” These “maturity assessment points” are the way to measure physical percent complete in terms of the product maturity rather than the consumption of resources or passage of time.

Now that we know about the existence of a Plan, what is the Schedule? Why is it different from a Plan?

The Schedule shows the work needed to produce the “deliverables” in the Plan. This sounds like a tautology – a statement of the obvious. But there’s more to it than that.

This work is ONLY the work needed to cause the “exit criteria” to appear of each individual definition of the criteria for named Accomplishment. In a previous slide we mentioned the definition of the Accomplishments come first. With these definitions – and most importantly the order in which these Accomplishments must be accomplished.

Another Presentation on Agile

Dennis Stevens gave a presentation on Agile Development at a PMI Atlanta Chapter Agile Local Interest Group, titled Agile Fundamentals. The early part of the presentation addresses some of the perceived failure modes of traditional project management needed to produce success. I suspect these are anecdotal experiences from the comments exchange we've had.

I have no doubt Dennis has experienced Bad Project Management. But it seems he converts that bad project manager experience into bad project management processes, in support of the suggested process replacement - Agile Project Management.

I'm worried that PMI is confusing software development practices - and Scrum is the best for some software domains - with existing Best Project Management practices. I was going to compose a long post, but instead built a briefing deck I can use in other places where I encounter this blending of "bad personal experiences," "red herrings from the agile community," and possible confusion between software development practices and the fundamentals of managing those performing the software development practices.

My motivation for this approach is:

• Agile software development is entering the DoD IT Acquisition domain. There are several boundaries in this domain, not the least of which is the mandated Earned Value Management for all programs greater than $20M - which in the DoD world is chump change.
• The DoD 5000.02 procurement process defines iterative and incremental steps for defining, developing, deploying, operating and retiring systems. This process is highly developed and mature. And at the same time burdensome and over stated. Dan Ward speaks to this directly. Dan and I share ideas. Dan is not your typical evangelist for change. Dan is an Air Force Lt. Col with "heavy cred" as we say in the acquisition community, with "hands on" program management experience.
• Dan's F.I.S.T. approach is a nice fit "agile" concepts in procurement of complex systems. But critical to the discussion is ow to build on past processes and principles using new processes and principles. This is the old "throwing the baby out with the bath water problem."
• This Baby and Bath approach has started to enter into my domain - DoD, DOE, DHS program management and I want to set to story straight about the current project management processes and how they can be augmented instead of replaced.

The core motivation here is to separate good project management processes from Bad Project Managers. Only then can we look for ways to improve these processes with Agile.

Keep the baby, toss the dirty bath water. This combination of good PM practices with Agile Software Development is the "real" PM 2.0.

Dennis stevens response

View more presentations from Glen Alleman.

So after you look at the response ask yourself this:

• What if you didn't define everything up front, lock it down, and insist on sticking to that specification? What if you defined the end to end outcome of the project in operational terms, in a set of capabilities for the user. Then defined the progressive details of those capabilities and operational concepts in Rolling Waves?What if there were simple ways to make changes to the Plan. Changes that took into account impacts on cost and schedule - it is someone else's money you're spending right? They have the money and you want it?

If you did that you'd be following DoD 5000.02

• What if you didn't just throw projects over the wall to developers but had periodic engagements with all the participants? Maybe not daily, maybe not in the same room all the time, but at least once a month on a large multi-year development project. During those meetings you talked about the technical aspects of the project as well as the cost and schedule performance. You talked about anything that was not "green" in the Measures of Effectiveness (MoE) - this is how the customer sees the project. And you talked about Measures of Performance (MoP) - this is how the developers see the project.

If you did you'd be following DoD 5000.02

• Since you work on teams inside your firm for external customers - that's how the business was organized before you got there, or even on teams for internal customers, what if someone in your firm was accountable for allocating the resources? What if your firm didn't have enough people to dedicate them 100% to your project? What if your suggested method of staffing did not meet the business model - the labor assignment of 100% created too high a cost for the projected revenue? Would you yell and stomp your feet that they are not following agile principles? Or would you put on your business manager hat and come to realize that somethings are just out of your control for now and when the firm grows some more and you can make a business case for dedicated teams, we'll all have to just deal with the situation until then, or maybe forever if we don't improve or sunk cost ratios.

Try putting on your business manager hat. You're handed a budget for someone. You - as the business manager - are tasked with "getting things done." You've simply have got to multi-task some of the staff. Deal with it in the best way possible. Think about practice over principle - with the business hat on.

• You got a Plan - a business Plan. That Plan calls for products or services to be built in software. The Plan is baselined in some meeting between you; and those asking for the products and services in the Plan. They come back 3 months later after a tour of the world with new information that makes changes to the Plan. You - the software development manager say "nope I'm not changing the Plan, you agreed 3 months ago and I'm sticking with it."

Yea, you're doing a really good job of managing in the presence of uncertainty. Nice try, change the Plan, update the schedule, reallocate your resources, make modifications to the processes and eventually changes to the source code.

• Your QA team or your developers using Junit comes to you and says "boss there's a problem." You say "well guys I don't know what to tell you, our problem detection and resolution process says you've got to wait until the management team meets before you can go any further." Only Managers can define defects and propose a solution here."

Does any of this about top down defects make any sense at all? Probably not. It might even be called Nonsense.

• While your going along developing product, your team comes across some really good ways of "making things better." They're simple, they're cheap, and they probably will actually work. You say to them "fellas we're going make corrections to our boneheaded work process when we're done with the project, so just hang on to those until next year."

Does the term NONSENSE ring a bell?

While there are places where these little examples likely take place, they are of course complete nonsense. They are Bad Management. Fix the Bad Management first...

Stop doing stupid things on purpose

What is a Plan?

Brad Egeland has a nice post about planning. He mentions three benefits of Planning:

1. Reduces uncertainty
2. Increases understanding
3. Improves efficiency

This is a good starting point, Planning is Good. But why is Planning good? First what is a Plan?

A Plan is a strategy for the successful completion of the project

The plan describes the "path" to the end. The Plan describes what "done" looks like. Not only at the end, but along the way to "done." The Plan describes the increasing maturity of the deliverables of the project. The units of measure for the maturity and when a specific value for this unit of measure should be present. This means the plan has a sequence of increasing maturity.

The Plan is NOT a Schedule, which is time based. The Plan is seqeunce based, relationship based, and maturity based. The Schedule tells us "when" this maturity should appear.

Here's an example of a "Plan"

The Plan shows the increasing maturity of the deliverables, moving from left to right. How at each point along the path to "done," the dependencies are related, and how they support the final deliverable.

At the points of assessment, there needs to be a set of assesment criteria to assure that the planned maturity has been reached. These measures of progress to plan can be Technical Performance Measures, quantities of material, or some other type of measure.

Turning the Plan into a Schedule

With the Plan established, at least in the first version - since Plans do change, we need to Schedule the work to cause the deliverables (intermediate and final) to appear.

In this diagram the "Plan" part is represented in the Program Events - the capabilties that are "planned" to be present at each assessment point in the project. The "Accomplishments" needed to provide these capabilities are defined in the "Plan." The "Criteria" for each of these Accomplishments - the measure of compliance with the planned maturity - are also part of the "Plan."

With these elements - arranged in a hierarchical structure, the "Work Packages," that perform the work are developed. The sequence of the Work Packages are the top level of the Schedule. The work inside the Work Package can be sequenced as well. But the starting point for the schedule is the sequence of the "Work Packages"

Once the "Work Packages" are sequenced, a Program Master Schedule is the result.

This is the master level sequence of work that is not Time Phased - a schedule. Below the Program  Master Schedule are Intermediate and Detailed schedule. Each defining increasing details of the work, down to the Work Packages and the Tasks or activities within the Work Packages.

An Intermediate Level schedule might look like this.

At the Intermediate Level, the dependencies that connect the major deliverables - the increasing maturity deliverables - are show. Below this are more levels of detail. The Master, Intermediate, and Detailed schedules are actually ONE file (.mpp in MSFT Project).

There are tools like, Chronicle Graphics and Kidasa Milestones that can be used to make nice Program Master Schedules from the detailed and intermediate MSFT Project schedules.

But in the end all this startes with a PLAN - a Strategy for success and the measurement points along the road to that success.

Domain Dependent Concepts

I'm surprised some times by the differences in domains and contexts in domains and some general notions of project management.My suprise is how simple concepts in one domain fail to make into other domains.

One is Mosaic Project's White on Duration Estimating.

Apart from very small short term projects (3 to 6 months and under $200,000) attempting to directly integrate the cost estimating process with the scheduling process in fraught with difficulty. Even on these smaller projects, direct linking is not really feasible unless the primary source of all costs is staff directly employed on the work.

This type of planning and controls is simply not allowed in programs subject to Earned Value for governments, including the Australian government.

Without connecting cost and schedule as well a technical performance measures, probabilistic risk and cost and scheduled margins you cannot build a credible Performance Measurement Baseline( PMB).

The concept in the last sentence cannot be true as well. The PMB includes Labor and Non labor Cost Element Relationships (CER).

See the current GAO Cost Estimating and Assessment Guide for how to integrate cost and schedule.

Without the integration of cost and schedule, the program cannot be managed to success.

Here's why:

• Cost drives schedule
• Schedule drives cost
• Subcontractors on FFP (Firm Fixed Prices) still have cost and schedule spreads inside their FFP subcontract. They have cost and schedule coupling.

It's this coupling between cost and schedule, along with the compliance with Technical Performance Measures (TPMS), that is both the source of problems and the source of transparency in program controls.

Here's the Pictures

The Peformance Measurement Baseline (PMB) is composed of the following elements:

• IMP/IMS - Integrated Master Plan / Integrated Master Schedule
• Cost baseline
• Statement of Work (numbers tracing the deliverables to the work delivering the work)
• Technical Perfomance Measures
• Work Breakdown Structure (the products and process structure that produce the products)

The connections between Cost, Schedule, and Techncial Performance drive the needs for "margins."

 

So the Mosaic paper is a good starting point if yuo ignore the conjecture that connecting cost and schedule is fraught with difficulty. Yes it is difficult. It is also mandatory for success. In the absence of these connections, including the TPM's you don't have visibility into the risks that drive the project.

If you don't have visibility into these "drivers" you're likely late and over budget before you start and you probably don't knwo it. No wonder many projects are in trouble.

Yes it is hard, that's what we get paid for.

Concept of Operations (ConOps)

In our aerospace and defense domain, a Concept of Operations is a common, and sometimes mandated, document. We rarely see these documents in the commercial domain. But in the commercial world, a ConOps would be a critically important piece of information for the project - for exactly the reason it is critically important for aerospace and defense.

The ConOps describes the "How" in "How to Accomplish the Mission." The Mission in commercial terms can be the business case, the strategy, the Raison d'être for the project.

The ConOps answers these key questions:

• What is the "Mission" to be accomplished?
• What is the intent of the business leaders for this project?
• Who are the key participants in the "Mission?"
• What are the relationships between these key players? The heart of this question is Who's in Charge?
• What resources are needed  for the project to be successful?
• What are the high-level operational tasks that must be performed for the "Mission" to be a success?
• What connectivity is required between the participating system elements? People, Processes, Tools.
• What are the training needs?
• How will we recognize that the results of the project meet the needs of the buyer?

What's Our Capacity for Work?

Knowing the staffs capacity for work is critical to forecasting cost and schedule. We have a working example taking place at our house. In Colorado, the landscaping has river rock around the house to move the snow melt away from the foundation. Over time, dirt fills in between the rocks, weeds grow and it's an endless battle to pull weeds, spray Round Up, and generally keep it clean.

Taking all the rocks out, replacing the cloth barrier under the rocks with new weed proof barrier, cleaning the rock, and putting then back is a job to be done every 10 years or so. We've waited 15.

The neighborhood landscaping crew - high school boys - are doing the work this summer. They bid a fixed price for labor, plus materials. They've built a cleaver machine to get the rocks cleaned and ready for replacement, using a power washed and a screen.

Now the capacity problem. They assumed they could get all this done before summer was over and they have to start school. They started in the front, removing rocks, cleaning, re-laying barrier, and putting the rocks back. A few weeks into this job it was clear to me that they were not going to make it at the pace they were going. So I asked a few questions:

• Guys (2 highschoolers), from the work you've done to date, how many linear feet of rock replacement have you done per period of performance? That is, how much rock are you replacing per hour, per day, per week so far?
• Do you have the measurement the rock beds around house and around the lawn perimeter - lawns are expensive things in Colorado, so much of the back yard is in wild grasses that can live on 11" of rain a year. Are these adjusted for the wide of the beds, which varies from 12" to maybe 3'
• With these two numbers - your rate of work to date and the "to go" distance - when do you think you'll be finished? Is this date on or before the start of school or the first snow?

This is the capacity for work calculation. At X feet per day, and Y feet total, minus Z feet to date, what is the completion date?

Capacity for Work

This is what the agile folks call velocity. It is a semi-calibrated work performance number. The Earned Value folks calculate the To Complete Performance Index (TCPI). This tells us the calibrated performance in terms of CPI (Cost Performance Index) and SPI (Schedule Performance Index) we need to maintain to complete on time. Earned Schedule has its own version of a performance index.

Many failed projects in the IT world we work have no idea of their capacity for work. They work at the rate they work.

Hope is their current strategy - We hope to get done by the start of school

Hope is NOT a Strategy. Measurement of physical percent complete of the work performed on the day is was planned to be complete at the cost it was planned to be completes is the only way to forecast the future performance of the project.

If you don't know your capacity for work, you can't know when you will be done or how much it will cost. No matter what the schedule and budget plans says.

What DONE Looks Like?

I started using the phrase "do we know what DONE looks like?" at Rocky Flats. It was first coined by Martin Radley, now an instructor at Carnegie Mellon. Martin used the term "done done" to add a little humor to those who cooked the books when stating they were done. This started in 2001 or 2002.

Knowing what done looks like must be in units of measure meaningful to the participants. In the previous post, there are links to briefings given at the PMI-CPM (College of Performance Management) Conference in Florida last week.

But what's more interesting, is I'm hearing and reading that phrase more often.

What does DONE look like?

Johanna Rothman has an article in which that term is used. Josh has started using it. The Bart Group. Pawel. You get the idea.

If we don't speak about actual physical progress against the planned physical progress, in units of measure meaningful to the project's participants (not just the customer), then we can't really speak about moving forward toward done.

What does DONE look like?

• Identify the needed business or mission capabilities
• Identify the technical and operational requirements needed to fulfill the capabilities or mission
• Build a Plan, build the schedule of work packages that deliver the product or services that fulfill the requirements
• Measure progress of those work package in physical percent complete of the technical performance measures at the time that physical percent complete and technical performance was planned
• Do all this, while performing continuous risk management.

That's what DONE looks like

A Malformed Crticism of Power Point

There is a thread going around from the General McChrystal briefing that used a complex Mind Map showing the interdependencies of the activities taking in Afghanistan. Like most gut reactions to things, all the authors got it wrong in several counts.

Granted there is really bad briefings made and really good ones. The starting point for building Power Points for certain contexts is Cliff Atkinson's Beyond Bullet Points site and book.

But care needs to be taken developing presentations and briefings at build them for the audience in question. Care also needs to be taken in critiquing the work of others. Especially in the absence of knowledge of the context and domain. And even more care, when newspaper reporters are the source of information.

Here's the slide in question...

When McChrystal commented "when we understand this slide, we'll have won the war." This is likely literally true.  The complexities of the situation are possibly intractable. The "slide" is a description of the complexities of the situation. The slide is the "picture" of the reality. To show this reality in simpler terms would miss the point entirely - which the New York did (and does on many issues of complexity, may favorite being the C-130J blather a few years ago by completely uniformed reporters). Simple is a term tossed around without context or domain. See the Mencken quote at the end of this post

The arm chair critics, some of which are self proclaimed rouge thinks, and in uniform even, have confused a "picture" of the map with the map.

The slide above is a Mind Map or social networking map made by the firm PA Consulting. Tools making these maps was used to locate the 9/11 relationships in the first months after the attack. Maps like these define the interdependencies between complex problems - UAV Operations in mixed flight war theaters, New Drug Applications, up stream and down stream petrochemical process optimizations, San Francisco coordinated law enforcement operations, the airlines baggage handling process improvement (a slight oxymoron I know).

The critics don't say how the map was produced, why it was produced, and for what purpose it was produced.

As a user of things like this to visualize complex problems - my current assignment is a $37B re-do of a DoD services C3 (command, control, and communications) infrastructure program just released for bid. Imagine trying to "map" the interdependencies of "everything over IP" on the planet.

Oh let's make a simple (meaning simple minded) diagram so the dumbest person in the audience can get the picture of literally millions of IP devices.

Come on, remember the H. L. Mencken quote:

For every complex problem there is an answer that is clear, simple, and wrong. - H. L. Mencken

When I hear critics of complexity, I rarely hear alternative suggestions. In our world, that's called un-informed whining. When you hear this, discount the content until the speaker comes up with a viable alternative.

The Absolute Worst Way To Plan a Project

I recently watched the demo of several web based project management tools. While listening to the web cast or watching the demo "video," I was shocked to listen to how the demonstrated built a project:

1. Set up the tasks
2. Define the start and planned end date
3. Assign the resources to the tasks
4. Identify some risks and some issues
5. 4. Blah, blah, blah

Wait Stop Right There?

Define the start and end date? This is not a project management tool. It is a tool that manages a list of tasks. A list you could create in Excel.

Never Ever define the end date for work. Only define the dependencies and the duration of the work efforts.

By Never, Ever, I mean Never, Ever. This is an absolute forbidden process in any credible project management paradigm. Here's why.

When you define the finish date, you've established a "must finish on" or a "finish no later than," constraint. Now in some domains, FNLT or MFO constraints can be set for milestones or deliverables as DEADLINES. Once you set a MFO or FNLT, you've turned your schedule into a Power Point presentation. You might as well use Power Point as your schedule tool.

No analysis of the critical path. No assessment of schedule float. No impact assessment of work moving to the right or work moving to the left. No risk impact assessment. No nothing. A nice Power Point presentation.

The only constraints in a credible project schedule - for the work activities - is Must Start On (MSO) for the Authorization to Proceed and As Soon As Possible (ASAP) for all work activities. Every other constraint you add moves your schedule toward a Power Point presentation.

A nice picture, but completely worthless as a project management tool.

Critical Path Obsolete? Probably Not

Josh Nankivel has a post on his PMStudent site about the Critical Path. This presentation is one of those "let  me re-examine the obvious." This worth the time to watch, but it brings out several issues in the domain of project and program management.

Some of the questions asked by the author include:

• Where does the CP begin and end?
• CP start and end on date constraints? 
• Fixed completion of intermediate milestone impact the CP?

But first you must watch the presentation to connect with the comments below.

But first a pre-apology. This topic is wickedly obtuse for all the right reasons. So this very narrow bandwidth blog channel is replaced in practice with a 3 day workshop, and continuous hands on mentoring, coaching, and keyboarding for the programs we work  using Monte Carlo Simulation.

If you'll go to the Lijit search box on the right panel of this blog and enter "monte carlo" you'll see the tip of the ice berg of this topic.

This discussion starts with a primary problem

When you put hard constraints in the middle of the schedule, you have essentially "broken" the schedule. In Murry Wolf's presentation, complexity is created by this approach.

In our defense domain, hard constraints such as Must Finish On (MFO), Finish Now Later Than (FNLT) are strong discourages in principle and in practice ruin the use of Monte Carlo Simulation, since the probabilistic finish date are now anchored in the schedule and cannot move.

Here's the Real Problem

In our domain, the notion of the Critical Path as an "entity" that exists in the schedule, is replaced by a probabilistic assessment of the confidence of completing on or before a planned date. This is the role of the Monte Carlo Simulation of the schedule.

This approach removes all the hand waving, re-definition of terms, and arguing what is meant by Critical and Path.

The identification of the paths the impact the confidence of a deliverable is called the cruliality of the schedule. Using Wolf's classification are useful however:

• Acute Path - this is a path through the schedule that has negative slack that will substantially impact a deliverable
• Watch Path - this is a path through the schedule that needs to be corrected so the deliverable can be made as promised
• Free Paths - are paths that have positive float

This is an interesting view from Construction

At one time I was a member of the College of Scheduling. But the construction paradigm did not sit well with me for a few reasons:

• The litigation support issues dominate the style of scheduling. In the defense world, the Integrated Master Plan is "on contract." The Integrated Master Schedule (IMS) is subject to change of course - through a change control process - but the contributes to the IMS - the Integrated Product Teams (IPT) have similarly "promised" to deliver on time, on schedule.
• Litigation is rare and usually takes place after some complete disaster occurs.
• The use of constraints is common and remove the ability of Monte Carlo Simulation.

It's the Monte Carlo Simulation that got me hooked on the method of doing the IMP/IMS. The IMS in the defense world MUST have minimum constraints. The DCMA 14-Point Assessment sets the upper limit of constraints.

To answer Mr. Wolf's pressing question - "which paths should we be looking at," means answering the cruciality question with a Monte Carlo model.

Cruciality is the combination of criticality and sensitivity of the drivers of the deliverables. That is what activities drive the "lateness" of the deliverable, how sensitive is this lateness and how critical are they?

That's how you answer all confusion created by constraints, multiple critical paths - which are always there in any real schedule, since the durations are random numbers - and the identification where to look for the next "emerging" problem with staying on schedule.

The Role of Program Planning and Controls (PP&C)

Ron Rosenhead has a wonderful post about "two project teams." Ron suggest, through an article he read, that a second team could

• Check out whether the risks are too great or acceptable
• Develop a business case for or against the project
• Identify whether there any project benefits - what they are and whether they are achievable
• developing a plan identifying resources required and how this will impact on other projects and business as usual
• Identify whether you have sufficient resources of the right caliber
• Check is there is a link back to the company strategy

Not all these are direct activities of PP&C. But the organization where PP&C typically lives - Finance and Business Operation (F&BO - say that fast 5 times to get the gist of our status) - does have this accountability.

F&BO includes PP&C, Contracts, Subcontracts, Procurement, and things like that.

These activities provide an external assessment of the project's performance through the "planned" performance compared to the "actual" performance. Earned Value is of course the basis of this assessment, but there can be other simpler assessments as well.

You planned to have 20 tables in the database done, by the end of the week (all of equal effort) and you got 18 of them done, you're 90% physically complete.

When you separate the "doing" from the measurement of the "doing," you start to establish credibility for the work being performed in ways you can't when they are not separate.

Who's doing what on this project?

On an agile forum there is a discussion about roles, Business Analyst, and other type roles and what these people do on software development projects.

This of course is backwards, like many discussions that fail to return to "first principles." It's the deliverables that the project produces, not the effort performed by roles or even by the people.

Deliverables Are King

So how can we connect deliverables with people and processes? Simple the Responsibility Assignment Matrix (RAM). The RAM is really called the Accountability Assignment Matrix, since accountability is all about determining who is going to cause something to appear.

The RAM defines the deliverables and the people accountable for them. Where do the deliverables come from? The Work Breakdown Structure of course. Don't do WBS on your project, I know you're an agile shop and agile shops don't do WBS's. Not true of course. What are those 3x5 cards stuck on the left hand side of your cork board. They are the deliverables. They have relationships with other cards pending or complete.

This is not a very good WBS of course, because the cards are not thought through to the end in many cases. But it's a start.

With a "real" WBS, which does think through the deliverables to the end, the terminal nodes or maybe the parner of the terminal nodes are the columns in the RAM. The rows are the names of the people accountable for making those deliverables appear at the needed time in the project.

The WBS and Estimating

Craig Brown and Mike Clayton have a related posts on the Work Breakdown Structure and estimating.

Building a credible cost or schedule estimate is a "process" focused effort. Let's start with the guidance documents we use:

• The GAO Cost Estimating and Assessment Hanbook
• NASA's Cost Estimating Handbook

While these guides appear to be applicable to only large government contracts, they have much to say about the process of cost and schedule estimating for all projects. The fundamental concpet is.

All variables in cost and schedule estimates are random. Knowing the statistical properties of these variables is a critical factor in producing a credible estimate.

Without this understanding all estimates are untrustworthy.

Building the Estimates Top Down

Mike Clayton suggests - as do others - that building the WBS can tale place top down, bottom up, or some combination.

Top Down is the only place to start. The WBS is a decomposition of the product and the processes that construct the product. If we start building the product bottom up, we'll end up with a hodge podge of parts assembled in some arbitrary manner. Image building a software product out of pieces and trying to assemble them into a production ready outcome.

Don't build the WBS bottom up - period. It's simply bad engineering.

Once the WBS is built top down - decomposed from the Statement of Work (SOW), the Statement of Objectives (SOO), the Statement of Project Outcomes (SOPO), the Concept of Operations (ConOps) - then and only then can it be adjusted from the bottom up.

Changes to the WBS are the same as changes to the Product Breakdown Structure (PBS). In fact the PBS is one part of the WBS. The other part is the Process Breakdown Structure, that identifies the processes needed to build the product.

Remember

The WBS is the combination of the Product Breakdown Structure (essentially the bill of materials for the Product) and the Process Breakdown Structure (the processes needed to produce the product).

With the WBS cost and schedule estimates can be made bottom up for each deliverable. Then revisit the total budget resulting from the roll out of cost and start making adjustments for the "planned budget."

This iterative approach maintains the product structure integrity while assessing the deliverables from the budget.

To do anything else, breaks every piece of the project's integrity. Cost, Schedule, and Technical Performance are now dis-connected and the project heads straight for the ditch. A common occurance in the absence of a project management process.

The Challenge Question to any Lifecycle Mehtod - Serial or Not

How long are you willing to wait before you find out your late? (Or over budget, or your the thing you're building doesn't work as specified?)

That's the question that must be answered by any development life cycle in any project domain, in any product context. Doesn't matter if you're planning a mission to Mars or planning a summer picnic, if you don't have some measure of physical percent complete against the planned physical percent complete, you're late before you start, and probably over budget as well.

The reason serial lifecycles are useful for executives is they bound the exposure. The critical success factor for serial lifecycles is to set the time boundaries appropriate to the work rhythms. To make blanket statements about the appropriateness of one lifecycle over another in the absence of the business, technical, and financial rhythms is to basically say -

"I don't really know what your problem is, but I have a solution for you"

So ask and answer the "waiting" question first, then choose a project lifecycle that "fits" the needs to know that answer.

There are other attributes of lifecycle that cannot be generalized as well:

• Risk reduction
• Product production cost
• Resource utilization and absorption
• Cost of Goods
• Supplier capacity to deliver

Better Estimating is the Solution to Poor Estimating

Johanna Rothman has got me thinking about why management and project managers want serial development and have such difficulty with estimating.

It starts with the missing understanding that all point estimates are in fact Wrong. This starts with the misunderstanding between statistics and probability.

Cost and schedule estimates are a forecast about a future event, the final project cost or schedule duration. Because future events are uncertain, they must be described probabilistically. But to do this, we need knowledge of the underlying statistical nature of the processes that drive the cost and the schedule. Without this statistical knowledge, we can not make credible estimates of the probability of the outcome.

First All Point Estimates are Wrong

But these types of estimates are desired by management. Wrongly desired of course, but still desired. So how can we "push back" on this request?

The first step is to define the range of values for any point estimate and the confidence that the point estimate falls within that range. Without this understanding all the participants in the project will be disappointed.

The first step is to define what the point estimate means. Is it the:

• Most likely - the mode of all the possible costs and schedule durations?
• The 50^th percentile - the median of all possible cost and schedule durations?
• The expected value - the mean of all possible cost and schedule durations?
• The 4^th percentile - the highest possible cost and schedule duration?

While these variables are being estimated, the "actual" project cost and schedule is also an uncertainty:

• The "point" or "best" estimate is not the only possible estimate - this means other estimates are possible.
• Use of the phrase "most likely" implicitly assumes that other value are "less likely."
• "Most Likely" (mode), 50^th percentile (median), and "Expected Value" (mean) are statistical terms characteristic of probability distributions.

This terminology implies that cost and schedule are statistical in nature and are defined by their probability distributions.

The use of statistical cost and schedule estimating provides insight into the variance of these estimates in way naive "roll up" estimates cannot. The use of simple (naive) roll up estimates usually leads to disappointment in the end, because they are statistically unsound from the beginning and get worse as time goes on.

"Earned Value for Cost Estimators," John Driessnack, Washington Area Chapter, SCEA, April 24, 2006.

There are Four Types of Uncertainty on Projects and the Behavior in Their Presence

1. Normal Variation - that occurs in the completion of tasks arising from normal work processes. Deming has shown that these uncertainties are just part of the process. Attempts to control them, plan around them, or otherwise remove them is a waste of time.
2. Foreseen Uncertainties - are identifies but have uncertain influences.
3. Unforeseen Uncertainties - are events that can't be identified in the planning process.
4. Chaos - appears when the basic structure of the project becomes unstable, with no ability to forecast the outcome of cost, schedule, or technical performance.

See - "Uncertainty and Project Management: Beyond the Critical Path Method," A. De Meyer, C. Loch, and M. Pich, INSEAD Work Paper 2001/04/TM and "Managing Project Uncertainty: From Variation to Chaos," A. De Meyer, C. Loch, and M. Pich, MIT Sloan Management Review, Winter 2002.

With this guidance, you will come to see what Johanna is suggesting does not have to be, should not have to be, and certainly is not the approach by those "managing in the presence of uncertainty."

So when someone (in this case Johanna) says:

... Of course, a serial life cycle provides a prediction that’s almost guaranteed to be wrong ...

the simple solution is to stop this type of thinking and start thinking in terms of probability and statistics and how the single point estimates are both wrong and can be improved by thinking in statistical terms.

So Here's Some Fundamental Principles of Cost and Schedule Estimating

• Naturally occurring variance is part of the underlying statistical behaviour of any netwrok iof activities.
• Attempting to control or Over Control these natural variances is a waste of time - this is the purpose of cost and schedule margin.
• Mitigate unforeseen uncertainties with risk buy down activities - have a Plan B for everything that needs cost and schedule protection.
• Have an alternative plan for Unforeseen Uncertainties.
• When Chaos emerges, replan the project

Project Performance Measurment

In Johanna Rothman's post it was suggested that predictions about cost and schedule are almost guaranteed to be wrong. Here's some guidance on why you should ignore that advice if there is not a statement about "doing the wrong thing will not usually produce the right results." In other words "bad project management is not the excuse for failing to understand it is bad project management."

One school of thought says we cannot manage what we cannot measure. Of course this is not true, since we manage things all the time that can't be measured - product innovation, manuscript development are managed without direct metrics. At best they could be managed with inadequate metrics. In the intellectual and creative domains, metrics do not provide insights needed to forecast the outcome.

What's missing from this approach to measurement is the answers to questions like:

• How good are your processes?
• What quality can we expect from our efforts?
• When will we be finished with this project?
• How much will it cost when it is finished?

What happen is project managers measure intellectual processes - like software development - with qualitative metrics rather than quantitative metrics.

Monitoring activities using metrics to evaluate broad issues like progress to plan are usually counterproductive. Measuring specific progress against identified quantitative constraints to determine the likelihood of meeting that constraint is a realistic process. Measures like defect rates, sufficiency of assigned resources, whether there is enough time and money to complete the project - the Earned Value Measurements - for the basis of the Program Controls function for any non-trivial project.

Software Sizing, estimation, and Risk Management: When Performance is Measured Performance Improves, Daniel D. Galorath and Michael W. Evans, Auerbach Publications.

Some well know thinkers have something to say about measurement and especially performance measurement:

• Apparently - all things being equal - it is better to measure than not to measure - Winston Churchill
• Quantities are measurement of qualities - Paul Kirchner
• Now, things do not, in general, run around with their measures stamped on them like the capacity of a freight-car: it requires a certain amount of investigation to discover with their measures are - Norbert Wiener 1920

So in the end it is very simple

No Point Estimate can be correct in the absence of statistical variances.

Suggesting that predictions about cost and schedule are almost guaranteed to be wrong without stating that those estimates are missing the underlying statistical model is both right and wrong at the same time.

Never provide point estimates without the statistical variances. To do so puts your in the same position as Johanna - your right, but wrong.

Sticky Notes and Other Ways To Capture The Project's Baseline

Pat Weaver suggested that 3M Sticky Notes are a good way to capture information about projects - and they are. Pat also suggests this is a social process, which it is sorta.

Let's start with a picture of the sticky notes on the wall after the dust has settled:

This is the picture of tranquility after a 2 weeks of haranguing by the team on the content of the "wall." This collection of stickies represents the Integrated Master Plan for a orbiting space craft test vehicle. But at this resolution, it could be any non-trivial product or process.

Like Pat's suggestion, this is an interactive but not always sociable process. The interest of each contributor is ferociously defensed during this process. Fixed time and bounded budgets are the starting point. As well, the simple fact the ego's are involved drives some of the conversation.

It is pure fantasy that the extremely talented engineers and their Control Account Managers simply all come together and sing camp fire songs to build this Integrated Master Plan. I've led these efforts in a variety of domains and contexts within those domains, and it just doesn't happen in the ways the "consultants" and book authors what us to believe. But it does get done and the team is better off for it, for a simple reason - we ALL have a stake in the Mission Success.

In the outer lobby of this building is a large banner ...

100% Mission Success

That's the starting point for any successful process, social or not.

Stickies are OK, but then What?

With the stickies on the wall, what's the next step? There are several choices, but the step from the picture above is to move the contents of the wall to a Mind Map. We use the MindJet tool to capture the arrangement of the Stickies as they are being developed on the wall.

From there several things happen:

1. The map is turned into the Integrated Master Plan structure as defined by Integrated Master Plan and Integrated Master Schedule Preparation and Use Guide. This is the starting point for the work we do as Program Planning and Controls professionals.
2. From there a Performance Measurement Baseline (PMB) is started when MindJet exports the map to MSFT Project.
3. The Work Breakdown Structure is built in the same way. Monetized work packages are attached and the whole mess is placed on baseline.
4. A second visual tool that uses the same source file is PERT Chart Expert, from www.criticaltools.com. This makes a picture of the IMP/IMS in a time phased for an hangs on the wall.

There are several critical aspects of this approach that are not obvious at first.

1. The BIg Visible Chart is the key to creating a shared understanding. Some folks in the agile community suggested this a few years back. Well they were 2 decades too late. We had BVC's hanging on the entry hallway in Building O6, TRW, One Space Park, Redondo Beach, in the late 70's. What the BVC does is just that - make everything on the program visible in a single BIG chart. To do this of course you need a plotter and a BIG wall. The wall is called the "wall of truth." If it ain't on the wall it ain't true.
2. Putting information on the wall moves it from a little window - you're laptop screen - to a big window the wall. But it also moves it from personal control to group. Lean and Kaizen process know this. When an individual is presenting information there is a naturally tendency to "defend" and "attack" that information. When the information is shared on the wall it moves from individual to group ownership. Again this is something developed in the late 70's at TRW and other A&D firms.

The key to success is to get from the wall to the Performance Measurement Baseline and start the Planning and Controls processes.

• Do we know what done looks like?
• Can we manage the risks along the way?
• Who is accountable for what?
• How do we measure progress?
• What is our forecast for completion - time, money, technical performance?

Does Methodology Matter?

Jesse Fewell posts a repeat piece about "Methodology Doesn't Matter." I've participated in the "method wars" in the past, with no progress. What I've come to understand through the school of hard knocks is that it's the outcomes we're interested in not necessary the path taken to reach those outcomes.

This removes all the theological issues around a method. But there is another important issue solved by this change in perspective.

When we focus on outcomes in place of focusing on the methods, we have made the connection to the business value. In some methods this connection has been made. The agile methods for example. For others, like Earned Value Management, the connection is juts now being discovered.

So what are the outcomes that are needed to make the connection to business value or mission success? here's my standard list.

1. Can we determine with some degree of confidence what the project will cost when it's done? How does this information match the expectations of the customer and their available funds?
2. The same answer for the duration? Can we determine the probability of completing "or or before" the need date for the project?
3. Can we discover, through some method or process, what the customer or funding agency considers "done." That is what are the capabilities that result from the project that allow the customer or funding agency to perform "their" job?
4. Along the way can we discover what impediments will impact our ability to answer the first three questions?
5. Can we determine with some degree of confidence that we are actually able to answer the previous 4 questions?

So if you can answer these 5 questions – with the proper degree of confidence - then "How" you answer them - that is what method you is to answer them - is not as important.

The inverse is also true. When ever there is a proposal to change a method, improve the process through a new method, augment the existing paradigm - one that may or may not be working, or conjecture any "new idea" - ask the following:

Does this thing that is being suggested provide improved information to these 5 questions? If yes, this is a desirable approach. If no, you're wasting your time.