Herding Cats

Knowing How Much a Project Costs is a Measure of PM Maturity

While sitting on our regular Business Development conference call this morning we talked about a client who is preparing to roll out a global project management system. One of the first questions we ask to set the tone new engagement in the commercial world is:

Do you track costs through time sheets and material accounting for your projects?

If the answer is "no we don't really care about knowing that," then we have established the level of maturity of the project management organization. It is LOW.

The simplest measure of project performance is cost, schedule, and technical performance. There are others, but these three are the foundation of answering the question - when will we be done, how much will it cost when we're done, and when we're done will the project do what we want it to?

So the next time you hear someone say, "we don't really care what our projects cost." You'll also know where that project based firm sits on the maturity scale. Using CMMI as a guide, they're at Level 1 - Chaos if they're spending other peoples money. And Laissez Faire if it's your own money. In French, Laissez Fair means an un-managed approach.

This is not a formula for success. And we wonder why IT projects have such a dismal success rate.

I'm Out Of My Lane

Watching C-Span Afghanistan and Pakistan Strategy hearing. General McChrystal made an interesting comment when asked something about the civilian processes. "I haven't been a civilian since I was 17 and I'd be Out of My Lane if I were to answer your question."

I wonder if the concept of "I'm out of my lane" would lead to clearer and cleaner conversations are topics like PM 2.0, risk management, program management, earned value, agile, etc.

Prefacing a statement with "in my experience," or in the domain I work, etc. may provide a common touch point for potentially  confusing and even controversial topics?

Another phrased used - and one we use in our daily conversations - is "we're working the problem."This notion makes a clear statement. The problem has been defined, work is being applied directly to the problem, and an outcome is expected that is acceptable to the stakeholders.

Just back in the hotel "working the baselining of the next phase of a large Unmanned Aerial Vehicle program before Christmas break."

Why is it so hard?

Steve Romero comments on the "Same Old Problem," regarding all the reasons projects fail into the ditch.

I've modified the statements a bit, but they're still true.

• We don't have enough resources to get the work done
• In our organization, there is no real planning process and when there is no one follows it
• We don't prioritize our projects or the features in them. every project is a #1 priority along with the "must have" features
• Our priorities are constantly changing with no explanation, impact analysis
• Everyone is by passing the approved processes and then wonders why things go wrong
• Funding tends to be very political, unstable, and not connected to the produced value
• We can't kill a project or a feature in a project. If we do kill a project or the features, it always seems to show up again. We never have any objective measurements to overcome the politics in decision-making. 

So why are things statements true in some domains and not others? 

The Minimum Buy In Cost for Managing a Project

In the aerospace and defense business the guide of the amount of Level of Effort work runs around 10% to 12% depending on the customer. This funding goes for administrative functions: Program Management, Contracts, Program Planning and Controls.

So where is the allocation of resources for "managing" the project? The commitment to managing the project? The skills and experience in managing projects?

The CMMI-DEV 1.2 framework asks:

• Goals
• Commitment
• Ability
• Measurement
• Verification

When I hear of projects that have "gone in the ditch," like those from Steve's post, I ask "do those project teams actually know how to manage a software project or a collection of software projects?

Sounds like the answer is no. So Steve has it right when he says 

It is heartbreaking to know so many folks continue to deal with these circumstances. I am very much looking forward to the day when these problems are a thing-of-the-past.

This is a Self-Inflicted Wound

From a recent NASA Cost Symposium paper "The Joint Confidence Level Paradox: A Historical Denial," the simplest explanation of the failure to apply sound principles to the management of projects has two conclusions:

1. We are not really seeking probable cost and schedule accuracy (and the management of scope and requirements that comes with that), but are primarily concerned with keeping programs viable and fundable as long as possible.
2. The variables are too complex, unquantifiable, and incalculable.

One more piece of background for failing to invest in the necessary planning and controls comes from Dan Galorath,  "Controlling Software Costs: Development is (Only) Job One"

The Whole Brain Look at Project Management

In Rechtin's Systems Architecture book, there is a list of what makes a good system architect. The system architect he speaks of is not the IT architect - although in the 2nd edition, he includes that. It is the product architect. Products like GPS, the Space Shuttle, complex military and civilian systems.

A recent paper in the INCOSE Systems Engineering Journal, revisits this concept. "Whole Brain Thinking in System Architecting," Tony Di Carlo, Behrokh Khoshnevis, and Firdaus Udwadia, Volume 13, Number 3.

I'd conjecture these attributes can be found in good project managers as well:

1. Communication skills 
2. High tolerance for ambiguity 
3. The ability to make good associations of ideas 
4. The ability to work consistently at an abstract level 
5. A level of technical expertise 
6. A tempered ego; the opposite of arrogance 
7. Leadership; gets the most out of others 
8. The willingness to backtrack, to seek multiple solutions 
9. The ability to build teams 
10. Charisma 
11. The ability to read people well 
12. Self-discipline, self-confidence, a locus of control 
13. A purpose orientation 
14. A sense of faith or vision 
15. Drive, a strong will to succeed 
16. Curiosity, a generalist’s perspective

So now, assuming my conjecture is applicable, where do tools fit into the discussion of increasing the probability of project success?

PM Insight Newletter

I've been invited to join an esteemed group on Weekly PM Insights hosted by Raymond Posch to speak about programmatic risk. 

The authors include:

• Hal Macomber
• Michael Greer
• Elizabeth Harrin 
• Bob Hartman 
• Lee Lambert 
• Aaron Shenhar 
• Bill Stewart 
• Martin VanDerSchouw

I know Hal personally and have corresponded with Elizabeth.  I look forward to meeting the other authors as the newsletter and web site grows.

My contribution is focused on programmatic risk and the process of managing this risk, guided by the DoD, DOE, and Software Engineering Institute's frameworks. This material will be practical in nature, based on the principles of the three guiding organization, but practiced on actual aerospace, defense, and Energy Department programs.

So just a reminder of the starting point for all risk management is the framework from the US Department of Defense.

There are other frameworks, even one from PMI. But all miss one or more aspects of the risk management processes needed to establish a credible Risk Management Plan (RMP).

Impediments to Success from Organizational Issues

Ron Rosenhead provides a great post about the organizational sources of project failure.

There are of course other sources of organizational failure modes, but Ron's list is a great starting point.

A Process Framework for Increasing the Probability of Project Success

The processes needed to increase the probability of a project's success include:

• The cost estimates for the project and the basis of estimate for those costs. The customer wants to know how much the product or service will cost
• A Work Breakdown Structure describing the products and their components to be delivered and the work process needed to deliver those products
• An Organizational Breakdown Structure describing who is participating in the project
• A Responsibility Assignment Matrix that connects the participants in the project with the deliverables of the project and identifies who is accountable for each deliverable
• An Integrated time-phased plan and master schedule showing the flow of increasing maturity of the deliverables and the sequence of work needed to produce each deliverable to its planned level of maturity
• A Work Authorization and Control Account process to allocate funding, control expenditures, and assure budget control of the project
• A means of accumulating costs from each work activity (Work Package)
• Analysis of the cost, schedule, and technical performance measures of the project
• A set of management actions that use the analysis information
• Baseline Change Control processes to maintain the integrity of all commitments of the project
• Training needed to perform the processes
• Surveillance needed to assure the process are being applied correctly and the expected benefits from these process are being delivered to the project

What form each of these processes takes is project and method dependent. But in the absence of some form of these processes, the probability of success is reduced.

More Matrix Management

Came across an article on Matrix Organization for Large Projects. Some good points here on setting up matrixed projects.

Can Matrix Management Work?

There's one of those unqualified, unsubstantiated, and pretty much curmudgeon quotes floating around. You know the kind. "It didn't work for me so it can't work for you." It usually goes like this…

"I think it is clear that matrix management is not and does not work, leading me to conclude that it is one of the LEADING CAUSES for the rather abysmal "success" rate of projects in most organizations."

Of course, there are no units of measure here. No technical, business, or operational context or domain. No actual evidence or facts, other than personal opinion and anecdotes – a dangerous approach to process improvement.

So let's see if we can get all this anecdotal "theory" about matrix management to somehow come in contact with reality.

In the manned space flight business and nearly every other DoD, NASA, and DOE domain matrix management is the basis of Integrated Product (Project) Teams (IPTs). See O 413.3-18 as an example for the DOE context. NAVAIR and NASA have similar practice guidance.

http://www.directives.doe.gov/pdfs/doe/doetext/neword/413/g4133-18.pdf

So here's how it works. Let's pretend I'm the Propulsion Engineering Lead (yes it is rocket science).

I'm accountable for the design and development of the propulsion systems for our flying machine. This is critical concept - accountability - that is missed in most programs that go into the ditch. First Accountable, then Responsible. See http://herdingcats.typepad.com/my_weblog/2007/02/responsibility_.html for discussion on the RAM.

I report to the Program Manager - single line of responsibility flows down. But it flows down through the Systems Engineering Lead. I take direction from two people. The PM and the SE. They work with me in a matrixed manner since we're on an IPT organization. Now if they tell me conflicting things - say how much thrust is needed to hold orbit at Mars, then I need to convene a meeting to get this sorted out. Why, because I'm ACCOUNTABLE to make the propulsion system work. It doesn't work, the mission fails.

Now, the PM has bigger role than the SE's. So during the meeting, the SE's will make their points, I'll make my point (all supported by technical data, no opinions allowed here) and the PM will take all this information in and we – as a team - will have CONCURRENCE on the outcome. Remember we're an IPT.

Now, in walks the Safety and Mission Assurance (S&MA) Lead, http://www.hq.nasa.gov/office/codeq/ . And she disagrees with our little decision. So who trumps whom? Well in the manned space flight business S&MA has a powerful role. More powerful than the PM, because certifying launch readiness is a S&MA role, not a PM role. We're back to accountability.

All these accountabilities are defined in the Responsibility Assignment Matrix (RAM). The responsibility, accountability, communication, information (RACI) model drives how the IPTs work inside the matrix. In fact the responsibility part of the RACI is never addressed until the Accountability assignments are made, and then it's up to the accountable person to flow down the responsibilities.

So to attempt to get the theory to come in contact with the reality of how matrix management "can" work. Because that's what real PM is about, the realities of projects.

No doubt there are numerous examples of how to make it NOT work. But that's called a "false premise" argument. Just because you've never seen it work, or have some quotes from a couple of dead guys who say it can't work, doesn't mean it's not working every day on 1,000's of programs. It's proof by anecdote. NOT

It's critical here to look around for places matrix management does work and see if what they do is in anyway useful and applicable to your project environment. Maybe it's not. But that doesn't mean it is generally the case that it doesn't work. In fact just the opposite. If it works someplace else, you have an anecdotal working example. Try and figure out how they made it work. If you can't figure it out, or can't put to work what they figured out – we've found the problem – YOU. Learn from others, don't belief everything you read in 50 year old books. Ask deeper questions. Investigate multiple views. Try to determine if a concept or statement is in fact universal, or just an anecdote of failure in a specific domain.

This is a common mistake of inverting the search for improvement. "Doesn't work for me, can't work for you," is a weak approach to trying to increase the probability of program success.

The Real Project Management 2.0

The Project Management 2.0 has yet to state how project's are improved in units of measure of the beneficial outcome. Oh there's some time savings in communications of the project team. Lot's of marketing hype. Hype 2.0 is a popular term applied to most 2.0 initiatives.

• Change is coming
• Improve your team
• Doing more with less

How does any process improvement effort actual show that processes are improved? In the absence of tangible evidence of how a project will be improved any 2.0 effort degrades into buzz words.

Not Buzz Word Words

• "We have increased our probability of success 14% for the SAP roll out by using this PM 2.0 process"
• "The non recoverable sunk costs (waste) have been reduced 17% using this PM 2.0 process"
• "Our capacity for delivering finished software components to integration test has increased 26% in the past 4 months with our new PM 2.0 process"

What's Missing?

What's missing from project's? Project Management 2.0 needs to define in units of measure meaningful to the customer (most PM 2.0 messages are product sales messages) and say how PM 2.0 provides the mechanism to:

• Deliver on or near the planned date
• Keep the project cost on or near the planned cost
• Deliver the technical capabilities needed for the business process to function as needed

PM 2.0 needs to be about Project Management, not just software development

The PM 2.0 solution needs to be actionable beyond buying a product.

PM 2.0 needs to:

• Produce actual improvements in "projects" not just communication between the people on the project
• Be attributable to these improvements
• Be sustainable across the organization
• Be repeatable across projects

Process Improvement Project Check List

Michiko Diby referenced a GAO (Government Accountability Office) report titled Business Process Reengineering Assessment Guide. This report, written in 1997 rings true today for any re-engineering for for that matter any IT enterprise project.

A quick (generic) summary

Assessing the decision to pursue a process improvement project

• Has the business reassessed its mission and strategic goals?
• Has the business identified performance problems and set improvement goals?
• Should the business proceed with process improvements?

Assessing the improved process' development

• Is this project properly managed?
• Has the develop team analyzed the target process and developed credible alternatives?
• Is there a credible business case for proceeding with the process improvement?

Assessing the project implementation and the results

• Is the business following the implementation plan?
• is management addressing all the change management issues?
• Is the new process achieving the desired results?

Notice the focus on planning, assessment, measurement, management. Now use the GAO report (adapted to the current context and domain) as a guide for enterprise process improvement projects.

Black Letter Project Law

Paul Ritchie introduced a new idea – the Black Letter Law. This is the notion that there are basic standards for any practice. The original practice for Black Letter Law was of course Law.

So are there Black Letter Law’s for Project Management? Paul mentions three. Here’s my take on expanding Paul’s great concept.

• Do we know why we’re doing this project?
• Do we know what components make up the deliverables for the project?
• Do we have any idea of how long it will take and how much it will cost for this project?
• Do we have any notion of what problems we will encounter along the way and what we plan to do about these problems before the bite us in the butt?

Now why is this important?

When someone comes along and claims to have something new. Be it PM 2.0, agile PM, some clever tool, or even a new take on a way of actually managing a project instead of trying to sell us something that can be used to manage projects – this is a way to ask some simple questions.

• Does your gadget help us understand better why we’re doing this project?
• Does the thing you're suggesting I use reveal the needed capabilities and business justifications?
• Does it do a better job of managing the requirements around the components of the project? Be they software, hardware, concrete, or steel?
• Can your gadget improve the management of my schedule or cost?
• Does your new approach provide any beneficial the impact to the risks impacting the project?

The answers to these and any other questions must be in units of measure meaningful to the buyer. No fluff. No “we’re the best thing since sliced bread,” no “everyone gives us wonderful reviews and votes us the best gadget in the valley,” no "here's a bunch of people who spent money with us, so you shoudl spend money with us as well."

Nope, the answers to these questions have to describe the unassailable beneficial outcomes. Stated in units of measure meaningful to the buyer. These units of measure should be dollars saved, risks reduced, value increased. Never can they be "we increase value." How much value - tell me the actual amount or at least a relative percentage. "We save you time." How much time?

Don’t hear that message? Move away away folks, nothing to see here.

Leadership Framework

The Deputy Secretary of Defense Gordan England has a simple foundation for how to "make" a leader:

• Treat every person with dignity and respect – nobody is more important than anyone else
• Be forthright, honest and direct with every person and in every circumstance
• Improve effectiveness to gain efficiency
• Cherish your time and the time of others – it is not renewable
• Identify the critical problems that need solution for the organization to succeed
• Describe complex issues and problems simply so every person can understand
• Never stop learning – depth and breadth of knowledge are equally important
• Encourage constructive criticism
• Surround yourself with great people and delegate to them full authority and responsibility
• Make ethical standards more important than legal requirements
• Strive for team-based wins, not individual
• Emphasize capability – not organization
• Incorporate measures and metrics everywhere
• Concentrate on core functions and outsource all other

Ask how these can be applied to project work?

The Eight Step Process

The US DoD has started using an "Eight Step Process" for logistics and other activities. These are (in general terms)

1. Look at the overall picture
2. Determine the work unit accomplishments
3. Determine individual accomplishments that support the work unit goals
4. Convert expected accomplishments into performance elements
5. Determine work unit and individual measures
6. Develop work unit and individual standards
7. Determine how to monitor performance
8. Check the performance plan

The source of these steps is rooted in the OODA Loop (Observe, Orient, Decide, Ac t), which originated in the 1950's by Col. John Boyd. It is an objective description of the decision making process. Due to an emphasis on the infinitely repeating nature of decision making, the OODA Loop is an excellent match for the AFSO21 (Air Force Smart Operations for the 12^st Century) principal of continuous improvement.

In the project management context and domain, the Eight Step Process works like this

1. What is the highest level description of "done" that is not "begin, do all work, done?" This means what are the work steps that result in done at a level that is independent of technology and details. This is the description of the path to increasing maturity for the deliverables.
2. What are the work efforts needed to produce these deliverables?
3. What are the significant accomplishments needed to move the deliverables from one level of maturity to the next?
4. What are the units of measure of these Significant Accomplishments?
5. What are the work packages and their Accomplishment Criteria (exit criteria) ?
6. What are the contents of each Work Package?
7. How would we measure the physical percent complete of each Work Package?
8. What are the measures needed to assess and forecast the future performance?

A Thought on Status Reports

Dmitri Ivanenko has some good things to say about status reports in "Are You Ready for your Next Status Report?"

One critical missing piece in this post the raw materials for the status report come from the plan. You plan the work and work the plan, you have the materials for the status report in your plan. This "plan the work and work the plan," is independent of any product development method. Agile has plans. They're called planning sessions. Agile works the plan, it's called an iteration. DoD has a plan, it's called an Integrated Master Plan / Integrated Master Schedule (IMP/IMS). DoD works the plan, with weekly Work Package assignments and Earned Value reporting. So no credible project can not - not plan the work and work the plan.

But the status report still needs to report status of the planned work:

• Perform no work that is not in the plan (in some way). Then you can report what was accomplished from what was planned to be accomplished. If your doing work that's not in the plan - why? Who asked for it? Who's paying for it?
• If you address risks in the status report that are not in the risk registry, then put them there and report on them.
• If you spent more money or delivered late, you should report against the planned spend and deliverables.

What a status report does is

Report the status of the planned activities. So have a credible plan and report progress against it using Dmitri's guidance.

PM 2.0 - Another View

Pat Weaver provides more insight on the issues of Project Management 2. I had not seen Wikipedia article he references. In it Traditional Project Manage is described as:

• Centralization of control
• Top Down Planning
• Authoritarian environment
• Implied structure
• Limited/Restricted access to the plan
• Local access to information
• Limited communications within the team
• Separate projects
• Overly complex tools
• Rigidity of tools

Wow. I didn't know traditional project management operated in this way! Now of course the number one author in the references list at the bottom of the Wikipedia page is selling a PM 2.0 tool based on this premise.

This list, repeated in other blogs and web sights about PM 1.0 attributes is actually a description of incompetent business management, ignorant personnel management, and malicious use of position. You know businesses that are going out of business.

Pat goes on in his post about the reasonableness of the PM 2.0 content of the Wikipedia article. Pat has restated many of the same points I've made and Quant.M.Leap made. My sense is the authors of the PM 2.0 approach defined in the Wikipedia track have somehow concluded all the world are fools and only they have the answer. Yes, ERP projects fail in many ways at an alarming rate in the press.

Of course these PM 2.0 advocates probably haven't actually managed ERP projects or seen successful ones to know the difference between the attributes of success and failure. Nor is it likely they can know the difference between the Boston Big Dig failure and the successful I-25 corridor here in Denver.

It's actually irrelevant in the end

Thank the stars above, no pubic institution or credible private enterprise is going to buy their message "as is" for a simple reason - governance.

Business and project governance. Governance of course does not assure project success. But governance assures that the nonsensical descriptions provided in the Wikipedia Traditional Project Management attributes do not exist in principle. Governance makes every attempt to also assure they do not exist in practice.

So Here's My Simple Minded Conclusion

PM 2.0 will go the way of the first round of "wanna be" extreme programming advocates. Note I did not use eXterme Programming, because Ron had and still has one of the viable alternatives to the "design, code, test" approach of the past - along with Scrum, Crystal, and DSDM. What the extreme programming advocates - those who read the book, but never did the work - wanted was to eliminate all the prep work needed to get in a position where eXterme Programming could work.

The great thing about approach taken my the misinformed, ill trained and inexperienced extreme advocates was they never got hired by any outside firms to practice their voodoo. Same will happen here.

As pat suggests

Beware zealots bearing gifts

Another Run At PM 2.0 Clarification

Here's another attempt to put some structure into what some authors are calling PM 2.0

I'll start with a simple list of what activities take place during a project. PMI calls these Knowledge Areas. This means knowledge of "what" to do is needed to increase the probability of success of the project.I'm not the biggest fan of PMI, but I am a member and the Knowledge Areas are a good starting point for shared vocabulary.

I've learned that without a shared vocabulary the conversation is hard to have. So here's PMI's simple vocabulary about what happens during a project. These are not product development verbs, they are project management verbs.

With this stating point - it's a starting point, but I can't think of a simpler one for this post - what are the activities performed which managing a project in the Traditional, Agile, and PM 2.0 paradigms.

By activities I mean, what does the project manager and the technical teams do while working the project that is independent of the engineering processes - writing code, welding metal, pouring concrete.

Several colleagues and I have tried to make a run at this. The topic of "what is project management" outside of the PMI paradigm has been talked about before. Here's my rough cut for the traditional processes.

People, Processes, and Technology

People, Process, and Technology is a common management consultant set of "pitches."

I'm here to say it's in that order. And the order steps done in logarithmic. Meaning Processes is e (2.71821) less than  People. And Technology is 2.71821 less than Process.

This put process 7.29 less than the people.

Want to find the way to increase the probability of a project's success - It's not the tools it's the people.

Project Management Principles from the Department of Energy

The US Department of Energy has $1.2B allocated for clean energy initiatives. Our firm is working a program that captures carbon from coal fired power plants and uses it to feed an algae farm that in turn makes a refineable "oil" (lipid) that can be burned in vehicles or turbines.

Along the way, we are building a Project Management Plan. The DOE has some useful guidelines for what makes a credible project plan.

1. Mission need must be defined and approved by the appropriate management official
2. A range of alternatives to meet the mission need must be considered, developed and evaluated
3. Project objectives must be defined upfront and used to judge project success
4. Project performance risks (technical, financial, and otherwise) must be identified and mitigated in an implementation strategy
5. Projects must be managed by qualified individuals
6. Scope, schedule and budget must be established for each project and serve as the basis for project management
7. Projects must be managed and reported against the established scope, schedule and budget

Any project, no matter the domain or context, would be well served when using these principles as a starting point.

Project Failure Sins and How to Stop These Sins

Thanks to Ron Rosenberg for another useful post about the failure modes of projects. The Lecture can be found HERE

1. The Challenge of Scale
2. Middleware or Fiddleware?
3. Method or Madness?
4. Where are we?
5. Software development is still an art
6. Integration, the final frontier
7. Will it really work?
8. Change our greatest Challenge