Herding Cats

Back To Basics: Increasing the Probability of Project Success

It's time (beginning of the 2^nd Quarter) to remind ourselves of the basics of project success criteria:

1. Identify the Needed System Capabilities - define the set of capabilities needed to achieve the program objectives or the particular end state for a specific scenario. This can be defined in a Concept of Operations document, where the Who, Where, and What are presented. This is a natural place for Use Cases and Scenarios. The activity answers the question: What Capabilities Are Needed to Fulfill Concept of Operations, the Mission and Vision, or the Business Case?
2. Establish the Requirements Baseline - define the technical and operational requirements that must be in place for the system capabilities to be fulfilled. First, define these requirements in terms that are isolated from any implementation of technical products. Only then bind the requirements with the technology.
3. Establish the Performance Measurement Baseline (PMB) - Build a time-phased network of schedule activities describing the work to be performed, the budget cost for that work, the organizational elements the produce the deliverables, and the performance measures showing this work is proceeding according to plan.
4. Execute the Performance Measurement Baseline - Execute the work packages descried in the PMB, while assuring all performance assessments are 0%/100% complete before proceeding. No rework, no forward transfer of work activities to nurture work packages. Assure every requirement is traceable to work and all work is traceable to requirements.
5. Performance Continuous Risk Management - Perform the 6 process areas of SEI's Continuous Risk Management (CRM) for each of the above process areas to identify, analyze, plan, track, control, and communicate programmatic and technical risks.

With these processes as guidance, the probability of project failure is reduced. Not eliminated, just reduced.

Standish Report and Naive Statistics

    The 2009 Standish Report is out. Many Blogs have made comments. All obvious comments. But there is a fundamental flaw in any report of this type:

The concept of a assessment baseline is missing from the conversation.

• Is there a Performance Measurement Baseline?
• Does this baseline represent a credible plan for cost, schedule, and technical performance?
• How would we know how many projects "should" fail in the population of all projects?
• Is the project sample used by Standish representative of the total population of projects?
• If so, what's the correlation between the sample population and the total population in terms of failure rates?

    These are basic statistical analysis questions. Of course the report does not state these for the right business reasons, but the result is now limited in terms of its interpretive power.

The first big question is - Is there a control group?

• How many projects should fail as a matter of course of being members of a underlying  population?
• What is the proper statistical population for projects?
• What is the underlying distribution of project success versus project failure for this population?

    In the absence of this information - being made public - the Standish numbers are just a marketing document for their training and consulting services. In the same way the weight loose, hair growth, lawn growth, and other sales approaches are frankly - Nonsense.

    The easy starting for discussion of this sort is How To Lie With Statistics. This little book wias written in the 1950's and is still an important contribution to the discussion. After reading this short book, you too will be suspicious of Standish style reports.

    Just remember how you feel, when you see and ad for a weight lose products reporting how much weight will be gone in the coming 6 weeks with their product.

    In the end though Standish provides no solution, just an 800-number to call for the "magic elixir"

The Profession of Project Management is a BIG Tent

Tonight's posts on PMHut, brought an interesting contrast

• Pawel Brodzinski speaks about the issues of projects that encounter wrong or missed estimates
• Robert Prieto continues his series on Lessons Learned from Major Projects

Both are in the business of managing projects. Both have experience in their respective domains. Not knowing either personally, I'll assume both are compete practitioners of their profession.
    But the language is on opposite ends of the spectrum. It just struck me as an example of the vast gap between the definition of "project management." I work in the domain familiar to Robert. All the same problems encountered by Pawel. But different responses.
    If one of our clients customers came and said "cut the schedule in 1/2" there would be a huge laughing sound from the highest ranks of the providing organization. If have personal experience watching our commercial side PM's receive the same request and respond with "we'll try every thing we can to meet your request."
    No wonder IT projects have high failure rates. They fail from self inflicted wounds.

Speaking Engagements

Tis the conference speaking season. So far this year I've done

• Denver Milehigh PMI Symposium - Earned Value Management using Deliverables Based Planning^sm
• Colorado Springs PMI Tools Symposium - Integrated Earned Value with Programmatic Risk Management
• Colorado State Department of Transportation - A Gentle Introduction to Earned Value for State Government
• Dallas MPUG - Increasing the Probability of Success, Integrating Processes, People, and Tools
• PMI EVM Work 2009 - Earning Value from Earned Value

I'm done for the season. The next stop is the Microsoft Project Management Conference in the fall in Phoenix.

The Mathematical Side of Managing Projects

One common concept in project management is the three independent elements of a project:

• People
• Processes
• Technology

The people side is being hashed out in previous posts. Processes of PM are debated all the time and technology for managing projects is a hot topic as well.
    There is a orthogonal dimension here. The mathematics of projects. This includes

• Cost models
• Schedule models
• Risk models

Models for people are essentially intractable. The Complex Adaptive Systems paradigm has no closed form solutions to any of the underlying math that has any relation what so ever to the activities and outcomes of project management. At the moment CAS is a sociology paradigm with some possibilities for application.
    But the cost, schedule, and risk models do have tractable forms that are applied in many domains. The sources of materials for these applications include:

• Journal of Cost Analysis and Parametrics from the Society of Cost Estimating and Analysis and International Society of Parametric Analysis
• Journal of the International Council on Systems Engineering
• The Association for the Advancement of Cost Estimating
• NASA Cost Estimating Handbook 2008
• JPL's Parametric Cost Estimating Handbook
• NASA Systems Engineering Handbook

The point of these references is that the process of managing projects is well developed in other fields. Fields that include software intensive products and services. Projects that have emerging requirements, unstable funding, and all the attributes found in commercial IT projects.
    The guidance developed in these domains provides the discipline for the teams to move beyond just software development and into system development. One of the primary failings in IT projects is the lack of understanding of the systems impact of decisions. This is not systems in the sense of hardware and software, but systems int he sense of "Systems Engineering."

Managing in the Presence of Uncertainty

    There are some in the project management and software development community that insist on use Waterfall, determinism, and point estimates as the starting point for the counter argument in support of their favorite method.
    For those sign up for the Uncertainty in Engineering course at MIT OPEN Courseware. This will provide some insight (hopefully) into how engineering and the related projects management profession work in the presence of uncertainty.

    The MIT Open Courseware site has a wealth of resources on a wide variety of topics. Many are project management focused in constuction and aerospace engineering.

Earned Value Issue of SoftwareTech

SoftwareTech is magazine from the Data & Analysis Center for Software, US Department of Defense, Rome Air Force Based, Rome, NY. The current edition is focused on Earned Value Management. There is one article on using earned value concepts in agile projects. Charts for Earned Value and Planned Value are used to show how the agile project is progressing. This approach is fairly benign in that it assumes "level of effort" consumption of labor resources, so no S-Curve is needed. The 0/100 percent completion is the right approach for EV, and the fixed periods of performance make it easy to compute the performance indicies.
    Paul Solomon has an article on "Applying Earned Value Management to Software Intensive Programs," that describes the more robust used of EV, using his Performance-Based Earned Value^tm (PB-EV) approach. PB-EV is one of the approaches we use in our Deliverables Based Planning^sm program management method as it is applied to DoD programs.
    This journal is free from DACS.

The other magazine that arrived in the mail today is Defense AT&L. This is published by the Defense Acquisition University. The DAU site has many useful sources for project and program management as well as the "human side" of these activities. This months edition is dedicated to just that "It's About People."

The Gardening Analogy for Project Management

The garden analogy suggested by Jurgen Applelo is a great concept.

Gardening around the world takes dramatically different approaches. Here in the United States there are many gardening zones. Having grown up in a "gentleman rancher" environment in North Texas (horse, cows, and wheat), then getting deep into organic gardening in Southern California I was very disappointed when we moved to Colorado. In California you can throw seeds in tilled furrow, splash so water to start and pretty much pick fruit or vegetables on the appointed ripening day. Minus the work to pick off the bugs and a bit of fertilizer, gardening in the coastal southern California area is straight forward.

Most everything grows with a little bit of attention to detail. Kind of like developing software with a couple of engineers and a customer sitting by your side. If you competent at the language and tools, you can get something up and running in weeks or at most a month or two. Skill is needed, you have to have a good customer, and be proficient at your job. But you don't need formal methods, management overseeing your efforts, and especially you don't need detailed plans, and regulations to follow. The code and the Strawberries can "emerge" as the season goes on.

Outside of previous Orange County coastal home (48 Sycamore Creek, Irvine CA), there's a different set of constraints on home gardening. How you manage the soil, what plants can be used, what irrigation systems, and a myriad of other external constraints determine the method used and the probability of success.

We have a moderate garden (under 200 square feet) here in Colorado. Gardening in Colorado is hard work. Low rain fall, high winds in the spring and fall, unexpected snow, and large and small animals all waiting to eat the results of your labor. Like projects that are outside the sweet spot of agile development. External customers, changing funding, poor COTS interfaces, rigorous requirements for documentation, and testing.

When we plant in May (it's snowing today - April 17th), we'll redo the soil both mechanically and with amendments (compost mostly). We'll rebuild the fall wooden boundaries (the underlying soil is clay and granite chunks), repair the irrigation (11" rain fall average). The choice of tomatoes foe example must be "tuned" to our climate. We need to protect them from early season frost, birds, and a few pests. High winds and deer need to be repealed. Careful watering around rapidly changing temperatures in the spring. Root protection with compost.

The garden analogy can be transferred to software.

The conjecture that "managers" are hands off for projects if the project is "self" managing makes some sense. Like a garden that has most of the conditions for success built into the environment. Sun, light rain daily, few pests, good air flow, and nice soil. This yields good results with just a little care and oversight. Keep the bugs off, the raccoons out, and the birds away and you've got nice berries in the planned time.

Are all projects "self managing?" That is a BIG question that no one in the agile community likes to talk about. Gardening in Southern California (and Northern Coastal as well) starts with Sunset Magazine. Picture of lush gardens, hanging flowers, and it may be that the commenter's on the Blog work in a domain where "self organizing" is possible or even encouraged. But there are other domains, where "management" as a verb is not only required it is the critical success factor.

What kinds of domains?

• Enterprise ERP
• Industrial controls
• Banking, financial, retail, commercial COTS products

When we generalize in the absence of a context and a domain, confusion about the topic results.

So someone says "the manager needs to be removed from the team..." in what context can this take place? What is the supporting evidence this is the best approach – not anecdotes, but scalable evidence to other domains and context in the absence of the anecdotal person physically being there? What are the constraints on the team for their behavioral outcomes, skill sets to self manage, and what external condition for their success and the success of the project – in the absence of management.

This statement is much too broad and general to be put to any practice use. Much like the statement the "projects are complex adaptive systems." But more importantly these types of statement are "bottom up" anecdotes of most likely "non managers." How would I scale this to a regulatory managed firm – say insurance, health care, engineering, construction and finally aerospace and defense? Sure Agile works well in Health Care and Insurance. There are agile development firms doing this all the time. But, and this is the BIG but, the agile processes are embedded inside HIPPA and other constraint based guidance. They are not "management free" domains. Same for defense. Scrum is a natural process for development of embedded systems. Scrum driven by IMP/IMS and MIL-STD-881A Work Breakdown Structure process. Process mandated by DoD, but implemented in Scrum-centric ways.

In The End The Absence of Context and Domain Creates Nonsensical Conversations

When a broad statement like "self managed teams are best," or "the manager needs to be removed from the team," needs a context and a domain. Defining the interactions with the team (as suggested by Mendelt Siebenga, is the right starting point. The challenge is how to actually determine what this interaction should or could be. Is this determined by those being managed? Partly. By the management itself – this is difficult having been there before. How about by external assessments and suggestions – this is the role of conferences, journal articles, and research.

But my experience – my anecdotes – tells me without a context and a domain in which to place the anecdotal suggestion is like taking the Sunset Magazine gardening zone suggestions for coastal Southern California and applying them to rural high plains Colorado. You're going be disappointed with the results.

Complex Adaptive Systems (CAS) and Projects

    The side discussion of late around Complex Adaptive Systems and their applicability to project work became clear tonight.
    I attended the University of Colorado lecture series, where Brian Greene spoke on String Theory and his book The Elegant Universe. He had this great quote, when encountering someone with a great new idea based on theory ...

This might be a compelling view, but how does your theory make contact with my reality?

That's my question for those suggesting that CAS is the paradigm for project work?

The Mathematics of Project Management

I've an interesting exchange of sorts on Jurgen Appelo's blog. It started with me making a comment about the application of Complex Adaptive Systems to projects and software development. I've somewhat familiar with CAS through several channels - my physics background tool me into deep inelastic scattering in the late 70's.This the use of electrons to look inside protons to see what they contain.

They contain quarks of course, but in those days how many, what are doing in there, and what can we learn from them was still emerging. My job as a lowly grad study was to write FORTRAN code to decipher the pictures on film from the camera. I also worker from there in defense, since I did not have any original ideas required to be a post-doc. Missiles flying at hypersonic speed through varying air densities is sorta like compressible fluid flow, which is a cousin of the math found in CAS adaptive systems.

Of course the science around CAS is not a single science, it is many sciences. Some are physics - compressible fluid flow and turbulence, some is biology, some is social science, and some is the co-opted terms used in the agile community. As you move from left to right in that list, the science is diminished and the hand waving increased. I usually loose interest right at the boundary of the compressible fluid flow equations and the start of hand waving.

So when I mentioned that the application of CAS to projects and software development was "sketchy" in my experience, the agilest on that blog came out and started down the normal path of "how could mathematics possibly be used to manage projects?"

Well here's how math is used to answer several questions:

• When will this project be done?
• What will it cost when it is done?
• What are the "hot" spots in the schedule that we need to look after while we're trying to get done on time?
• What confidence do we have that the cost and schedule numbers are credible?
• How does that credibility increase with the passage of time?
• What can we do to assure the credibility is actually increasing with time?

The answers to these and other question like these starts with a Monte Carlo Simulation (MCS) tool. Monte Carlo tools are probabilistic assessments of dynamic systems. The first Monte Carlo simulations in the modern ear came from nuclear weapons research during WWII. Ulam, Fermi, von Neumann, and Metropolis developed an algorithm to simulate the collisions of neutrons inside the atomic bomb, without having to actual explode the bomb.

The MCS approach to schedule simulation goes like this:

1. Construct a well formed activities network. This means no widows or orphans. Every task has a predecessor and successor, except the first and last task. These are usually Authorization to Process and Program Termination.
2. Assign a probability distribution to the duration of each task. This usually starts with a Triangle Distribution is there is no information about the task. Other distributions can be used is there is knowledge.
3. Assign the Most Likely duration. This is the duration that is "most likely" to take place. This value can be derived in a variety of ways. Too many for this post
4. Assign the Lowest and Highest durations around the Most Likely.
5. Indicate which tasks you'd like to learn something about.
6. Run the Monte Carlo Simulator. We use Risk+ and @Risk for Project. PERTMaster is another.
7. The "watched" tasks create a sampling of completion times through the following means:

• A random duration value is drawn from under the curve for each task.
• This number is entered into the DURATION field of MSFT Project
• Do this for all the Tasks
• "Push" F9 to calculate the network
• Record the finish date for the "watched" tasks in a histogram
• Do this 500 to 5,000 times

After the selected number of "runs" build a histogram of how many times a specific date came up. This is the Probability Density Function showing the confidence that the watched task will complete "On or Before" a specific date.

This approach can show the project management team the confidence in the schedule. If the plan - with the schedule margin - is to complete on or before 10 Nov 2009, and the MCS shows a 14% confidence of completing on or before 10 Nov 2009 - we need to talk.

That's all there is "talk." The MCS doesn't fix this. The MCS doesn't solve the problem. What it does is provide an assessment of the confidence of meeting the planned completion date.

For cost the work is similar. Build a probabilistic model of the cost for each work element. ONLY in LEVEL OF EFFORT projects does Time = Money.
    This is the hidden outcome of agile project management.

• Time boxed scheduling - fixed iterations
• Fixed resource loads - the same number of people for each iteration
• Fungible resources - everyone does more of less the same job

Time = Money.

This is rarely true on any non-trivial project.

Finally the MCS tools can tell us how much margin (cost and schedule) is needed to raise the confidence to say 80%.

1. Build a schedule with no margin tasks on the critical path
2. Run the MCS
3. Determine where the 80% confidence end date is
4. IF the end date is to the left of the planned for (hoped for) date, then see what that gap is.
5. IF it is something non-trivial - say 45 days for a year long project, then you need 45 days worth of margin tasks to hold the planned completion date.

Of this is a notional explanation. Real projects, with real costs and schedule, real risks, and real changing requirements will require more work. But this approach is used every week on aerospace and defense programs that are software and hardware intensive.

So to answer the question on the Blog - where the math model. It's mandated by DID-81650 and several other DFARs (Defense Federal Acquisition Regulations) for programs we work.

Earned Value Means Measuring Progress Against Plan Cost Not Actual Cost

The Project Management Institute  has several Blogs. Michael Hatfield writes about Earned Value - a topic near and dear to my heart and paycheck. I was excited to see the title of the Blog. When I read the content I was confused.  Because the Estimate At Complete (EAC) formula did not contain the cumulative earned value (BCWP) or the cumulative planned cost (BCWS) or the Budget At Completion (BAC), the way the DoD does Earned Value.
    Let's first start with the source materials. Anyone interested in or actually using Earned Value needs to have the DAU Gold Card in the hands at all times. Access to the DAU site gives a Certificate Error. Just proceed with the link and you'll get to the right page. All DoD sites do this for those outside the net. While you're at it, go to the Defense Acquisition University site and start reading the articles on Earned Value, Programmatic Risk, and subscribe to the Defense AT&L Magazine.
    At a minimum in their notebook. We have a scaled down version of the Gold Card on our lanyards, where our picture security badge, the financial calendar, and other "feathers" hang.
    The Gold Card is a summary of the Earned Value principles. Principles that seem to have been missed in the PMI Blog.
    Michael speaks about a quick way to compute the Estimate at Complete (EAC). Something every project or program manager should know on a weekly basis. But first there is some needed background:

• Earned Value is a powerful tool in the hands of the right people. But good numbers must be in place. With these numbers, experienced program controls people as well. These good numbers come from the Performance Measurement Baseline (PMB). Establishing the PMB is a straight forward process. Here's a talk I gave at the PMI College of Scheduling with the same title. There is more to the PMB than described here, but the principles are simple.
• The three variables of EV are ACWP, BCWS, and BCWP. Only two are independent. BCWP is calculated. Forget the PMI notation of AC, PV, EV. The DoD world has the first notation baked into every form and every ERP system on the planet. Learn the notation.
• Finally, program and project managers are most interested in the Schedule Performance Index (SPI). Cost analyst are interested in the Cost Performance Index. So measuring the performance of a project through the Actual Cost for Work Performed (ACWP) is not allowed under the DCMA (Defense Contract Management Agency) guidelines. Cost Variance is reported, but the Control Account Manager (CAM) lives by the schedule performance.

Simple Formulas Are Just That Simple - Maybe Too Simple
    In the PMI Blog post, there is a formula for EAC (Estimate at Complete)

EAC = ACWP(cum) / Percent Complete

ACWP(cum) is the cumulative ACWP (Actual Cost of Work Performed). This is the cumulative cost of the work to date. In other words "how much money have I spent so far?" The Percent Complete is a measure of the physical progress. BUT this measure is not against the money spent to date, but against the "planned" money that SHOULD have been spent to date. This value is the Budgeted Cost for Work Scheduled (BCWS). Measuring against ACWP provides little or no insight into the project performance, other that the expenditure of money. And as we all should know spending money has little to do with progress other than spending money.
    The gory chart below shows that using ACWP can be a forecast of the future, ONLY if you have spent exactly what you have planned to spend. This is almost never the case unless the project is a Level of Effort activity.
 
Guidance for Applying Earned Value

1. Determine what money should have been spent at a period of time (the period of performance). This is BCWS.
2. Determine the physical percent complete at that period of time. This should - this MUST - be evidenced by some physical deliverable, that is 100% complete. No almost done, no "we'll be done on Monday," no rework, no bugs. It's either done or not done. Any other result is unfunded work past the due date. You're know late AND over budget.
3. The "earned value" is calculated by the Planned Cost (BCWS) x The Physical Percent Complete. This the BCWP - The Earned Value.

With these values the Estimate at Complete can be calculated using:

EAC = ACWP(cum) + [(Budget at Complete (BAC) - BCWP(cum)] / Cost Performance Index (CPI)
CPI = BCWP / ACWP

The notion that:

EAC = ACWP(cum) / Percent Complete

only works if the spend rate is linear and therefore ACWP = BCWP. And that's the definition of Level of Effort.
Actually LOE is defined as

BCWS=ACWP

Why We Need Professional Project Managers

The Federal Acquisition Streamlining Act, Title V states, “the head of each executive agency should achieve, on average, 90 percent of the cost and schedule goals established for major and nonmajor acquisition programs of the agency without reducing the performance or capabilities of the items being acquired."

And how might this goal be achieved in the commercial sector? Achieved in a software development domain?
    Who keeps score for the engineering side software development activities? The developer themselves? This is poor management practice in aerospace and defense. Optimism is the disease, feedback is the cure is a Kent Beck quote.

Project Managers that's who

The notion that project management "gets in the way," is both naive and uninformed.

The Peter Principle Lives

Bob Sutton's Blog has a nice piece applicable to nearly every malady in today's headlines, from the financial crisis to failed project management.

In Praise of Simple Competence

Opinion on PMBOK(r)

Dr. Jame Brown has an opinion piece on the current state of PMBOK(r) 4th Edition. I've read and reviewed Dr. Brown's book The Handbook of Program Management. The book is a "must read" for any serious project or program manager. The PMBOK(r) article has a critically important point

Project Management is nothing but structured organized common sense!

This is one of the best statements about project management I've heard in awhile.