Herding Cats

Business Analysis Body of Knowledge

If you'd like to see an example of how a Body of Knowledge can be build for a specific domain that includes planning look at Business Analysis Body of Knowledge and scroll down to Page 100 for the planning section.

The difference between planning and scheduling can be seen more clearly.

Assumption Based Planning

Just finished Assumption Based Planning: A Tool for Reducing Avoidable Surprises, James A. Dewer, Cambridge University Press, 2002. In it was a powerful statement regarding planning and the difference between planning and scheduling

A plan is a tentative solution to the inexact problems posed by an uncertain future.

Even if the future could be known with certainty (which is unlikely), there are likely to be uncertainties about whether a given plan is the proper solution or approach to that future. Any plan, contains assumptions bout about what the future holds (the problems in the future) and about whether the plan will perform as hoped (the solution to those problems).

Planning is Not The Same As Scheduling

The schedule has found in most project management processes is not the same as a plan. The schedule is the list of tasks, their duration and their order for the work needed to implement the plan. The aerospace community uses the Integrated Master Plan / Integrated Master Schedule paradigm to connect these two. But the traditional project management guidance does not make use of this paradigm.

Instead, the current guidance for scheduling that is out for review by PMI, fails to separate planning from scheduling. And therefore project failures will continue unabated. Good news for our strategy consulting firm, bad news for the project stakeholders and funders.

Take a look at this book, along with a similar book also from RAND on Capabilities Based Planning, to see where the traditional (and official) source of project management falls into trouble from the start.

Missing Semantics of Project Management

A had a coffee on the way to a clients this week with a friend and colleague who teaches at CMU-West in the Project Management Master Program. He mentioned an interesting concept. The current traditional PM literature does not have the necessary "words" to have a proper discussion of how to successfully deliver a project. The PMI process areas are necessary but far from sufficient in this effort.

The fundamental problem with the proposed approach with the PMI Scheduling Guidance is that it starts with the assumption there is a plan. While all the steps in Figure 2.2 of the Practice Standard for Scheduling may be necessary, they represent a primary failure mode of almost all traditional scheduling processes - the plan is missing.

I'll hopefully get to the making a review of this guidance sometime soon, maybe in time for submission to PMI.

Large Projects and Complexity

Jack points us to a post about the slippage of Windows Vista and some possible root causes. These include:

• Too many interfaces
• Too many cooks and their spoilage of the soup
• Optimistic and probably unrealistic schedules

But there probably many more to be uncovered during the post mortem of the project. Since Vista is a large project and most likely has many attributes of a "system," I'll look through my current favorite book The Systems Bible, John Gall, for some guidance. BTW this book is one of those texts that needs to be read and re-read all the time. I've started highlighting and tabbing the areas of importance. At the current rate, the entire book will have yellow highlights and a tab on every page. It's one of those book.

As a small aside, the book has one of theoretical basis of agile development processes. Chapter 29 describes the systems aspects of feedback. Buy the book, read about how complex systems can't be managed in the ways we're trying to manage them.

A Systems View of Complexity

With this article will come a variety of solutions most of which I'd conjecture will have missed some fundamental problems with large projects.

1. The number of interfaces between the work parts of the system correlates nicely with the complexity and results instability, reliability, and survivability of the system - the more interfaces then less desirable the outcome
2. The number of voices defining the needs of the system correlates nicely with the instability of requirements - the more voices the less stable
3. A schedule is different from a plan. A plan addresses the risk adjusted activities needed to produce the desired result. A schedule shows how the plan is to be executed in time.

I'm not qualified to discuss the problems of Vista from a software development point of view, but I do know something about planning high risk, complex large projects. There are several critical success factors for such a project:

1. No schedule is valid if it does not contain both technical and programmatic risk reduction activities, that are explicitly managed "in line" with the product development activities.
   • If the schedule is not risk adjusted, using some form of statistical analysi, then you're late before you start and youprobably don't even know it.
   • If the developers of the schedule do not have a deep understanding of both the statistics of the development performance distributions and the probabilities associated with programmatic and technical risk, then you're late before you start.
2. No plan can survive for very long without calibrating the productivity of the producers of products. This means an "all in" metric of productivity, not just code, or even working code. But "capabilities" of the system in some unit of measure defined by the end users - "can we do out business with the product in a way we thought we could when we bought it or hired youto build it?"
   • Without feedback on the performance of the project in units of measure connected with 100% working deliverables, you'll be driving in the rear view mirror. Tomorrow weather is yesterday's weather.
   • The result is you're late before you start and won't know about it until to get near the end
3. A plan that does not have "maturity assessment" points along the way will reflect the fiction that progress is a function of effort and the passage of time - when in fact progress is ONLY a function of delivered value for a known cost against a planned level of maturity. Otherwise you've spent more money to deliver less value in the same time period. In the EV world this is called being underwater.
   • Aerospace long ago learned this the hard way. The measure of maturity starts with defining the desired level of maturity up front and working the schedule backwards to discover the Significant Accomplishments and the Accomplishment Criteria needed to deliver the planned level of maturity
   • Only then can the work activities be defined to deliver the level of maturity.

Would Any of This Helped Vista?

Can;t really say, but probably not. Because the political aspects of such projects overwhelm the technical aspects. One very nice thing about aerospace projects - and there are disasters there as well - is that the levels of command between the Program Manager (usually a senior VP) and a line engineer are around 3. Not the 12 or so mentioned in the Vista article.

This lays the ground work for clarity of purpose. This does not reduce in anyway the difficulty, but it significantly shortens the communication channel when things go wrong.

One interesting observation that occurred here and in some Blog posts from other MSFT people in the know. It is how much email is used to communicate status and problems. In the two successful projects I've participated in, communication was always face-to-face for important things like status, problems, solutions. There is a cultural bias against email for project management in the aerospace firms I've worked with. Maybe there's something here. Maybe aerospace engineers just aren't very good with email.

Big Dig Project Update

Big Dig Project update provides some sources of the current woes of the Big Dig.

BTW Scott Sehlhorst provides some of the best Blog materials in the general software and project management domain around.

Waterfall and Agile (again)

Mishkin Berteig suggests that the differences between agile and waterfall are similar to the differences between a bank loan and a Visa cash advance. The trouble with this analogy is one loan is secured and one loan is unsecured. The cash advance from Visa is an unsecured debt, where the bank loan requires some sort of collateral for security - ranging from your house for a mortgage, to the car in an auto loan.

But that is not really the point. The continued comparison between agile and "waterfall" fails to make clear the differences between agile and other approaches to development.

The granularity of the deliverables have varying sizes. Small granularity in the deliverables needs to have a matching small granularity buckets in the receiver of the deliverables. If not there is a mismatch - an impedence mismatch between production and use.

If I can't use the rapidly produced results then they have less value than the advocates would suggest.

Agile software development process produce outcomes on fine-grained boundaries - weeks. Development processes that produce outcomes on larger grained boundaries are less agile for one simple reason - the feedback sample time is larger and therefore the interval in which the system can respond to change is large.

If the development of software is seen as a "system" in the systems engineering or systematics sense, then the role of feedback becomes clear. There are several important points though.

• Just calling it feedback doesn't mean it is feedback - it's not feedback until the system changes its direction.
• Feedback intervals need to be tuned to the needs of the system - the rate at which feedback is made must match the response characteristics of the system.

With this background the comparison between agile and something else MUST address the needs of the system. One topic currently making the rounds in the aerospace business is the use of weekly earned value. One might assume that have earned value numbers on a weekly basis is better than on a monthly basis. This may or may not be true and depends on the underlying development processes.

A similar example is the delivery of working software on a bi-weekly basis versus longer intervals. The value of this software to the end-user depends on whether the "system" can make use of the this software. What is needed in any discussion of agile software development is to first identify who is receiving this software on each boundary.

• Fine grained - build working code, check it into the repository and 100% complete. Fine grained receivers are most often QA and CM.
• Moderate grained - assemble the working components for integration testing. Moderate grained receivers are often test, tech pubs, intermediate V&V.
• Large(r) grained - deliver the "system" to the user on some boundary that matches the needs of the business. Large(r) grained receivers are often Beta, small user communities, limited production.

These distinctions never seem to enter the conversation. The problem with the simple examples of Visa versus Banks is they never really address the problem of software (or for that matter any engineered product) delivery - what is the appropriate interval for sampling the performance of the project to gain corrective feedback for its course of progress? Over sampling a digital (in this case the finite interval deliverables) system creates noise. Noise drives the system into oscillation in undesirable ways. "Chasing ones tail" is a good analogy. The feedback gain in an analog system has simialar outcomes.

One way to view the problem (and this is a systematics approach) is to invert the problem. "Who is the receiver of the resulting outputs?" Once this is determined, then the appropriate receipt interval can be determined. Questions like:

• How fast can the receiver receive partially working product?
• When received, how fast can the receiver provide feedback for corrective changes in the plan?
• At what rate can changes be tolerated in the production system to avoid oscillations in the underlying processes?

Continuing to compare agile with waterfall provides no value, since it hides the need to determine the appropriateness of the sampling interval and the ability of the "customer" to receive value from the output of the agile process.

As Kent Beck suggests - in software development optimism is the disease, feedback is the cure. Well maybe not the cure, but feedback can be the diagnosis. The cure comes from making changes as a result of the feedback. Changes that actually change the outcome of the project.

My Summer Vacation

Remember that writing assignment in the 6th grade - "what I did on my summer vacation." Our two teenagers, their mother and I went to France and the UK. Not that that has anything what so every to do with project management - other than the "project" of getting there, getting around Paris, London, Somerset, and Northampton with bags, cars, a 15 and 16 year old and not enough understanding of the basic cost of traveling, eating and sight seeing.

What did come to mind is that there are huge construction projects in both the UK and France. Cranes nearly block out the skyline in London. Major road work on the motor ways. I was an experienced UK driver 15 years ago, and my skills came back in a few hours.

We have cars that have navigation systems here in the US. Honda's. They're OK, but not great. Weak UI and sometimes illogical behavior. The navigation system in our rented Ford Mondeo had a Blaupunkt system without a moving map. It spoke to us, giving clear and concise commands when to exit a round-about, when to leave the motorway, when we had arrived at our destination. If with our Ordnance Survey detailed maps, there is no way we could have found our way to many of our destinations. A Hamlet is 10 houses or less, near a Village, which is near a Town. Our two friends both live in Hamlets and we'd been lost from the get go with this very clever system.

What's important about the project management aspects of the trip started with a sign at Heathrow. As you exit the Hertz lot heading for the motorway, there is a very large billboard describing the "on time / on budget" scheduled completion in March of 2008 for a total cost of 4.2B Pounds Sterling of the new terminal - Terminal 5.

BAA is the prime contractor with a small number of direct subcontractors. 2nd and 3rd tier subcontractors are the responsibility of the 1st tier subs. What is most important if you look through the Terminal 5 website, is the emphasis on risk management. The prime (BAA) accepts most of the risk which enables the subcontractors and suppliers to focus on:

• managing out the cause of problems, not the effects if they happen
• work in truly integrated teams in a successful, if uncertain environment
• focus on proactively managing risk rather than devoting energy to fight litigation

This risk based approach is similar to the Rocky Flats contract through the prime contractors - CH2M Hill and ICF Kaiser.

What Do These Approaches Mean for Project Management?

The traditional approach of trying to manage uncertainty does not work very well. Managing in the Presence of Uncertainty is a better approach. Terminal 5 certainly has uncertainties. RFETS was nothing but uncertainties, since such a project had never been attempted before.

Managing in the presence of uncertainty requires a deep understanding of the risks, their mitigation. Both technical and programmatic risks. Statistics and Probability need to be understood. This turns out to be critical to the success of a risk management approach to project management.

The difference between probability and statistics is shown in the figure to the left. What the project manager is interested in are things like "what is the probability that a task will complete on or before a specific date?" What is the probability that this completion will cost less than our equal to some amount of money?

In order to answer that question the underlying task completion times and cost must be understood - or guessed in some reasonable manner. With both of these pieces of information, the project management (or program office) can provide information about the "probable" performance of the project.

All of this comes back to a recent post by Jack on the difficulties MSFT is having getting Vista out the door. I know knowing of their problems, but I might conjecture that if the schedules are being "cooked" they entered into the realm of confusing probability with statistics and they have replace statistical estimates with "hope." I have a great book on my shelf Hope is Not a Strategy. It is claimed that

When General Custer was completely surrounded, his chief scout asked, "General what's our strategy?" he supposedly responded, "The first thing we need to do is make a not to ourselves - never get in this situation again."

Even if this were not true (an urban legend) it still says much about projects that are headed for disaster. I'm not suggesting Vista is one, or that SBIRS is one. I'm not in any position to say for those projects. But I been on and have recently advised projects that are headed for disaster. Their "hope" is things will get better, progress will be made in ways it has not been made in the past. That the undiscovered risks of the past are in the past and the schedule now reflects the reality of the future. And most of all these projects "hope" that the laws or probability and statistics will not come into play on their project before completion.

Risk Management is How Adults Manage Projects

If there is not a risk adjusted, risk mitigated plan in place, that is continuously updated with new information at a rate that mathces the availability of this information (every 2 weeks for an XP project, every month for a FAR 15.2/DID 81650 project) you're late before you start.

Change of Venue

I've moved from one consulting firm to another. My tenure as a Program Architect completed with the submission and transition to Phase I of a major spacecraft procurement initiative. This experience was a wonderful opportunity to expand my experience in Program Management, risk assessment, and general management skills. Many of the processes used in these programs are directly applicable to IT and software development.

I've taken a position as a Practice Director in a local consulting firm Lewis & Fowler. My focus is on developing a Balanced Scorecard and Performance Management practice for IT organizations. Parallel to this practice is an existing group delivering program and project management services.

As agile software development moves into corporate IT departments, the management processes surrounding Scrum and XP become more important. IT Governance, Corporate and IT strategy, and program performance are the baseline principles by which boards, governance committees and stakeholder judge the some "value" produced by IT departments.

I'll be focusing on these areas as well as core project management as time permits.