Large Projects and Complexity

Jack points us to a post about the slippage of Windows Vista and some possible root causes. These include:

• Too many interfaces
• Too many cooks and their spoilage of the soup
• Optimistic and probably unrealistic schedules

But there probably many more to be uncovered during the post mortem of the project. Since Vista is a large project and most likely has many attributes of a "system," I'll look through my current favorite book The Systems Bible, John Gall, for some guidance. BTW this book is one of those texts that needs to be read and re-read all the time. I've started highlighting and tabbing the areas of importance. At the current rate, the entire book will have yellow highlights and a tab on every page. It's one of those book.

As a small aside, the book has one of theoretical basis of agile development processes. Chapter 29 describes the systems aspects of feedback. Buy the book, read about how complex systems can't be managed in the ways we're trying to manage them.

A Systems View of Complexity

With this article will come a variety of solutions most of which I'd conjecture will have missed some fundamental problems with large projects.

1. The number of interfaces between the work parts of the system correlates nicely with the complexity and results instability, reliability, and survivability of the system - the more interfaces then less desirable the outcome
2. The number of voices defining the needs of the system correlates nicely with the instability of requirements - the more voices the less stable
3. A schedule is different from a plan. A plan addresses the risk adjusted activities needed to produce the desired result. A schedule shows how the plan is to be executed in time.

I'm not qualified to discuss the problems of Vista from a software development point of view, but I do know something about planning high risk, complex large projects. There are several critical success factors for such a project:

1. No schedule is valid if it does not contain both technical and programmatic risk reduction activities, that are explicitly managed "in line" with the product development activities.
   • If the schedule is not risk adjusted, using some form of statistical analysi, then you're late before you start and youprobably don't even know it.
   • If the developers of the schedule do not have a deep understanding of both the statistics of the development performance distributions and the probabilities associated with programmatic and technical risk, then you're late before you start.
2. No plan can survive for very long without calibrating the productivity of the producers of products. This means an "all in" metric of productivity, not just code, or even working code. But "capabilities" of the system in some unit of measure defined by the end users - "can we do out business with the product in a way we thought we could when we bought it or hired youto build it?"
   • Without feedback on the performance of the project in units of measure connected with 100% working deliverables, you'll be driving in the rear view mirror. Tomorrow weather is yesterday's weather.
   • The result is you're late before you start and won't know about it until to get near the end
3. A plan that does not have "maturity assessment" points along the way will reflect the fiction that progress is a function of effort and the passage of time - when in fact progress is ONLY a function of delivered value for a known cost against a planned level of maturity. Otherwise you've spent more money to deliver less value in the same time period. In the EV world this is called being underwater.
   • Aerospace long ago learned this the hard way. The measure of maturity starts with defining the desired level of maturity up front and working the schedule backwards to discover the Significant Accomplishments and the Accomplishment Criteria needed to deliver the planned level of maturity
   • Only then can the work activities be defined to deliver the level of maturity.

Would Any of This Helped Vista?

Can;t really say, but probably not. Because the political aspects of such projects overwhelm the technical aspects. One very nice thing about aerospace projects - and there are disasters there as well - is that the levels of command between the Program Manager (usually a senior VP) and a line engineer are around 3. Not the 12 or so mentioned in the Vista article.

This lays the ground work for clarity of purpose. This does not reduce in anyway the difficulty, but it significantly shortens the communication channel when things go wrong.

One interesting observation that occurred here and in some Blog posts from other MSFT people in the know. It is how much email is used to communicate status and problems. In the two successful projects I've participated in, communication was always face-to-face for important things like status, problems, solutions. There is a cultural bias against email for project management in the aerospace firms I've worked with. Maybe there's something here. Maybe aerospace engineers just aren't very good with email.