Herding Cats

Oh Canada!

I'm married to a French Canadian. The Olympic win of Canadian is a mixed emotion of our house.

The country that invented hockey, put in the "best team won," performance

O Canada!
Our home and native land!
True patriot love in all thy sons command.
With glowing hearts we see thee rise,
The True North strong and free!
From far and wide, O Canada,
We stand on guard for thee.
God keep our land glorious and free!
O Canada, we stand on guard for thee.
O Canada, we stand on guard for thee.

Ô Canada!

Terre de nos aïeux,
Ton front est ceint de fleurons glorieux!
Car ton bras sait porter l'épée,
Il sait porter la croix!
Ton histoire est une épopée
Des plus brillants exploits.
Et ta valeur, de foi trempée,
Protégera nos foyers et nos droits
Protégera nos foyers et nos droits.

Quote of the Day

I almost wish I hadn't gone down that rabbit-hole - and yet - and yet - it's rather curious, you know, this sort of life!

- Alice

Standish Report

I've written several times about my personal dislike of the Standish Reports.

• Standish Report and Naive Statistics
• Finally, A Challenge to the Standish Report
• Project Failure Rate

The current assessment of the Standish Report is:

"The Rise and Fall of the Chaos Report Figures," J. Laurenz Eveleens and Chris Verhoef, Vrije Universiteit Amsterdam, IEEE Software, January/February 2010, pp. 30-36.

The core findings are:

• There are misleading definitions - the definitions used by Standish are based solely based on estimation accuracy of cost, time, and functionality. Standish labels projects as successful or challenged, suggesting much more than deviations form their original estimates.
• Unrealistic Rates - estimation accuracies are questionable. These estimates are "one sided" and neglect under runs for cost and time and overruns for the amount of functionality delivered for either the under or over runs on cost and schedule. Institutional biases exist and are unaccounted for in the Standish numbers. (see Software Engineering Economics, Barry Boehm, Prentice Hall, 1982)
• Cost Forecasting - using independent cost forecasting databases, it can be shown that the Standish number forecast a 59% success rate for cost adherence, when the baseline database from actual projects - 140 projects from 2005 to 2006 at a large financial services firm - follow Boehm's "conical shape" for accuracy and produce an estimating quality factor of 8.5. This means that half the projects have a time-weighted average deviation of 12% or less from the actual forecast.
• Accuracy Again - using their definitions of cost and schedule overestimation and large functionality underestimations perverts the estimation accuracy.

All of this information and the references point to a statistical assessment gap. The core problem is:

• There is two-sided measurement from a statistical point of view. They only ask about failure rates, not the success rates. This would produce a biased outcome.
• There is not a statistical sample selection process. Send out a survey, get back the responses, and total the number. How many surveys are needed to establish a credible sample size in order to establish the confidence intervals.

The Real Web 2.0

While the notion of PM 2.0 is still bouncing around, Web 2.0 impacts on procurement in the Pentagon is the title of an article in the current edition of Defense AT&L. "Defense Acquisition Enterprise 2.0," speaks to the use of Web 2.0 technologies in the acquisition process.

The message is "technologies can generate innovative methods to develop and field capabilities sooner by allowing those in the acquisition world to cut across functional stovepipes and better collaborate with the operational communities."

Tis was the case in the early days of the Crew Exploration Vehicle program where the Digital Environment was deployed to connect NASA, the Prime contractor, and the many subcontractors. These tools were crude compared to today - the dreaded "Live Link" product for example, was slow, cumbersome, and many times simply didn't work.

But just to remind those thinking collaboration is the same as project (or program management in this case) - it is not. The detailed face to face conversations needed to actually "manage" the program can never be replaced, with email, IM, blogs, wiki's social networks and the like.

The speed of information flow supports the management of the program. It is not a replacement for the hands on program management processes. It augments these processes only after they are functionally proficient.

In the same issue is LtCol Dan Ward's "Faster, Better, Cheaper" article - which was pretty much a "bad idea."

What is a 1 Inverse FemtoBarn Anyway?

An Inverse Femtobarn is a measure of the number of particle collision events per femtobarn. A barn is defined as 10^-28 m². A 100 square femtobarns is approximately the cross section of a Uranium atom.

If units of measure are well defined in this domain, why can't we have defined units of measure in project management, software development, and general IT processes? I reading Johanna Rothmann's new book about Portfolio Management, with the intent of writing a review - since I was sent the book from the publisher.

The first thing that pops out from the Rothman book is the management of "value" in the portfolios. Not definition of value, but the agilest use the term value when they say "increasing value." I like the book so far, absent these units of measure and an orthogonal definition of Portfolio Management from all other portfolio management books.

So what does it mean to increase value? Value to who? in what quantities? 

No wonder we get confused about the activities in project management. In the classic PM 2.0 presentation stating Enterprise 2.0 solutions are "easy to use," "saves time," and "works everywhere." Everywhere like at the bottom of the ocean. saves time, like billions of seconds, or just a few. Easy to use, like my mutt rescue dog can use it?

This is a core problem - without definitive units of measure, measures of performance and measures of effectiveness in my Systems Engineer Pooh brain, the conversation about increasing the probability of success is just noise.

Quote of the Day - Any New Version of Any Thing

• Politeness is the poison of collaboration
• Science is a method to keep yourself from kidding yourself
• Don't undertake a project unless it is manifestly important and nearly impossible

- Edwin Land

Project Management Blogs

The are many lists of project management Blogs. Some actually contain Blogs that talk about project management.

The Construction Management Degree blog has a list of blogs that speak to the management of projects. I'm happy to say I was selected to be in the company of several top notch project managers who write Blogs - Paul Ritche and Hal Macomber.

Software Engineering Method and Theory (SEMT)

Software Engineering has taken a turn for the worse since I left that field back in the mid 1980's. My last software job was on the team that developed a fault tolerant programmable controller for the emergency shut down, gas turbine, fire & gas system, and general mission critical controls domain.

Software Engineering Method and Theory is a group of thought leaders and supports reforming software engineering based on a solid theory, proven principles, and best practices. This is the domain I work in. The clients we work for "engineer" software intensive systems, most of which would be considered "mission critical."

Edwin Land has a good quote, applicable here:

Don't undertake a project unless it is manifestly important and nearly impossible.

If there are projects that are not mainifestly important and nearly impossible, why bother with all the formality?

Estimating

The estimating process for project cost and schedule is usually one of the poorest skills for the project. Not because it is difficult or even because it is not well understood. But because there is a myth that estimating is not really possible.

 

Estimating cost and schedule requires skill, experience, and a method (there are many) and some tools. Here's how it starts:

• You must have some notion of what done looks like. This can be "done" for this iteration. Or "done" for this rolling wave. Or "done" for this release.
• You must be capable of controlling the scope of "done" within the period of performance of the development process.
• You must measure productivity - the capacity for work - in some unit that allows you to forecast future performance.
• You must be able to engage everyone in a conversation about estimating the future using the past.

In the absence of these basic capabilities, estimating is simply not possible. The the suggestion that estimating in the software development world is not possible, will come true.

Project Management Principles Need To Have Units of Measure

PM Hut has a list of Project Management Principles. Good stuff, but no units of measure. How would we know we're being successful with these principles? No units of measure means the principles are hollow.

OK, maybe number 2 has evidence that it is being met - a RAM is hanging on the wall.

Here's the 5 principles that have served me well over the past 30 years along with their units of measure and the tangible evidence that these units are meaningful to the project participants

1. Where Are We Going? 

We have a master plan that shows the increasing maturity of the product or service being produced by the project. This Integrated Master Plan states the strategy of how we are going to reach each level of maturity, the accomplishment needed to reach this level, and the criteria by which we will judge these accomplishments

2. How Do We Get There?

We have an Integrated Master Schedule, showing the sequence of work needed to produce the accomplishments. This master schedule is at the Work Package level, with resources, budget allocations, and defined outcomes for each Work Package.

3. Do We Have Enough Time, Resources, And Money To Get There?

Now that we know the planned duration, planned budget and resource requirements, do we actually have enough of those items? If not can we get them? If not, then what changes do we have to make to the cost, schedule, and resources?

4. What Impediments Will We Encounter Along The Way?

Do we have a risk plan? Are the retirement or mitigations of the risk embedded in our Master Schedule? Do we have a formal risk management process? Are we following this process? Are the risks being retired or mitigated?

5. How Do We Know We Are Making Progress?

Do we have tangible evidence we are making progress to plan? Making progress is NOT measured by the consumption of resources (money) and the passage of time. It is ONLY measured by the production of tangible items at the time they were planned to be produced.

All measures of performance MUST be in units meaningful to the project stakeholders. This means customers, participants, and suppliers. No units of measure, no measure of progress to plan.

No Units of Measure - You're driving in the dark with the lights out

Is This Really How PM 1.0 Works?

Once again Andrew Filev - through Elizabeth Harrin - has a description of PM 1.0 straight out of the 1980's. This little story of how projects are chartered and then managed is reminisent of how projects were chartered and managed before personal computing.

It was not that long ago that a printed out Project Charter would be the start of project approval. The key stakeholders would physically sign the document, which would be passed in the internal mail between parties, finally returning to the project manager to update the version control for the document to version 1.0. She would then file it away for safe-keeping and proof that the initiation phase was over and that the real work could begin.

If "not long ago," means 1988 maybe.In 1988, I was the director of software development for a Fault Tolerant Process Control company, where we had world wide operations for our real time control system. All specifications, project plans, correspondence, and procurement approvals were done over email.

Steve Garfein essentially invented the notion of EDI for defense procurement on the AH-64 in 1974. HP-9000's used to exchange contracts, specifications, maintain configuration control and the Master Schedule for the Apache in Hughes Fullerton plant. Steve - a long time colleague - is an emeritus member of PMI, EPO of the Apache, Program Manager for numerous mission critical programs from defense to Dream Works.

He'd be surprised to hear that "not long ago," PM's filed the charter in a file cabinet. So would the 100's of PMs at TRW, Northrup, Boeing, Lockheed Martin, and other government and defense contractors.

Maybe the PM 2.0 folks need to get out more and see how all this talk of PM 2.0 has been in place for 3 decades

 

Couples Story

A needed diversion for a cute story ^†...

When our lawn mower broke and wouldn't run, my wife kept hinting to me that I should get it fixed. But, somehow I always had something else to take care of first, the truck, the car, playing golf. Always something more important to me.

Finally she thought of a clever way to make her point. When I arrived home one day, I found her seated in the tall grass, busily snipping away with a tiny pair of sewing scissors. I watched silently for a short time and then went into the house.

I was gone only a minute, and when I came out again I handed her a toothbrush. I said, 'When you finish cutting the grass, you might as well sweep the driveway.'

The doctors say I will walk again, but I will always have a limp.

† - from a colleague working PP&C on a manned space flight program

Who's doing what on this project?

On an agile forum there is a discussion about roles, Business Analyst, and other type roles and what these people do on software development projects.

This of course is backwards, like many discussions that fail to return to "first principles." It's the deliverables that the project produces, not the effort performed by roles or even by the people.

Deliverables Are King

So how can we connect deliverables with people and processes? Simple the Responsibility Assignment Matrix (RAM). The RAM is really called the Accountability Assignment Matrix, since accountability is all about determining who is going to cause something to appear.

The RAM defines the deliverables and the people accountable for them. Where do the deliverables come from? The Work Breakdown Structure of course. Don't do WBS on your project, I know you're an agile shop and agile shops don't do WBS's. Not true of course. What are those 3x5 cards stuck on the left hand side of your cork board. They are the deliverables. They have relationships with other cards pending or complete.

This is not a very good WBS of course, because the cards are not thought through to the end in many cases. But it's a start.

With a "real" WBS, which does think through the deliverables to the end, the terminal nodes or maybe the parner of the terminal nodes are the columns in the RAM. The rows are the names of the people accountable for making those deliverables appear at the needed time in the project.

The F.I.S.T. Principles

Dan Ward is one of my new favorite Blogs. Dan is a frequent author in the Defense Journal AT&L.

His F.I.S.T. is Fast, Inexpensive, Simple, and Tiny.

I've restated them here for effect:

• A project leader’s influence is inversely proportional to the project’s budget and schedule
• Creative constraints foster creativity
• Adding time and/or money generally does not improve outcomes
• Fixed funding and floating requirements are better than fixed requirements and floating funding
• Complexity is a cost
• Complexity reduces reliability
• Simplicity scales
• Complexity doesn’t
• An optimal failure costs a little and teaches a lot
• When FIST projects fail, they fail optimally
• Iteration drives learning, discovery and efficiency
• FIST is iterative
• Talent trumps process
• Teamwork trumps paperwork
• Leadership trumps management
• Trust trumps oversight

Quote of the Day

Yet all experience is an arch wherethro'
Gleams that untravell'd world whose margin fades.
For ever and for ever when I move...
- Tennyson - Ulysses

What is a Project Plan?

I'm working two programs in parallel - a really bad idea by the way. One is a DOE R&D project. The other is a DoD integrated battle management system.

Working in these domains produces a bias towards what is means when someone says "project management." This is an ongoing discussion for several reasons, not the least of which is how these projects answer the questions:

1. Where are we going?
2. How do we get there?
3. Do we have enough time, resources, and money to get there?
4. What impediments will we encounter along the way?
5. How do we know we're making progress?

In my domain these questions are answered through the Performance Measurement Baseline and several work processes used to execute the project with that PMB.

The PM 2.0 Examples of Projects

If I look at examples of processes used for PM 2.0, they are primarily "list management" tools. There are many examples of these. Ranging from email parsers, the task dashboards, to Share Point addins.

So here are some questions:

1. What types of projects can be managed with "list management" processes? These include assignments, due dates, narratives of the work on the list. All the stuff that the list management tools provide.
2. What are the limits of the communication channel using email, IM, and other Web 2.0 tools?
3. What is the evidence that the 5 questions above are being answered in sufficient fidelity to determine the project is moving toward success?

Looking at the users of BaseCamp for example, are they managing "operations" or are they managing "projects." When you look at their statements, many of the examples are "operational" in nature - ad agencies, and the like. Check list style projects.

I'm starting to see the target for PM 2.0 tools - projects without formal cost, schedule, and technical performance attributes.

Quote of the Day

One experiment trumps A 1,000 analytical assessments
- Lead Investigator, Department of Energy, Algae Bio Fuels, program

The notion that "books" can tell us what to do, also needs to evidentury materials that the ideas in the books actually result in visible beneficial outcomes. This is the case for all project management and software development processes and methods.

We've moved beyond the point of having the guru's tell us to "follow me grasshopper."

Bookable, tangible evidence that the suggested approach is now needed. This is especially the case for the band wagon of PM 2.0 tools.

I love the quote from Ryan Endres's previous comment here.

PM 2.0 is nothing more then a new communication tool. Will 2.0 help with the 4 area's above? Maybe or maybe not, but it may open up communication to help discuss about them.

This new communication tool has 3 conditional "mays" before we can ask how it improves the probability of success.

PM 2.0 Communications Channel Bandwidth Requirements

What happened to the agile manifesto's first statement?

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

1. Individuals and interactions over processes and tools.

Are we really managing other peoples money by sending texts, chatting on IM's and exchanging and parsing emails around?

No meetings, no "war room," no big visible charts, no "team daily stand ups for the Scrum Team, no planning sessions, with cards and hand waving on the white board, no customer briefings?

According to Claude Shannon - effective communications is dependent on the channel bandwidth and the symbols sent over that channel.

In the late 1940s Claude Shannon, invented a mathematical theory of communication. His motivation was how to design telephone systems to carry the maximum amount of information and how to correct for distortions on the lines.

His approach introduced a simple abstraction of human communication - the channel. Shannon's communication channel consisted of a sender (a source of information), a transmission medium (with noise and distortion), and a receiver (whose goal is to reconstruct the sender's messages).

Shannon introduced the entropy rate, a quantity that measures a source's information production rate and a measure of the information carrying capacity, called the communication channel capacity. He showed that if the entropy rate, the amount of information you wish to transmit, exceeds the channel capacity, then there were unavoidable and uncorrectable errors in the transmission.

This is intuitive enough. What was truly surprising, though, is that he also showed that if the sender's entropy rate is below the channel capacity, then there is a way to encode the information so that it can be received without errors.

This is true even if the channel distorts the message during transmission. Shannon adapted his theory to analyze ordinary human (written) language. He showed that it is quite redundant, using more symbols and words than necessary to convey messages.

Presumably, this redundancy is used by us to improve our ability to recognize messages reliably and to communicate different types of information. In the study of complex systems today, we view deterministic chaotic processes as information sources and use Shannon's entropy rate, as adapted by Kolmogorov and his student Y. Sinai in the late 1950s, to measure how random a chaotic system is.

Now for Project Management Communication Channels

When we revisit the first agile manifesto statement - individuals and interactions over processes and tools - we can see that the communication channel capacity and the encoded information sent over that channel are critical to the understanding of the message.

So why reduce the channel capacity on purpose. Why restrict the number of symbols that can be sent over that channel (128 characters for example). Why not increase the channel capacity and increase the number of symbols. Face to face, pictures, hand waving, tonal inflections, body language, hands on touchable outcomes that demonstrate progress?

I have a speculation, OK several speculations:

1. Those suggesting the "next big thing" is a narrow half-duplex pipe with a limited character set are infatuated with the technology of these gadgets - IM, Twitter, and the like. It's part of their culture. They grew up with these tools. They think everyone will like them.
2. The projects they have applied these concepts to are their own projects. IT projects for the most part. They haven't worked construction, defense and space, energy, petrochemicals, pulp and paper. Projects like that.

In these project domains - and those domains where agile has succeeded - face to face, hand waving style communications dominates. Large enterprise IT has the same channel capacity needs. Possibly those 2 to 3 person 3 month projects that make up the vast majority - by count - of IT projects, such a narrow channel communication is effective.

But of course those showing the histograms of the number of projects fail to monetize those projects in the population of all projects. One large flight software program for a major weapons systems, swamps 100's of small 2 and 3 person IT software development projects. It's just bad statistics when such numbers are used

As Dr. Phil might say (had he read Shannon), "how's that - narrow bandwidth - limited character set channel - working for you?"

The WBS and Estimating

Craig Brown and Mike Clayton have a related posts on the Work Breakdown Structure and estimating.

Building a credible cost or schedule estimate is a "process" focused effort. Let's start with the guidance documents we use:

• The GAO Cost Estimating and Assessment Hanbook
• NASA's Cost Estimating Handbook

While these guides appear to be applicable to only large government contracts, they have much to say about the process of cost and schedule estimating for all projects. The fundamental concpet is.

All variables in cost and schedule estimates are random. Knowing the statistical properties of these variables is a critical factor in producing a credible estimate.

Without this understanding all estimates are untrustworthy.

Building the Estimates Top Down

Mike Clayton suggests - as do others - that building the WBS can tale place top down, bottom up, or some combination.

Top Down is the only place to start. The WBS is a decomposition of the product and the processes that construct the product. If we start building the product bottom up, we'll end up with a hodge podge of parts assembled in some arbitrary manner. Image building a software product out of pieces and trying to assemble them into a production ready outcome.

Don't build the WBS bottom up - period. It's simply bad engineering.

Once the WBS is built top down - decomposed from the Statement of Work (SOW), the Statement of Objectives (SOO), the Statement of Project Outcomes (SOPO), the Concept of Operations (ConOps) - then and only then can it be adjusted from the bottom up.

Changes to the WBS are the same as changes to the Product Breakdown Structure (PBS). In fact the PBS is one part of the WBS. The other part is the Process Breakdown Structure, that identifies the processes needed to build the product.

Remember

The WBS is the combination of the Product Breakdown Structure (essentially the bill of materials for the Product) and the Process Breakdown Structure (the processes needed to produce the product).

With the WBS cost and schedule estimates can be made bottom up for each deliverable. Then revisit the total budget resulting from the roll out of cost and start making adjustments for the "planned budget."

This iterative approach maintains the product structure integrity while assessing the deliverables from the budget.

To do anything else, breaks every piece of the project's integrity. Cost, Schedule, and Technical Performance are now dis-connected and the project heads straight for the ditch. A common occurance in the absence of a project management process.

Credible Estimates

John Goodpasture has a post on estimates from Fred Brooks. Fred's quote, which I'll repeat for effect, from John's site is

It is very difficult to make a vigorous, plausible, and job-risking defense of an estimate that is derived by no quantitative method, supported by little data, and certified chiefly by the hunches of the managers.

In many domains, estimating is performed in ways similar to Fred's description. Here's some phrases found in RFP's from the Defense Department:

• The estimate must be easily traceable from the lowest level at which the Offeror’s estimate was substantiated to the CLIN.
• When using historical data, the Offeror should describe why the system is comparable to the proposed program. This includes a functional and technical comparison explaining the differences as well as similarities between the historical and the proposed system. Also include an explanation of the relationship between the analogous element cost and the total program cost.
• Adjustments to derive the proposal estimate relate to reasons and justification for any adjustments made to programmatic, technical and actual cost data for the historical system. The Offeror should provide the basis and document any adjustments applied to the historical data (e.g., complexity factors and normalization methods) that reflect the characteristics of the proposed system. This includes an audit trail sufficient for the Government to reconstruct the proposed estimate and judge its credibility.

Now these are heavy duty words, probably not applicable outside the procurement of similarly heavy duty weapons systems.

But as John points out through Brooks' quote, to do otherwise is complete nonsense on anything but a trivial project.

So wjy does this continue to be the practice in many areas? Some would say, it's because IT projects are software projects and you really can't estimate software projects. To this I say frankly - BS. The program we work int he defense are predominately software development projects. Software development with emerging requirements, changing requirements, state of the art technology, all the things IT project managers encounter.

My personal opinion is

Project estimating, and project management is hard work. Software development is fun work. Still hard but fun. Project Management is hard work, with almost no fun.

So why work on something hard if it's not fun? Secondly, there is no real downside for missing estimates in the commercial IT world. Oh sure, there is hell to pay, people get fired, the project gets canceled. But the company rarely goes out of business, people never die, launch dates are rarely missed. I'm not talking about silly commercial product launch dates, I'm talking about launch dates to Mars, launch dates for NRO (National Reconnaissance Office) launch dates. Those kind of launch dates.

What's missing is the discipline - at appropriate levels of course - to "do the right thing." That's the root cause of most problems in the commercial world - pure missing discipline.

It's the Micky Rooney school of management. Where Micky and Judy Garland are in the alley behind a NYC theater, when they say to each other and their friends...

Hey guy, I have an idea, let's put on a show

It's that, let's get started and see where this goes, kind of approach to project management. Where it goes many times is straight into the ditch.