Herding Cats

Alert - Just So We Don't Get Confused As To The Source Of Our Little Problem

Quote of the Day - Sketch of the Project's Value Stream

In preparing for battle I have always found That plans are useless, but planning is indispensable - Dwight D. Eisenhower, 34th President, General of European Forces, WWII.

Craig Brown has a nice post titled Sketches go a long way in generating understanding. But where's the Sketch of the Project's flow? In the Integrated Master Plan paradigm, the increasing flow of maturity of the deliverables of the project is represented by a diagram that looks like this.

What this diagram tells us is what is the "value flow" for the work effort. Each chunk of work activity "Pilot" with "Data" gives us a deliverable that can be assessed for its maturity. By maturity I mean a general term. This can be usefulness. It can be verification of the concept. It can be a test of the willingness of the funders of the project to proceed.

Agile would call this an iteration or maybe a release. But what is shown here is the end to end map of the project. In this case an ERP system for insurance claims processes. The Vision of DONE in just that DONE. The senior management of the insurance copy can see to the end of the project. They can see what it means to be DONE. As well the work flow to get to DONE is there as well.

We need to have some data and a pilot before enrollments are developed. This "dependency" structure is part of the Planning process. And of course the Planning processes results in an Integrated Master Schedule. Not for the whole project. That would be nonsense, no one can see that far into the future.

But we do have a Plan for the future and this is it. We need to do these big chunks of work in the order they are planned for the business to receive the planned value from this project. Oh yea at or near (within agreed variance) for the planned cost, since we asked the Board of Directors for the budget we needed and they approved it and put that cost in the GL as an obligation and that obligation got reported in the 10-K for the stock holders to see that 100's of millions are being committed of their money.

Is there a home agile development in this approach? Sure at the work package level that is 3 to 4 levels down. This there a home for Earned Value? Absolutely, that's how progress to plan is measured down at that work package level. This progress to plan is progress against the planned budget, the Budgeted Cost of Work Scheduled (BCWS). BCWS times Physical Percent Complete = Earned Value. We "earn" our budget, if we show up on time, don't over spend, and the thing we're building actually does what we said it was supposed to do. What a concept. No hand waving, no whining about this or that. Do what you planned to do, once that plan is agreed to and baselined.

Emergent requirements? Absolutely. Plans change. Requirements change. That stuff on the right is way out there in the future, it depends on the success of the stuff on the left. No one in their right mind woudl commit to a cost and schedule for stuff that has not been vetted through "past performance" and verification of capabilities.

Just don't commit to this beyond your ability to actually make them appear. This is called rolling wave. It's the basis of Earned Value, it's the basis of enterprise project management, DoD 5000.02, and just plain olde good Project Management.

But if you consider things like detailed planning of current rolling waves, applying Earned Value to measure physical percent complete against cost and schedule "hog wash." Then try doing this and not explaining to senior management how much it's going to cost in the end, when are we forecasting completion (within a variance), how many people are needed (300 or so to start), and what is the sunk cost along the way that can be compared to the expected business return.

Try doing this without those and many other "measurement" processes, and your career as a PM will be short lived. Try doing this without measures of performance based on budgeted cost, physical percent complete, and planned percent complete and if you can, then you'll be very close to Earned Value.

Anchoring and Adjustment in Estimating

Josh from PMStudent.com has a post about estimating. There is a discussion of using 3 point estimates.

Some time ago, I came to the conclusion that making 3 point estimates is a bad idea. Instead use confidence intervals for risk ranges and drive the variance from those bands with a Monte Carlo Simulator.

This is a critical topic that goes to the credibility of any estimate no matter the domain.

The starting point is to understand the concept of anchoring and adjustments to the estimate and their impact on false optimism (or pessimism). The paper to start with is "Anchoring and Adjustment in Software Estimation," Aranda and Easterbrook.

When you read this, you may rethink the approach that asks the estimator the Most Likely estimate and Optimistic and Pessimistic estimates.

The original work in this area is from Tversky and Kahneman and their "Prospect Theory." A good place to look for some details (besides their work) is Against The Gods, Peter Bernstein, in Chapter 16, The Failure of Invariance."

It's time to move on from the classic "simple" approach found in places like PMBOK(r) to the processes used in other places that depend on probabilistic and statistical analysis - engineering, finance, medicine, physics.

We Have Met the Enemy and He is Us

From the Project on Government Oversight, a POGO moment.

It just gets worse every day.

Our Energy Crisis

Before starting any discussion around alternative energy, nuclear energy, any kind of energy, you need to watch Dan Nocera's talk. Not so much for the solution but for a description of the problem.

Standish Report

I've written several times about my personal dislike of the Standish Reports.

• Standish Report and Naive Statistics
• Finally, A Challenge to the Standish Report
• Project Failure Rate

The current assessment of the Standish Report is:

"The Rise and Fall of the Chaos Report Figures," J. Laurenz Eveleens and Chris Verhoef, Vrije Universiteit Amsterdam, IEEE Software, January/February 2010, pp. 30-36.

The core findings are:

• There are misleading definitions - the definitions used by Standish are based solely based on estimation accuracy of cost, time, and functionality. Standish labels projects as successful or challenged, suggesting much more than deviations form their original estimates.
• Unrealistic Rates - estimation accuracies are questionable. These estimates are "one sided" and neglect under runs for cost and time and overruns for the amount of functionality delivered for either the under or over runs on cost and schedule. Institutional biases exist and are unaccounted for in the Standish numbers. (see Software Engineering Economics, Barry Boehm, Prentice Hall, 1982)
• Cost Forecasting - using independent cost forecasting databases, it can be shown that the Standish number forecast a 59% success rate for cost adherence, when the baseline database from actual projects - 140 projects from 2005 to 2006 at a large financial services firm - follow Boehm's "conical shape" for accuracy and produce an estimating quality factor of 8.5. This means that half the projects have a time-weighted average deviation of 12% or less from the actual forecast.
• Accuracy Again - using their definitions of cost and schedule overestimation and large functionality underestimations perverts the estimation accuracy.

All of this information and the references point to a statistical assessment gap. The core problem is:

• There is two-sided measurement from a statistical point of view. They only ask about failure rates, not the success rates. This would produce a biased outcome.
• There is not a statistical sample selection process. Send out a survey, get back the responses, and total the number. How many surveys are needed to establish a credible sample size in order to establish the confidence intervals.

Estimating

The estimating process for project cost and schedule is usually one of the poorest skills for the project. Not because it is difficult or even because it is not well understood. But because there is a myth that estimating is not really possible.

 

Estimating cost and schedule requires skill, experience, and a method (there are many) and some tools. Here's how it starts:

• You must have some notion of what done looks like. This can be "done" for this iteration. Or "done" for this rolling wave. Or "done" for this release.
• You must be capable of controlling the scope of "done" within the period of performance of the development process.
• You must measure productivity - the capacity for work - in some unit that allows you to forecast future performance.
• You must be able to engage everyone in a conversation about estimating the future using the past.

In the absence of these basic capabilities, estimating is simply not possible. The the suggestion that estimating in the software development world is not possible, will come true.

NYT Chart on the Economy

Frank Patrick got this first, but all of us with the New York Times subscriptions should look at this clever charting processes for the Current State of the Economy.
    This reminds me of the "bulls eye" charts in winSight we used for plot the Cost Performance Index and Schedule Performance Index from Earned Value across the life cycle of large programs. Look at Page 20 of this presentation on applying EV to IT projects.

OMB IT Investment Management

    The Office of Management and Budget has released an IT Investment Dashboard for US Government IT investments. This is driven by the OMB Circular No. A-11, Part 7 Planning, Budgeting, Acquisition, and Management of Capital Assets.
    The notion of making public the IT investments and their performance is of course long over due. At the same time non-government firms would be well served if an internal version of the IT Investment Dashboard were used for the portfolio of IT projects,

A Program Manager's View of AIG

    In the application of Earned Value of programs, there is a value below which we move on when trying to get all the control accounts to balance for the period of performance. Usually if you can get all the numbers to line up for 99% of the total value, its called DONE and let's print the Contract Performance Report (CPR) and send it to the customer. 1% error depended of course on the size of the program. There are time when 5% error is acceptable. And other times when less than 1% is needed.
    But to spend time futzing around with 1/1000 of 1% is not only a silly it is a waste of time. So why is the US Congress burning huge amounts of time and money on 1/1000 of 1% they gave AIG? Mostly likely to cover their asses for not having language in the bailout bill in the first place. Probably because those sitting behind those desks have actually never really managed anything. Nor would we want then to. No because they shoudl not, but because they are representatives of the people and should be focused on doing the people's business. Making Laws, no micro managing what is clearly a bone head decision by AIG.
    National politics is all about image and sometimes about substance. So getting your face on TV is part of image. If someone would go on TV and state in clear and concise terms that spending time managing the clearly poor decision of AIG's imbattled president is a wast of time, then maybe we poor souls here in the field woudl have some convidence things will improve.
    Indepedent of one's politics, we're loosing sight of good program management practices - focus on the big picture in times of crisis.

A Bold Faced Lie

    Watching TV today and saw an ad for the Cadillac Escalade Hybrid compared to the Mini Cooper - S I assume. The ad by GM claimed the Escalade gets better millage than the Mini - 20MPG in town.
    I don't own a mini, but I do own a Honda Civic SI, with a comparable engine and drive train. 198HP and a 6 speed manual transmission and a very snappy response curve when you can heal and toe the drive train.

    While the SI is not as snappy as the Volkswagen GTI and certainly not comparable to the STI WRX or Evo-10 which have near race car performance, it is a scary ride when pushed to the limit. with a 8,900 RPM red line in a slight drift in the off ramp, a close ratio 6 speed and moderate breaking, it does have a near "no body roll" turning behavior. It'll kick ass in the unaware BMW and Audi driver waiting for the light to change in the common Colorado intersection, where two lanes go down to one in a few hundred feet. Chirping the tires in the first 3 gears is common. Add high performance snow tires and it's a fun ride for commutes and general back country road driving.
    The ad GM wants us to believe that GM has cracked the code for mileage in a SUV platform. When you go to the EPA mileage site you can compare the two vehicles. Turns out though the some kinds of numbers are being cooked.
2009 Mini S - 26MPH city and 34 highway. I can get 33 out of the SI on a 50 mile commute on flat highway. Down to 28 highway on the Denver to Breckenridge route. No tongue in the 2 liter Honda engine, so 5th gear running at 5,600 RPM going up I-70 to the tunnel.
    Having owned a Suburban (350, automatic) it had crappy torque as well. Our Honda Pilot out performed the Suburban in all ways except pure volume.
    The Esclade hybrid is rated at 20MPG city and 21MPG highway. It also give up towing capacity.
The Mini convertible is much less 23 City and 29 Highway. But both Mini's are turbo charged and the SI is not - if it were, it would produce 230HP and be really scary.
    So what's with GM. Are they trying to convince us that the Esclade is now a "green" car?
    Lies and damn lies is my response. I'll record the ad next time its on to do a more detailed assessment. I was some dumb founded when I foirst saw it, I forget to push record on the DVR.

$35B to build advanced manufacturing? Too Late!

See the video from the Detroit News on an advanced Ford Plant - in Brazil.