The 2009 Standish Report is out. Many Blogs have made comments. All obvious comments. But there is a fundamental flaw in any report of this type:
The concept of a assessment baseline is missing from the conversation.
- Is there a Performance Measurement Baseline?
- Does this baseline represent a credible plan for cost, schedule, and technical performance?
- How would we know how many projects "should" fail in the population of all projects?
- Is the project sample used by Standish representative of the total population of projects?
- If so, what's the correlation between the sample population and the total population in terms of failure rates?
These are basic statistical analysis questions. Of course the report does not state these for the right business reasons, but the result is now limited in terms of its interpretive power.
The first big question is - Is there a control group?
- How many projects should fail as a matter of course of being members of a underlying population?
- What is the proper statistical population for projects?
- What is the underlying distribution of project success versus project failure for this population?
In the absence of this information - being made public - the Standish numbers are just a marketing document for their training and consulting services. In the same way the weight loose, hair growth, lawn growth, and other sales approaches are frankly - Nonsense.
The easy starting for discussion of this sort is How To Lie With Statistics. This little book wias written in the 1950's and is still an important contribution to the discussion. After reading this short book, you too will be suspicious of Standish style reports.
Just remember how you feel, when you see and ad for a weight lose products reporting how much weight will be gone in the coming 6 weeks with their product.
In the end though Standish provides no solution, just an 800-number to call for the "magic elixir"

Recent Comments