At the Earned Value Analysis 2 Conference, in November of this year there were many good presentations on applying Earned Value to projects and programs.
The measures of success using cost and schedule metrics is not the way to discover the reasons of failure. These measures are used by the typical IT project success reports. But that's not the problem. The real problem is that there are not measures of DONE. DONE described in units of measure meanigful to the buyer.
What struck me about one presentation was the PMI definitions of Project and Programs. Here's a clip from the presentation. These descriptions state
In fact the measures of project success MUST start with two other measures.
- Measure of Effectiveness - The operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions.
- Measure of Performance - Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions.
Measures of Effectiveness (MoE)
- Are stated in units meaningful to the buyer,
- Focus on capabilities independent of any technical implementation,
- Are connected to the mission success.
MoE's belong to the End User. They define the units of measure
Measures of Performance (MoP)
- Attributes that assure the system has the capability to perform,
- Assessment of the system to assure it meets design requirements to satisfy the MoE.
MoP's belong to the Program - Developed by the Systems Engineer, measured by the Control Account Managers, and analyzed by program controls staff.
Key Performance Parameters
Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program.
- Have a threshold or objective value,
- Characterize the major drivers of performance,
- Are considered Critical to Customer (CTC).
The acquirer defines the KPP's during the operational concept development - KPPs say what is "DONE"
Techncial Performance Measures
Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal.
Technical Performance Measures:
- Assess design progress,
- Define compliance to performance requirements,
- Identify technical risk,
- Are limited to critical thresholds,
- Include projected performance.
These measures are the "success criteria" for the project. This includes the cost, schedule, and technical performance. This means that the notion of "quality, cost, and schedule," - the famous Iron Triangle - is not very useful for the following reasons:
- Quality is part of a Technical Performance Measure
- Measures of Effectiveness and Performance speak directly to the acceptance of the project's output to the customer.
This last statement is where we get in trouble with the project's measure of "success." With these two Systems Engineering measures, both the customer and the supplier cannot recognize DONE when it walks in door.
The result is a high percentage of reported fail. Forget the "over budget" and "over schedule." These are secondary to the MoE and MoP compliance.
