When the discussion turns to measuring project performance, the question of "what is a good metric" comes up. There are many answers depending on your view of the world. Burn Down charts, cost performance measures, schedule adherence, technical performance measures.
Here's what has worked well for me in a variety of domains - software, hardware, construction, process improvement, commercial, defense, energy, paper mills, power plants, ERP systems, and building our house.
- Define what done looks like in units of measure meaningful to the decision maker.
This means something that has business value. Story points for software probability aren't it. An ability to do something with the software that is connected to the balance sheet probability is. This is called a "capability." This capability should be found in some strategy document somewhere. This document answers the question "why are we doing this?" "Why is these feature needed?"
"We need the capability to process claim transactions at $0.07 versus our current $0.11"
- Define the budget and schedule needed to produce "done."
If you're spending someone else's money, you're going to need some form of budget control. These means you'll need to know what the project will likely cost with some level of confidence. You'll also need to know some level of detail about where those cost are being allocated. "How much do we need for licenses?" "How much for those external testers we've used in the past?" "Do we have a burdened cost for our staff and if so what is that compared to the budget we've got for this project?"
- Define intermediate assessment points along the way to answer the simple question how long are you willing to wait before you find out you're late?
Planning looks to the future to recognize what done looks like before the time and the money run out. Planning is essential for all project success. From mowing the lawn to flying to the moon. We need a plan.
But more importantly, how can we recognize that we're making progress to the plan? What are the measures of progress? Physical Percent Complete is a good one. This is why Agile is a nice way to measure progress at the software development level. "We planned these features, some appeared some didn't at the end of iteration."
- At each assessment point, determine the actual cost to reach that point, compared to the planned cost to reach that point.
When there is talk of "business value," we need to incldue the "cost" of that business value. Collecting costs is part of project management. Agile projects have it easy. They are essentially Level of Effort work activities. The staff level is steady, the period of performance is fixed (iteration). For other projects, not so much.
- At each assessment point determine the physical percent complete of the planned deliverable in units you defined before you started the work compared to the actual percent compete at the same assessment point.
Measures of physical percent complete need to be defined before the work starts, That way you'll recognize them when they appear. These measures must be meaningful. They must be tangible.
At each assessment point assure all measures are represented by tangible evidence that the physical percent complete is actually true.
- At each assessment point, assure that the measures of performance, effectiveness, and technical attributes are within the boundaries of the planned performance. This the P-Chart from Six-Sigma. Errors bands for EVERYTHING are a must, no point estimates without variances.
With these measures taken on the day you planned to measure them, for the values you planned to achieve, you will learn if:
- You're ahead or behind schedule
- You're under or over budget
- The thing you're building works as planned or it doesn't
If you kept track of the planned cost versus the actual cost and you kept track of the physical percent complete at the assessment point versus the planned percent complete at the same point, then you've got the numbers needed to forecast future performance.
This approach is called Earned Value - you earned the planned budget at the time you planned to earn it, with the planned performance attributes (effectiveness, performance).
There are no magic beans here. Just units meaningful to the decision makers. Units in money and time.
Do this simple process and you'll have all the metrics you need to have visibility into your projects performance.
