I've been chatting back and forth on email, blogs, and forums around several things of interest to me this week. Some conversations were fruitful some were not. These topics include: Earned Value, Project Management, and the competencies needed to successfully manage project.
On the way home from Montgomery Alabama and the Air Force IT Conference at Maxwell Air Force Base, I come to understand that humans have a proclivity for fuzzy thinking and logical inconsistency.
This should come as no surprise when reading blogs and forum messages on almost any topic. The topic of earned value, competency measurement of project managers, success rates of government contracts, the effectiveness of agile software development, the interpretation of CMMI–DEV V1.2 are a few discussions that easily succumb to fuzzy thinking.
Fuzzy thinking and logical inconsistency is not the same as differences of opinion, or even differences in fact. The level of fuzziness and logical inconsistency usually follows the level of maturity of the speaker. According to Dreyfus gaining expertise in any subject has 5 levels
- Novice
- Advanced Beginner
- Competent
- Proficient
- Expert
Let's look at Earned Value Management as an example.
- Novice – a set of context free rules are acquired. These rules ignore the context sensitive features of Earned Value, such as the stability of the Performance Measurement Baseline, the management of Management Reserve, the flow of funds from the Total Allocated Budget down to Control Accounts and Work Packages.
- Advanced Beginner – through practical experience, the novice begins to recognize situations where the rote rules cannot be described in objective, context free terms. Size implicit size of a Work Package and the associated budget (BCWS) compared to the implicit or explicit risk determines the method of physical performance measurement.
- Competent – superimposing an overall strategy on the development of the Performance Measurement Baseline (PMB) on top of the Advanced Beginner and Novice's understanding of the principles. Does this PMB describe the inherent risk of the program and the mitigation activities needed to handle that risk? What impacts on physical percent complete arise from this risk?
- Proficient – in the previous levels of maturity decisions are made using deliberative and conscious choices. The proficient user of Earned Value makes decisions, as well, on the "feel" of the data. "These numbers don't feel right for our current phase of the program." "This SPI should be decreasing after we have passed Preliminary Design Review, and since they're not, there is some other issue we have to discover." The instinctive reactions to information come of similar experiences in similar situations. There is a spontaneous response comes from an "understanding" of the plan or strategy.
- Expert – no longer sees Earned Value as a data gathering and reporting process. The PP&C expert uses Earned Value as a paradigm to "manage" the program. The problem solving and decision making processes become integrated with the information produced by Earned Value. These "actionable information" processes are embedded in the daily rhythm of the program.
Now To The Problem
When there is a suggestion to "simplify" something or replace it with an alternative, or even to redefine formulas and words, say the application of Earned Value, the proficient or expert will respond with "OK, here's the minimal set of activities within the 32 Criteria of ANSI/EIA–748B that can be used and still call it Earned Value." The Novice might say, "I've got a better idea, let's copy ACWP to BCWP and we'll be doing Earned Value." (This is a nonsense approach of course, but one I have actually seen in practice on a $130M ERP deployment.)
The Advance Beginner might say "Let's move our unused Work Package funds back to Management Reserve (MR) to be used latter," or "let's extend duration of this Work Package and re–spread the BCWS." This is called "rubber banding the baseline."
When there is a discussion on a forum or a blog about applying EV and response come back, "why are you making this so complex?" the first response might be "well to get actual value from Earned Value you need to apply it at the Competent level or above.
Simplification "tricks" are just that "tricks." To move from the Novice to Advanced Beginner requires careful step–by–step deployment. Seeking advice and feedback from more mature sources. These don't have to be people, but working examples of how EV is applied at the expert or at least the proficient level. The best source of expert and proficient application of Earned Value is to look at the Earned Value Management System Descriptions that can be found on the web. These are DCMA (Defense Contract Management Agency) assessed processes for programs subject to Earned Value Management (EVM).
Not every use of EV is guided by DCMA, but every use of EV should – I'd say must – be guided by ANSI/EIA–748B and the NDIA Earned Value Management Intent Guide (EVMIG). In the absence of this framework, you might call what you're doing Earned Value – using BCWS, ACWP, and computing or "making up" BCWP – but it "ain't so."
Of the 32 Criteria from 748B there 10 or 11 "must do's" for anyone to say "we're doing EV." They are:
Organization
1. Define the authorized work elements for the program. A work breakdown structure (WBS), tailored for effective internal management control, is commonly used in this process.
2. Identify the program organizational structure including the major subcontractors responsible for accomplishing the authorized work, and define the organizational elements in which work will be planned and controlled.
Planning and Scheduling
6. Schedule the authorized work in a manner which describes the sequence of work and identifies significant task interdependencies required to meet the requirements of the program.
7. Identify physical products, milestones, technical performance goals, or other indicators that will be used to measure progress.
8. Establish and maintain a time-phased budget baseline, at the control account level, against which program performance can be measured. Budget for far-term efforts may be held in higher level accounts until an appropriate time for allocation at the control account level. Initial budgets established for performance measurement will be based on either internal management goals or the external customer negotiated target cost including estimates for authorized but undefinitized work. On government contracts, if an over target baseline is used for performance measurement reporting purposes, prior notification must be provided to the customer.
9. Establish budgets for authorized work with identification of significant cost elements (labor, material, etc.) as needed for internal management and for control of subcontractors.
Accounting Consideration
16. Record direct costs in a manner consistent with the budgets in a formal system controlled by the general books of account.
Analysis and Management Reports
22. At least on a monthly basis, generate the following information at the control account and other levels as necessary for management control using actual cost data from, or reconcilable with, the accounting system: (1) Comparison of the amount of planned budget and the amount of budget earned for work accomplished. This comparison provides the schedule variance. (2) Comparison of the amount of the budget earned the actual (applied where appropriate) direct costs for the same.
27. Develop revised estimates of cost at completion based on performance to date, commitment values for material, and estimates of future conditions. Compare this information with the performance measurement baseline to identify variances at completion important to company management and any applicable customer reporting requirements including statements of funding requirements.
Revisions and Data Maintenance
28. Incorporate authorized changes in a timely manner, recording the effects of such changes in budgets and schedules. In the directed effort prior to negotiation of a change, base such revisions on the amount estimated and budgeted to the program organizations.
Is EV for Experts Only?
No, but it's not for Novices either. Some level of knowledge and experience is needed for success. The advanced beginner will soon run into trouble once the reporting process starts to be "cooked" in any way. No amount is "simple" process can seem to prevent this even in sophisticated environments. Constant vigilance is needed to assure the numbers have meaning.
There are dozens of ways to make the number look better than they are. Some level of competency is needed to get beyond "reporting yesterday's weather," and start using EV to forecast the future. This requires experience and skill and most all the INTENT of Earned Value as described in the Earned Value Management Intent Guide and some higher level overview from Neil Albert on Earned Value Management Application, Guidance, and Education.
If you're doing all of these, you're doing Earned Value Management. If you're doing some but not the others, you're NOT doing Earned Value. You may be doing something else, but it's not Earned Value. The level of maturity needed to perform these 10 activities is beyond the Novice. Possibly the Advanced Beginner can take a stab at it. But competency is needed to use Earned Value Management in a project management context. Beyond just reporting. Managing the project.
So in the end the usefulness of Earned Value starts with the Competent user and gets better as you move to Expert. But even programs looked after by experts have troubles. All programs have troubles. It's simply the nature of complex, high risk programs. EV must become of tool to provide actionable information to the program manager (as one of our tag lines suggest), otherwise it's just a bunch of meaningless numbers.
