Ideas, Practices, and Resources about increasing the Probability of Project Success from direct field experience with proven principles. It doth often trouble me to thinke that in this business we are all to lerne and none to teach - Decon Robert Cushman of Canterbury, 1620
Edward Carroll's Blog On Projects, has a post about Work Packages in Prince2. This is the first I've heard of Work Packages outside out Earned Value Management System description. The EVMS System Description of the guide describing how programs are managed at the program planning and controls level. Here's a sample of the instructions provides for Earned Value based programs.
The Work Package appears as the bottom of the Integrated Master Schedule in the majority of programs. It is the Work Package that is "on baseline," rather than individual tasks. There are several reasons for this:
The Work Package usually has a duration of less the 60 days. This depends on the System Description, but the 60 days is so the planned completion of the work crosses only one accounting period. This answers the question How long are you willing to wait before you find out you're late? The answer 60 calendar days. This sounds long for small software projects, but it's VERY short for large defense and space programs running 5 to 7 years.
The Work Package has ONE and only ONE deliverable. This deliverable can be an intermediate outcome for the entire program. This removes all the blather and red herrings of uninformed agilest about the defunct Water Fall process in Big Projects. Work Package outcomes are close stories. They can be partial completions of work - these are 5 to 7 years programs, so it'd be hard to produce fully formed outcomes in a single cycle.
The Work Package "exit Criteria" is the Accomplishment Criteria in the Integrated Master Plan. Here's some background on the IMP/IMS paradigm. IT and software development programs would be well served to look through the guidance for defense and space software intensive programs - to see that Scrum, iteration, incremental, test driven design and other attributes are in place and used everyday. That quote - What has been will be again, that has been done will be done again; there is nothing new under the sun. - Ecclesiastes 1:. Yea that quote, it's true in program management as well.
The Work Package is owned by the "work package owner." This person is singularly accountable for delivery the outcomes from the Work Package. The Work Package can have a team, they can be organized in any why needed to get the work done. But there is only one person ACCOUNTABLE for the results - the Work Package Manager.
Work Packages live in Control Accounts (the intersection between the WBS and the Organization Breakdown Structure (OBS). The Control Account Manager (CAM) is singularly ACCOUNTABLE for the progress to plan of the Control Account. The notion of shared responsibility and accountability works nice in theory or on small self contained software development projects. On mission critical programs is does not.
In the whole scheme of things, here's where the work packages live...
A question, along with some others like, "when will we be done?" "What will this cost when we're done?" that need to be constantly answered if you're to be considered a credible project manager.
Many of the places we work have "Master Schedulers." They have "Chief Systems Engineers." They have "Lead Cost Analyst," and "Program Architects" (my role).
These people and the roles they play are critical to the success of the program for several reasons:
They know how the systems work. These systems are complex, because the programs are complex. A typical Integrated Master Schedule (IMS) for a large manned space flight programs runs in the 30K activity range. Not all are active of course, but the critical path from Contract Award to Launch must in the IMS if it is to credibly shows cost, work effort, resource requirements and programmatic and technical risk management.
The Cost Lead looks after $600M to several Billion dollars of allocated funding.
The two Chief System Engineers "own" the design of the space craft, with support from all the technical disciplines - avionics (our area, with Guidance Navigation & Control and Command & Data Handling being the principle software systems), propulsion, life support, structures, safety and mission assurance.
The notion of a Scrum Master has a bit of the smell of these spaceflight program roles, but where is the Master Architect, the Master Programmer, the Master Tester?
In our world the role of a "Master" are formally recognized, compensated, and part of the culture. The two Systems Engineers have business cards with the company logo, the program's logo and the title Chief Systems Engineer.
The positions are filled from the cadre of Systems Engineers that have trained, acquired Masters Degrees, professional certification and shown they are "fit for purpose" to be called Chief Engineer.
They also have gray hair or no hair, many "school of hard knocks" war stories about what not to do, and most of all the admiration and response of their community and the Program Managers that they work for.
Developing a Profession
Could it be that software development outcomes will improve if a paradigm like this were applied? That Master and Apprentice were the spectrum. To be a Master you must not only earn the position from skills, knowledge, and experience, but you must be recognized by your peers. IBM has Fellows, as does the IEEE and the ACM. INCOSE (the system engineering profession organization). These societies have refereed journals, software intensive product areas, a set of guides for practicing the profession of engineering.
When I encounter programmers claiming some process or another is the "right" way to go. Or a blogger making claims that practices that have been in place since the early 60's are flawed and not applicable in his domain (the same domain BTW that those practices are applied to today). I think about the Chief Engineers and the Master Scheduler I encounter on our programs. I wonder if we're missing something in the software development business?
I was poking around while on a conference call, looking quickly on Google for the current DoD guidance for Earned Value. I should know the answer cold, but it was a long night and I needed an "official" reference. There is of course the DoD and DAU sites, but here's an easier to read version.
That led me to poke around Google Reader for Earned Value blogs in case I had missed someone. I came to John Rusk's paper on EV and Agile. The link to Mike Griffth's nice blog with a post on Earned Value.
And then I came to the phrase...
Despite its usefulness, the confusing terms and heavy use of maths puts off a large percentage of the population.
Let's see if I understand this. "Heavy use of maths." Algebra with two (2) independent variables (BCWS - Budgeted Cost of Work Scheduled and ACWP) - Actual Cost of Work Performed) and one dependent variable - BCWP (Budget Cost for Work Performed) is considered heavy by a large percentage of the population. By large I naively assumed > 50%.
Yikes, this EVM stuff is complicated! Really? Is this really true? Were > 50% of the people working in project management or software development asleep in grade school arithmetic?
And then I encountered...
Debates digress beyond numerical accuracy - More challenging still is when several people do understand EVM and start debating which versions of the formulae to use and everyone starts obsessing on the math instead of the project. “Should we use EAC = BAC/CPI, or EAC = AC + ETC, or EAC = AC + [(BAC-EV)/CPI ?” Geez people, the developers probably pulled half of these estimates out of the air and we are applying advanced math to them?
That would be a NO. There are three (4) and sometimes four (4) formulas for ETC (Estimate To Complete) and EAC (Estimate At Completion). Which one to use? Use them all. One is optimistic, one is pessimistic, the others are in between.
I'd suggest looking through Dr. David Christensen's bibliography at the EAC and ETC topics to see how this is handled.
If you hold a PMP you need to know this. If you are employed in the program controls world or are a CAM (Control Account Manager), you better know these or start looking for work else where. Same for project's subject to SOX.
And if you're a credible Project Manager and you let your staff "pull estimates out of the air," you need to look for a new job. To invert the quote - Geez, can people be that stupid to proceed in the presence of bad estimates - no wonder IT projects fail.
But more importantly, if you work on a project where you are spending other people's money, you must be able to answer the following questions to be considered credible:
What is this project going to cost when we're done - within some error margin?
When are you planing to be done with this projects - within some error margin?
How much progress toward DONE have you made so far - in tangible units of measure?
How much more work is there to go toward DONE - in tangible units of measure?
The answers to these questions establishes the credibility of the project management team - no matter if they are formal DoD Program Managers or a Scrum Master with her team.
Using Earned Value with Scrum
Scrum and Earned Value are bedfellows, with some simple assumptions. First they both answer the questions above. But there are some assumptions that must be made for Scrum to answer correctly:
Is a Total Allocated Budget (TAB) established somewhere, by someone? What is the "Not To Exceed" (NTE) budget for this project? Some has to know this.
The total set of deliverables for the project also needs to be known. If not, the project is considered Level of Effort (LOE). On LOE projects, the passage of time and consumption of resources (money) is the measure of progress. This is sometimes called "Train Watching" after the German Railroad system of having crossing be "watched" by a person in a shack, to lower and raise the cross barrier.
With these answers in hand, Scrum can partition the work across the planned iterations and define the planned deliverables for each iteration. This is successful when the project is close ended, just like the definition of the project says it is when you read PMBOK®
Some now we've "connected the dots," back to Mike's post.
What’s the value in tracking progress against a flawed plan? - EVM compares actual project performance to planned performance at a point in time. So, if our initial plan is wrong we could be effectively trying to do the equivalent of tracking our progress on a road trip from Calgary to Salt Lake City on a map of France! The quality of the baseline plan is a critical success factor of EVM. In agile projects we acknowledge initial plans will likely need changing and so the basis for effective EVM is quickly eroded with evolving plans.
I have a suggestion , get a credible plan. Use Scrum's planning process for this. The olde saw, works here...
Doctor, doctor it hurts when I do this (raising his arm). Then don't do that.
Come on folks. We had a predilection for poster (banners) campaigns at a large DOE program I worked a few years ago. One poster read...
Don't do stupid things on purpose
If you don't have a credible plan to get to DONE, a credible "not to exceed" budget, some credible set of processes that will be used to get to DONE? You're doing stupid things on purpose. Stop
Finally
Where’s the Quality? - We could be on time and on budget, but building a horrible product that the business does not like or is low in quality. We should be aware that cost and schedule are not the whole picture. EVM and agile alternatives are just a part of the picture.
Mike has hit on an important topic. The connection between Cost, Schedule, and Technical Performance Measures (TPM). Here's a training briefing from the College of Performance Management, on this critical topic.
I find it ironic and amusing that “Earned Value” is a traditional project management term and agile projects often track progress in nebulous “Points”. This is backwards. Agile projects deliberately relate things back to real business value, and yet many traditional project track progress against tasks that add little or no business value and call the process “Earned Value”.
This another of those uninformed blanket statements. The "value" in Earned Value is not the same as the "Value" in business Value. Several VERY highly regarded agilest have made this same flawed assumption.
The Value in Earned Value is the consumption of the budgeted cost for the item being produced. The Budgeted Cost for Work Scheduled (BCWS) is the planned cost for that item. The Budgeted Cost for Work Performed (BCWP) is the measure of the consumption of this budget as a function of time.
We budgeted $100 for the item. The items is "worth" $100 when we are 100% done.
We planned to spend $100. We completed on-time and on-budget and consumed our $100. We're golden.
What's the business value of the $100 item we just completed on-time and on-budget?
This is not the role of Earned Value. This is the role of the business plan, it's ROI calculations and its connection with the business strategy.
OK, The Final Final
Creating a detailed requirements document that formalizes an incomplete, premature view of requirements that should change as details emerge and business continues to evolve is not earning much value at all. Yet prioritizing features by business value and tracking progress against the business oriented features is an excellent example of real “Earned Value”.
This is COMPLETELY dependent on the domain and context. If we're building a flying machine like this real one we work. Then having on incomplete, premature view of the requirements is called PDR (Preliminary Design Review). The program can't wait for the complete design. It needs some Rough Order of Magnitude (ROM) of the design. See Page 36 of the presentation above.
It seems there are assumptions being made about how EV and Agile are connected, possibly in the absence of knowing how EV actually works in practice.
John Goodpastuer has a nice post on the logic of Schedule Variance. Anyone using Earned Value should download the DAU Gold Card, fold it and keep it in your pocket. This can be found by Google'ing "Defense Acquisition University Gold Card." Ignore the security certificate warning.
For those of us working programs that require badges, we have a laminated version hanging on our lanyard, along with the accounting calendar, and other "reminders."
There is a notion going around that Schedule Variance can be zero (0), that is SV shows 0, and the project can still be late. This is technically possible, since the calculation of SV is simple algebra - SV = BCWP = BCWS. It's just simple math.
I struggled for awhile to understand how this concept got confused. That is if SV=0 (you've earned all the value you've planned to earn) AND you're late. What concept was missing, from the persons claiming this was a serious flaw in Earned Value.
What's missing is best explained in a Douglas Adams quote (remember him from Hitch Hikers Guide to the Galaxy)
I love deadlines. I like the whooshing sound they make as they fly by.
So let's try and see where the problem starts. Here's a short explanation.
The first edition of “Organizing for Work” by Henry Laurence Gantt (1861-1919) was published in 1919 and summaries the life experience of one of the founders of Improvement Science.
Henry Gantt was a mechanical engineer from John Hopkins University and an early associate of Frederick Taylor. They disagreed on the human aspects of production where Taylor proposed one best way for managers to instruct workers, while Gantt proposed many ways for workers and management to work together for mutual benefit of themselves and their organization.
Gantt devised the charts to know at a glance what stage production was in – and modern use of Gantt charts seem to ignore this simple and powerful feature. This revised edition of Gantt’s original book was written in 2007. (from Improvement Science)
So when there is a discussion about Taylorism, take a look at the reprinted Gantt book.
The notion that product development (engineering a product) is the same a Managing the efforts of the project that produces that product is a popular myth in the agile community. This notion was being taught to graduate students at one time. When I was asked to contribute to a course on "Managing Software Projects," I was joined by a product development manager from the worlds most popular electronic payment processes business. We approached the issue from the point of view of Scrum and general project management.
The glue that connected these threads together is CMMI DEV V1.2. For many reasons this is a good starting point for the discussion of software development processes and the structure of software development projects. As well the University hosting the graduate programs is also the university hosting the Software Engineering Institute.
Here's a slide taken from a course on this topic and the picture that connects the dots between product development and managing projects performing product development
If you can accept that methods like Scrum are applied to the development of products, then the connections between Project Management and Product development can be made. Note there are two other Process Groups in the business of software products, Support of the processes and Process Management.
Doing is not the same and Managing the Doing
Now certainly "doing" has elements of management. We're not robots. And doing and the management of doing have overlapping processes. But the management of doing has exclusive elements as well.
There may be a case - I'm waiting for the details - where the processes of Scrum can be applied to the general management of a software development project.
Many of the conversation I have with others around agile, start (and sometimes end) with why agile methods are the first choice for what ever problem is at hand, be it software development, project management or general problem solving.
I came across the OPF (Open Process Framework) in my travels this week. I'd seen this before and lost the link. We do Process Engineering work on the commercial side of our business. On the defense side the processes are well defined around Earned Value Management and the supporting activities. This does not mean they are well implemented, but they are pretty well defined.
The OPF site has a wonderful list of issues in the business software intensive systems domains. These include:
Business Engineering
Businesses are rarely engineered.
Business processes are obsolete or inappropriate.
Businesses are often poorly or inappropriately organized.
Businesses have difficulties introducing new information technology that could enable new applications that can improve the way the businesses do business.
Businesses are information-intensive and require numerous applications to support their business processes.
Businesses have difficulties selecting and prioritizing new applications.
Development Organization
Development organizations are not implementing the best industry practices and are sometimes even implementing known worst practices.
Development organizations are improperly implementing best industry practices.
Development processes are not properly specified and communicated to those who will use them.
System Development, Usage, and Retirement
Systems typically contain large amounts of software, which is intrinsically intangible, abstract, and complex.
Software-intensive systems are often highly complex due to both intrinsic and accidental complexity.
Software-intensive systems are expensive to develop and maintain.
Software-intensive systems requirements often do not meet customer goals.
Software-intensive systems are typically delivered:
Behind schedule.
With large cost overruns.
With less capabilities than promised or expected.
With inadequate or obsolete documentation.
That are neither reliable nor robust.
Some 25% - 33% of software applications are never delivered at all.
Software-intensive systems are often difficult to:
Develop.
Extend to meet new requirements.
Integrate with legacy applications and databases.
Port to new environments (e.g., the Web, n-tier client/server).
Use.
Maintenance is often complex and error-prone.
So Here's The Big Question
When there is mention that agile - or any method, process, paradigm - is mentioned as the "solution" to the woos of the software world...
What specific, tangible, fact based evidence is there that the suggested approach is better than what is being used now?
There are obvious examples of improvement, there is no doubt there. Having incremental and iterative development cycles is a no-brainer. Even DoD 5000.02 (the penultimate formal procurement specification) demands incremental and iterative processes. So that battle has been won for the writing of software or bending of metal into fighter planes.
But what about all the other things in the list above?
Brad Egeland has a nice post Cover Your Risk From All Angles. Brad lists Risk Avoidance and Risk Mitigation as two (2) approaches to Risk Handling. While these are legitimate approaches, there is a critical flaw in all these approaches.
When we have identified a risk and a mitigation plan, the execution of that plan occurs when the risk turns into an issue. When the issue is be "handled" there must be budget assigned for this handling. And there must be time in the schedule for executing this "handling" work effort.
Now if the project is disciplined, the Risk Management Budget will be withheld and there will be "margin" in the schedule for this new work.
A Better Approach
In the current mission critical programs we work, the approach is Risk Retirement. This approach identifies the work activities needed to make the Risk go away before it turns into an Issue. This approach can be applied to High Risk items. If these were to turn into Issues, the schedule would likely be impacted.
In less disciplined environments, it is also likely that the budget for the Risk Handling would have been spent. So there is a double impact on the project - you're late and you're over budget.
Think about Risk Retirement for those risks that if they came true would have unfavorable impact.
In our aerospace and defense domain, a Concept of Operations is a common, and sometimes mandated, document. We rarely see these documents in the commercial domain. But in the commercial world, a ConOps would be a critically important piece of information for the project - for exactly the reason it is critically important for aerospace and defense.
The ConOps describes the "How" in "How to Accomplish the Mission." The Mission in commercial terms can be the business case, the strategy, the Raison d'être for the project.
The ConOps answers these key questions:
What is the "Mission" to be accomplished?
What is the intent of the business leaders for this project?
Who are the key participants in the "Mission?"
What are the relationships between these key players? The heart of this question is Who's in Charge?
What resources are needed for the project to be successful?
What are the high-level operational tasks that must be performed for the "Mission" to be a success?
What connectivity is required between the participating system elements? People, Processes, Tools.
What are the training needs?
How will we recognize that the results of the project meet the needs of the buyer?
This is why we need a formal Earned Value Management certification process. I don't know how many unsuspecting "Bright Minds" read this, but it is pure nonsense.
How Earned Value is Limited and the Is Your Plan Correct? at Bright Hub, makes claims that are pure bunk.
I love the claims:
EV works best on smaller, simpler projects
Getting into technical projects or projects with many tiers can cause the project manager to allow the project to become derailed in the details
EVM does not tell the whole story on evaluating your project and might not accurately represent what is necessary for a project to achieve specific functionality. But, perhaps most importantly, EVM only covers cost and schedules and, therefore, no quality control is factored into EVM.
The last statement is partially true, but the first 2 lead me to believe that last statement is probably not understood. See if Technical Performance Measures could improve the understanding that Earned Value guidance explicitly states that "Technical Performance" is put of the process - defined in ANSI-748B and the NDIA Earned Value Management Intent Guide.
Lord help us if these types of information sources are actually read by decision makers.
"Don't say you don't have enough time. You have exactly the same number of hours per day that were given to Helen Keller, Louis Pasteur, Michelangelo, Leonardo da Vinci, Thomas Jefferson and Albert Einstein."
Borrowed directly from Moj Zarei-Kesheh's face Book page, Thanks Moj
From a recent NDIA (National Defense Industry Association) PMSC (Program Management Systems Committee) meeting - the root causes of project failure
Poor Planning is the Number One Cause of Project Failure
Planning is the filter between the complex, dynamic, and risky “open” systems of the organizational environment and with the “closed” project system (environment).
Project planning must provide structure, while preserving flexibility, especially for those projects involving inventive and/or long term complex work
Inventive and complex work involves strong elements of discovery, and requirements tend to emerge and evolve as the project evolves
It is often not possible to determine precise answers and create detailed plans.
(We) Need ways to “respond” to change and evolving innovations
Project planning must balance emphasis on project goals with capability (existing resources and knowledge) takes shape
Mike Griffith's has got me thinking about PMBOK®, the Agile software development processes and a general notion of "what to do." Alec Satin has a nice "electronic book" of 72 tips for project managers. Please subscribe for some useful information.
Anyway, I've got an idea to move the several decades of experience, ideas, and actual beneficial outcomes into a book like Alec's. Here's the outline...
Now I don't have experience in all these areas. But I do have deep experience in some. So I'm going to start collecting advice from the 3 dozen or so notebooks I've filled.
Along the way, it'd be interesting to see how agile software processes can be connected as well. By connected I mean practical, field proven activities that "answer the mail" (as we say in the defense proposal business).
My personal experience is focused on building the Performance Measurement Baseline (PMB), performance measurement processes (Earned Value), programmatic and technical risk management, the ubiquitous Work Breakdown Structure (WBS), and the programmatic architecture represented by the Integrated Master Plan and Integrated Master Schedule, the measures of increasing maturity, Measures of Effectiveness, Measures of Performance (MoP), and Technical Performance Measures (TPM).
I'll make an effort to add to the list every week and store this "map" and the information on the www.niwotridge.com site.
Rarely, if ever, is the conversation around agile and its
relation with project management processes set in a domain and a context in that domain. It's time to reset the
conversation with the just those items. To establish the "rules of engagement" for the conversation.
When we say Project Management what do we mean?
When we say agile, in what domain and context are we speaking about?
When we say anything about anything, what is the domain and context?
The discussion around PMBOK and agile first needs clarity. PMBOK is a set of principles (of sort) that can be applied to a wide variety of project domains and contexts within those domains. It is agnostic about "how" these principles are executed.
For example, 5.2 Define Scope, says
Define Scope is the process of developing a detailed description of the project and product. The preparation
of a detailed project scope statement is critical to project success and builds upon the major deliverables,
assumptions, and constraints that are documented during project initiation. During planning, the project scope
is defined and described with greater specificity as more information about the project is known. Existing risks,
assumptions, and constraints are analyzed for completeness; additional risks, assumptions, and constraints
are added as necessary.
There Inputs, Processes, and Outputs for Defining Scope. This "clip" is from PMBOK.
Now if an agile software development process - say Scrum - has artifacts or process steps that can be mapped to this process flow - great.
But Here's the Rub In The Discussion
The starting point in the PMBOK and Agile discussion is PMBOK. The Guide to the Project Management Body of Knowledge®. Not the other way around. Or at least not the other way around from my point of view.
So when Mike Griffith s makes a nice map from PMBOK to Agile, we're starting to connect the dots. But here's the rub.
That map - the connection of the dots - is context and domain sensitive. That is the practices that are connected to the principles of PMBOK has a domain - software development, and a context - agile software development team, with all the associated attributes of a software development project using agile.
This is all fine and good, but without stating the Domain and Context it might be seen as a replacement for the principles of PMBOK®
This is not normally the case, but it needs to be said upfront. In the proposal business and on program execution, we use a term - "rules of engagement." How are we going to engage each other in conversation? Essentially what is the domain and context in that domain for moving forward with our work?
The discussion around agile software development methods being considered Project Management processes brings up a core issue that has been around for a long time.
What do we mean when we say Project Management, when we say Project? Here's one starting point. There are other starting points, but this one is straight forward. It's the PMBOK Process Groups, with the PMBOK Knowledge Areas arranged under them. From the PMBOK Point of View, this is the full coverage set of processes and knowledge areas that are in place when you're managing a project.
Now when we start a conversation about an agile software development method being equivalent to PMBOK's project management framework, maybe it would be useful to ask a few questions.
Does the proposed software development method have coverage for the elements of this diagram?
Is this coverage equivalent in the ways PMBOK describe? Beyond just a name?
Questions like that, may be a starting point for a conversation about what is a project management method, what is a project.
Just a note when you hear any suggestion that new ideas can solve old problems.
In the business and practice of science - I come from the particle physics world - any new theory, suggestion of a change in principles, or a change in the practices of applying the theory, must be falsifiable to be considered a candidate for experimentalist (that was me) to be interested.
Falsifiable theories must make predictions that can then be tested.
So when someone postulates that such-and-such version 2.0 is now a replacement for the old such-and-such version 1.0, they are putting forth a "theory." In order to falsify this theory - that is show it is a candidate for replacement, the theory must make predictions is some sort. Something like...
My new proposal solves problems in this way and can be tested in that way in a broad enough set of examples to be statistically significant.
"Enough samples" usually means 15 to 30 - considering the Clopper-Pearson testing applied to the population of possible places the new "thing" can be applied.
So the next time you hear someone say, my approach is much better than the old approach and you shoudl start using it, you can ask, how can you show that your approach can be tested?
Recent Comments