Herding Cats

Does Methodology Matter?

Jesse Fewell posts a repeat piece about "Methodology Doesn't Matter." I've participated in the "method wars" in the past, with no progress. What I've come to understand through the school of hard knocks is that it's the outcomes we're interested in not necessary the path taken to reach those outcomes.

This removes all the theological issues around a method. But there is another important issue solved by this change in perspective.

When we focus on outcomes in place of focusing on the methods, we have made the connection to the business value. In some methods this connection has been made. The agile methods for example. For others, like Earned Value Management, the connection is juts now being discovered.

So what are the outcomes that are needed to make the connection to business value or mission success? here's my standard list.

1. Can we determine with some degree of confidence what the project will cost when it's done? How does this information match the expectations of the customer and their available funds?
2. The same answer for the duration? Can we determine the probability of completing "or or before" the need date for the project?
3. Can we discover, through some method or process, what the customer or funding agency considers "done." That is what are the capabilities that result from the project that allow the customer or funding agency to perform "their" job?
4. Along the way can we discover what impediments will impact our ability to answer the first three questions?
5. Can we determine with some degree of confidence that we are actually able to answer the previous 4 questions?

So if you can answer these 5 questions – with the proper degree of confidence - then "How" you answer them - that is what method you is to answer them - is not as important.

The inverse is also true. When ever there is a proposal to change a method, improve the process through a new method, augment the existing paradigm - one that may or may not be working, or conjecture any "new idea" - ask the following:

Does this thing that is being suggested provide improved information to these 5 questions? If yes, this is a desirable approach. If no, you're wasting your time.

What is a Concept of Operations?

The Concept of Operations (ConOps) is a paradigm found in space, defense, and enterprise IT projects. It provides value to the designers of the system through a perspective not available from the requirements alone. 

It provides a view of the practical applications of the product in which to couch the design approach, helping to ensure the designer’s interpretation of the requirements is meaningful to the customer’s anticipated use of the product. The ConOps puts the operation of the product into a world view perspective by stating: 

• Which operations need to be performed by the product and in what order? 
• What the operator will be providing as inputs? 
• When those inputs will become available in the operations timeline? 
• What he/she is expecting as output? 
• When that output is needed? How it will be used?

The ConOps is useful in a variety of domains and development methods. It defines the operational capabilities" of the system in the absence of specific technical requirements. It answers the question "what would I do with this system if it were operational now?"

If enterprise IT projects used ConOps in the same way software intensive space and defense do, the probability of success would be in creased for a simple reason - the requirements elicitation process has a dsecription of what "done" looks like before starting the capturing process.

What is the Unit of Measure of Your Benefit?

Tom Gilb has a wonderful post about defining "business value." When anyone makes a statement about a beneficial outcome, my first question is "what are the units of measure?"

I have some favorites:

• Our tool saves you time - how much time? Can this time actually be returned to the business in terms of bookable hours that I can see in the time keeping system?
• Our process significantly increases the quality of your product - can I see the before and after measures of quantitative quality measures?
• Scrum is a risk reduction method - how much risk is reduced? Is this risk reduction reflected in saved cost and time for risk mitigation or risk retirement
• Our Enterprise 2.0 solution is "easy to use" - what are the units of measure of easy?
• Our solution integrates into your workflows - really "my" workflows? Say my ABAP SAP workflows, how about my WfMC compliant workflows? Oh not those kind of workflows. What kind of workflows?

In other words, how will I recognize what you are telling me in the absence of your physical hand waving? How will I physical confirm what you've told me is the benefit?

The absolute best unit of measure of IT projects is a monetary currency. I put money I, I get money out. All other "units of measure" cannot be exchanged at the back.

If the speaker has no tangible evidence of the beneficial outcomes in units of measure meaningful to the buyer, firmly but kindly show him the door.

Capabilities Based Planning

One of the five process areas of our program management method is "Capabilities Planning." Here's a small picture of a moderately complex topic. In the same style of asking the Five Questions, these question need answers if you want to have any hope of being successful.
I get questions (outside the defense business) about "what do you mean when you say capabilities." Most everyone in commercial and defense gets the idea of requirements, the need for a master cost and schedule to fulfill those requirements, the actual running of that plan during execution, and of course the continuous risk management needed to keep everyone and everything moving in the right direction.

Well OK, not most everyone, otherwise we wouldn't have all those projects in the ditch upside down with their wheels spinning would we.

But capabilities is not always obvious. This week the Defense A&R Journal arrived while I was on the road.

Joint Acquisition Command Doctrine: A Success Story is a good example of Capabilities thinking. There was a comment from "Chart" (who never uses his - or her - real name and has a gmail address) about managing global integration projects. The concepts in this article can be applied to very large programs. Those project where the processes used for smaller project always seem to fail - those seemingly impossible ERP projects that every talks about.

1. Treat global ERP projects like a military campaign - planning in depth is the watch word for military processes. Have a Plan B and a Plan C. Have a plan for everything that can go wrong. Have a plan for this you don;t even know about, that can go wrong. This is he definition of "planning in depth." This is called "managing in the presence of uncertainty." It is the opposite of "trying to manage uncertainty," which is the common red herring of the uninformed project manager who rails against having a plan and a schedule - oh yea and knowing the cost  of things - just couldn't resist.
2. Defined the needed capabilities first. Only then ask what requirements are needed at what time during the program to produce those capabilities. If you start with requirements you'll simply create a huge mess, follow every bunny down the rabbit hole, have no way to tie requirements with benefits and ask that dreaded question that's been floating around from the last couple of posts - "how can I connect the cost of this project with the value delivered by the outcomes. If anyone here still has any doubts about why you absolutely must have some understanding of a projects (program) cost no matter what you're doing this is the reason.
3. Build full traceability between capabilities, requirements, and the work packages that deliver those requirements. Then establish a Performance Management Baseline and actually "manage" the project.
4. Manage everything as a Risk Manager. Use Tim Lister's quote - Risk Management is How Adults Manage Projects. Along with the related quote - "what's the difference between this place and the boy scouts?" "The boy scouts have adult supervision."

If you really what to know how to do this, here's a presentation of how to build that Performance Measurement Baseline from the top level of discussion.

The Flip Side of the Softer Side of Managing Projects and the People That Work Them

Several Blog posts this week speak to the softer side of project management, leadership of projects, and guiding the staff of a project. Eugion's PM Workshop Blog has several good posts around this topic. His blog has been improving with every post since I discovered it earlier this year.

I'd like to speak a bit about the flip side of the "softer side." First the principles of project management in our domain. This domain - at least the domain I'm accountable for in our form - includes aerospace and defense program planning and controls and civil and commercial government projects. Commercial government projects are executed by recipients of government money - usually federal and state funds.

In this domain we apply our Deliverables Based Planning^sm method in its full glory. This means:

• A Plan of the Week for the physical deliverables
• Measures of physical percent complete for these deliverables
• Risk adjusted cost and schedule baselines

In simple terms - tell me what you're going to do, do it, and tell me what you did on a weekly basis. This weekly Plan of the Week (PoW), is guided by the Plan of the Month (PoM). This Plan is of course guided by the Statement of Work (SOW). The result of all this "planning" is that everyone - the client, the stakeholders, the participants, and especially the project manager and the subject matter experts from our firm know what DONE looks like at the end of every week. On some projects or programs we also have a Plan of the Day. Just like eXtreme Programming - what a concept.

So now comes the "softer side" of managing projects. There is a minimal set of human behaviors on our projects. Professionalism of course. Recognition of subject matter expertise. We don't have any junior position. We're too small to hire people who need training. 10 to 15 years PM, PMP, core business skills.

But what drives all this is a critical success factor for all the project and programs we work.

Define what done looks like, then deliver to that description

What happens when that doesn't happen?

What happens when the deliverables don't show up on time? The first thing is that "not showing up on time" is crystal clear on the Monday after the Close of Business (COB) of Friday. The status for those deliverables is now YELLOW at least and possibly RED.

You didn't do what you said you were going to do. Hal Macomber's Securing Reliable Promises is a good example of how to apply Deliverables Based Planning^sm.

When the deliverables for the week don't show there are several steps that must be taken:

1. Determine the root cause. There can be good reasons or bad reasons. Good reasons need a Plan B to get back to GREEN. Bad reasons need intervention to get back to GREEN.
2. In all cases a plan to "get to GREEN" must be the next step. "You missed your deliverable for Friday, how will you provide that deliverable this week, without impacting the deliverable also due this week?" Otherwise your LATE.
3. The major root cause of being late is starting late. This is a fundamental law of "the physics of project" - LATE START = LATE FINISH

Now what happens if the "late" behavior repeats? Is it the person, the process, the environment, the client, the technology. This is now where Eugino's conversations start. But in a way possibly different than he intended.

The Project Manager or the Site Manager must rapidly determine what the root cause of "lateness" is. And make a change to "keep the project GREEN." If not the project will head for the ditch. The softer side of project manager has limited benefit in the presence of a Late project. Especially when the project staff is senior. Junior members need more "softer side" management. Senior members are on the job because they are senior - they know better. Or at least they should know better.

In The End Its the Deliverables That Matter

All the softer side process are needed for success - less so for senior members. But in the end the client did not buy self actualization and staff development for external projects. They bough a solution. This is a critical different between internal and external projects.

This BTW is why construction project managers are continually cranky. They're usually late and over budget all the time. And if not, they've spent huge amounts of time and energy keeping the project on schedule and on budget - and they're tired and cranky from just doing their job.

I've never met a Aerospace or Defense program manager that had a smile.

Progress can be made by

• Defining what done looks like on a monthly, weekly, and possibly daily basis
• Defining the work to get to done
• Measuring progress as physical percent complete - 0% /100% is a good indicator of progress
• Start on time, and you'll have a chance of finishing on time
• Don't let anyone get away with being YELLOW for more than 2 reporting periods
• Replace anyone who can't figure this out

Planning

The idea of planning seems harder than it looks. The idea of scheduling in the absence of a Plan is not only harder than it looks, it is a complete waste of time. Let's start this is a familiar quote:

"He who every morning plans the transaction of the day and follows out that plan, carries a thread that will guide him through the maze of the most busy life. But where no plan is laid, where the disposal of time is surrendered merely to the chance of incidence, chaos will soon reign." — Victor Hugo

In the absence of a Plan, the Schedule has purpose. This is the source of many of the failures in the IT project world. It is also the source of many of the alternative project management methods. Ranging from self proclaimed feral project managers with "magic beans" to self proclaimed heretics to credible alternatives that have made the connection between product development methods and describing what "done" looks like from the customers perspective.

So What Is Planning?

Making plans is a critical success factor for almost every human effort. Plans describe the desired direction of motion...

• Let's plan on having a tennis match at the neighborhood picnic
• Let's plan on leaving on Friday evening to visit Mary at college this week
• Let's plan on meeting at Starbucks Wednesday morning before we go to the clients on Adams and 5^th.

You get the idea. No let's look at the Plan in the context of a project.

• What is our plan for disaster recovery for the new data center?
• Do we have a plan for standing up the cell site next quarter to cover the emergency responders?
• What's our plan to respond to the inquiries about the next version of the database?
• If the integrated system test has difficulties, what's our plan to hold the schedule no matter the cost?

So What Does The Plan Tell Us?

Plans tell us what "done" looks like. A schedule tells us how long it will take - more or less. Schedules tell us the sequence of work needed to get to "done."

More importantly, the Plan tells us the intermediate descriptions of "done." Without these descriptions, there can be no incremental measures of progress. There are several paradigms of building these intermediate assessments of progress. My favorite is the Integrated Master Plan / Integrated Master Schedule approach used in the US DoD and NASA.

Integrated Master Plan and Integrated Master Schedule Preparation Guide

is one of the source documents. There are others, but this one describes how to construct a Plan that says what "done" looks, how to measure progress to getting to "done," and how to hook all the elements together into a credible Plan and Schedule.

Top Down / Bottom Up = Project Management 2.0? Not with this approach

Bas de Baar posted a re-post of a concept about Project Management 2.0.

Here's the picture that started my post.

If we look at the three boxes individually, here's some observations:

Top Down

• The description of this style of manage.There may be firms that operate projects in this way, but this is complete nonsense. No credible project management or business management method works this way. If it does, the firm is likely headed for the dust bin, now or soon.

Bottom Up

• These words are of course "god and motherhood" and not actionable in any way.
• Agility is a core process attribute of Product Development Kaizen used at one of the world largest defense contractors.
• Team driven processes are the core attribute of NASA Integrated Product Team (IPT) for the current manned space program. Same for large (multiple billions) of US Department of Energy construction and nuclear weapons programs.

Project Management 2.0

• Control and collaboration is the core process of the Integrated Master Plan / Integrated Master Schedule (IMP/IMS) mandated by US Department of Defense.
• Clarity of project goals is the core attribute of IMP/IMS and Performance Based Earned Value.
• Coordination and collective intelligence is established in the integrated Program Performance Management system - usually ranging from desk top tools to enterprise ERP and the DCMA certified work processes around them

So What's the Rub Here?

When all three boxes are put together, there is a major disconnect.

• First there are no actionable statements in the boxes. Nor are there any units of measure. By this I mean, how flexible is flexible? How rigorous is rigorous? When we same "agile" is that a noun or a verb? When we say control and collaboration, what are the units of measure.
• Some of these words are "content free." All projects,
• This approach is a non-starter in any non-trivial business environment. Any product or project leader would consider these words "whining" - no actionable outcomes on the Bottom Up side.
• The "agilest" message here is not only naive it is arrogant - based likely on poor past experience, not based on observing or even participating in credible project management environments.

In the end this approach is a bad example of what Project Management 2.0 should be. A another missed opportunity.

Because Project Management 2.0 needs to be pushed outside of the domains that currently use the Top Down and Bottom Up approach - the discussion needs to be how to combine the best of both worlds, not whine about how bad project management was used, then replace that with platitudes of the non-actionable buzz words.

If Project Management 2.0 is going to succeed, it must provide unassailable beneficial outcomes, in units of measure meaningful to the buyer. This approach to improving the process of managing projects is DOA.

What Does It Mean To Have A Plan?

A Plan is the strategy for the successful completion of a project.

A strategy is not a list of objectives, but the means of achieving those objectives.

Any project without a Plan is a project wondering in the wilderness.

What does the Plan tell us?

• What Done looks like?
• What activities are needed to get to Done?
• What are the potential roadblocks along the way to Done, and the activities that will be performed to remove these road blocks?
• What are the alternative paths to Done - literally Plan-B?
• What are the measures of progress along the path to Done?

Now that the Plan is in place, the Schedule answers the questions:

• What is the sequence of work needed to produce the results of the project?
• What is the work effort and duration of that work effort for each result?
• What "margin" is needed for the work effort and the duration to increase the probability of success for the project?

The schedule is a fluid description of the sequence of work. The Plan is the guidance for performing that sequence of work. When changes arrive, the Plan must tell us what alternatives we have. Or if there is no stated alternative, what steps we can take to develop one.

Plan the Work - Work the Plan

The old saw of Plan the Work, then Work the Plan is at the heart of the success of any non-trivial project. The core concepts go like this:

1. If we don't know what done looks like, how will we recognize it when it arrives?
2. If we claim to be making progress, what are the units of measure of this progress?
3. Are these units of measure meaningful to the participants, especially the customer?
4. When there are changes, how will we know what the impacts are to the plan?

But there is a body of voices out there equating Planning with waterfall process. The common one is the Agile Manifesto

Responding to Change Over Following the Plan

A quick look reveals the fundamental flaw in this notion. The Plan is the Strategy for the successful completion of the project. With no Plan you have no Strategy. With no Strategy, you have no guidance on how to respond to change. You driven by the "winds of change." The first result of responding to change in the absence of a plan, is you can only know you've arrived at Done, when the money runs out or the date for the product's shipment arrives. Usually not an acceptable option for those funding the project.

What Does Done Look Like?

I've used this phrase since 2002. It was first  used by Martin Radley who was a Project Manager in our Program Office at Rocky Flats. Martin's other phrase was what does "Done Done" look like?

If we don't have a definitive description of what Done looks like, it's going to be difficult to determine what work needs to be done and in what order it is to be performed. The description of Done at the highest level is usually some capability needed by the customer to fulfill the business plan or assigned mission:

• Process claims transactions for $0.07 each after 60 days of operations
• Dock at the International Space Station with 4 crew members on board

How Do We Measure Progress?

With a description of what Done looks like, we can decompose this into small descriptions of Done. This decomposition can go down to a granularity needed to manage the project. For Scrum projects this is an iteration and the work efforts in the stories. For larger programs this is the Work Packages and the Rolling Waves that implement those Work Packages, along with the Planning Packages for future "emerging" work.

By The Way, in the space and defense world Work Packages typically have duration of less that 60 calendar days or 45 work days. Just a little bit longer than iterations in Scrum.

What is the Unit of Measure of Progress?

This measure of progress must be units of Physical Percent Complete. Tangible evidence that what was planned to be completed was actually completed in the ti me frame planned for the allocated budget.

Managing in the Presence of Change

Trying to control or restrict change is not a method for success. Change happens. It is unavoidable.

In many domains encouraging change is not the preferred approach. In other domains accepting change is part of the process of reaching the end. Domain and context are needed before any discussion can be had about how to manage in the presence of change.

No matter the domain and context, change happens and you must manage the project in the presence of this change. The core process for doing this is to determine what the impact of the change is, how to moves the project forward, backward, or sideways. What to the change cost in terms of money, time, risk, value to the customer.

Answering these questions is the role of the change management process - no matter what method you are using to develop the product or service.

In the End

• Plan the Work - state what done looks like in some meaningful way
• Work the Plan - perform the work, measure progress, make the appropriate adjustments
• Continuously Manage Risk - during all steps, the management of risk is how adults manage projects

What Does a Good PLAN Look Like

The recent post about What Does a Good Schedule Look Like? brought out a response on What Does A Good Plan Look Like?

This brings up the issue of planning and scheduling inside of projects, especially projects where planning and scheduling is not the normal paradigm - Agile projects. Of course this not not really the case, because Scrum has planning and scheduling as part of the process. Plans are plans, and the schedule is the list of deliverables in the current iteration.

But one reminder for all those who have convinced themselves that scheduling and planning are not really needed and that emergent development and measures of velocity are the replacements.

The definition of velocity is the measurement of the rate and direction of motion.  Velocity is a vector.

What is the purpose of a velocity, if you do not know the distance to the end?

• Are you going fact enough?
• Are you going too fast?
• Are you going in the right direction?
• Is the direction you are going the least energy path?

What Does a Good Schedule Look Like?

A good schedule doesn't tell what you want to hear, it tells you what you need to know, so you must ...

• A good schedule is predictive - it shows what is going to happen in the future and what the alternatives are if that doesn't actually happen
• A good schedule is reflective - it shows where the project stands in relations to the planned position against the actual work that has been accomplished
• A good schedule is dynamic - it can be adjusted when the reality of the project changes

How to Improve the Credibility of Your Schedule

• Build the requirements in a requirements management tool

Trace these requirements to the work activities in the schedule. Only do the work that produces the planned value to the customer. This is a "deliverables based" schedule.

• Build the PLAN before you build the SCHEDULE

Define a strategy for the successful completion of the project - the PLAN. Do this before defining the work steps needed to implement the PLAN.

• Manage the project with a project management tool

Excel and Power Point are worthless. All tasks need successors and predecessors. So don't use your favorite project management tool as a generator of Power Point presentations. You need a critical path, you need to know how much slack you have for each deliverable, you need to know what resources are assigned to work work. You need a credible schedule.

• Make task duration fit the duration guide

Answer the question: How long are you willing to wait before you find out you're late? Make the task duration match this risk.

• Use a RACI or RAM chart to assign accountability for all the deliverables

Single points of accountability are required for all deliverables. A single named person is accountable. None of this collective responsibility stuff. That only works in the agile training books.

• Every task has a deliverable

Do only the work that produces value to the customer. No level of effort in the plan or the schedule. If the work doesn't produce something of value, then why are you doing it?

• Have a Plan B and a Plan C

Bad things always happen to good projects. Plan for it. Have the Plan B built into the schedule.

• All cost and duration values are random variables

Know what the variances are and you'll have the basis of a credible schedule. Assume they are scalar values and you have a nice Power Point presentation

• In the end, it's all about the people

Work gets done through people. Have the best people possible, treat them right, reward them appropriately

The resulting schedule, derived from the Plan is actually many plans and schedules...

Percent Complete?

Here we go again, redefining project management terminology to make a point that doesn't need to be made if we simply read the literature and followed the guidance and explored a little outside the narrow confines of agile...

There is no such thing as percent complete

Is Johanna Rothman's post about how percent complete makes no sense. Johanna needs to do some homework along with her class on this subject.

1. Physical percent complete is a contractual statement used on large project acquisition to mean How much of the physical deliverable has been completed.
   • How much of I-25 has been completed as of this date and how much should the State of Colorado and the Federal High Way Administration pay the prim contractor for this work?
   • This is not an arbitrary measurement. Nor was it an arbitrary assessment.
   • The measure of Physical Percent Complete is clearly and concisely stated in the contract
2. Physical Percent Complete is a Earned Value Management System (EVMS) term, clearly and concisely defined in ANSI 748B.
3. Physical Percent Complete is evidenced by artifacts from the work effort. Pieces of metal, working segments of code, walls of a building, assembled elements of an aircraft, numbers of drawings for the design of that aircraft, subcontracts issued to the components for the assembly of the aircraft.

In the software world, without tangible evidence of physical progress, the project is late and no one knows it.

Physical Percent Complete is the Only Measure of Progress.

In Johann's class room example how many features were planned to be done? How many features were done? Johanna is factually correct

Percent complete makes no sense. Features are done or not done. You can count done features and see how far along you are. You can't reliably count any percentage done.

Percent complete makes no sense if you don't know what done looks like. Counting features when you don't know how many features were planned for this iteration makes no sense. 

This is a core failing in all software projects. No wonder they're over budget and behind schedule. No one knows what done looks like. So there is no way to assess if progress toward done is being made other than the passage of time and the expenditure of other peoples money.

 

Physical Percent Complete

There is always a discussion around how hard it is for software projects to measure progress.

They're not like construction or building ships or airplanes

I'm calling BS on any approach like that. The notion of measuring physical percent complete is at the very core of every project management method starting with eXtreme Programming and ending with full DoD Earned Value based IMP/IMS (Integrated Master Plan / Integrated Master Schedule).

In fact if you're not measuring physical progress to plan with tangible evidentiary materials, then you are working on a Level of Effort project - where the passage of time means progress.

So What Does Evidentiary Material Look Like?

The real question is "What Does Done Look Like?" That is the question that needs to be answered at all times.

• What does done look like for today?
• What does done look like for the week?
• What does done look like for this iteration?
• What does done look like for the delivery cycle?

You get the idea. If you not speaking about "what do done look like" you can not recognize done when - or if - it arrives.

So what does done look like?

• A running piece of software that does something that the customer actually wants it to do.
• A functioning piece of hardware that passes a predefined set of tests.
• Dried concrete in the piece your wife wanted it to be - between the garden and the grass

Something tangible. Something that was described BEFORE you started. The idea that emerged code or emergent concrete walkways is quaint for projects that have no important purpose. But if you are spending other peoples money - or your home improvement budget - when you are done, you want the person who gave you the money to agree that you did what they asked you to do.

So predefine what done looks like for this increment of the project. The Plan of the Day, the Plan of the Hour, the Plan of the Week, ... Then check that you actually did what you said you were going to do.

That's how you measure physical percent complete. If you can not measure Physical Percent Complete, than only one of two things has happened. You are working on a Level of Effort project, or you chould not start work.

Follow the Baton Not the Runner

When someone says - "we have this much better process," or "you should do XYZ this way instead of the way you're doing it now." Stop, think, and ask some questions:

1. What are they selling. Maybe a product. Possible a philosophy. Maybe a political position. maybe themselves.
2. How can you know what they're suggesting actually works in your domain? What domain does it work in? Work domains doesn't it work in?
3. How can you tell it will actually work in your domain, if that's a domain they claim it will work in?

A phrase from the track and field domain for relay teams is to:

Watch the baton, not the runners.

This means never confuse effort with results. Never confuse a self-proclaimed expert with a practitioner. The more credentials behind their name, the higher the probability they is either the world's expert on a topic, or poser. Both use degrees, certifications, and credentials to impress you.

Instead ask - what value to they bring to the conversation? Always determine the unit of measure of  this value and the results of the processes. Make sure this unit is measure is meaningful to you or your customers. Without a unit of measure for the claimed beneficial outcome, there is no way to confirm there is actually any beneficial outcome to the suggestion.

This is an all-too common approach. As project managers we need to poke deeper at every claim of improvement. Ranging from tools to processes. So when someone says try my "magic beans," or "I've got this very clever spread sheet that short cuts all the work you have to do in your ERP system." The first responses should be:

• What's the evidence you gathered showing is provides improvement in your domain?
• Is this evidence statistically meaningful?
• Have you thought about how the benefits would be transferred from your domain into my domain?
• What evidence should I look for the see that your improvement is actually happening in my domain?

Yes It Is Rocket Science

In the book The Seven Secrets of How to Think Like a Rocket Scientist is advice we can all use in the management of projects.

1. Dream
2. Judge
3. Ask
4. Check
5. Simply
6. Optimize
7. Do

Each of these has sub-secrets. The total package has many lessons applicable to complex projects in general in almost every business domain.

The Bigger Issue

What becomes clear from this book is that projects are not actually about the writing of code, the bending of metal, or other "doing" kinds of things. Project are not actually about all the softer skills around the people. Or the money. Or even the customers as individuals.

These are all necessary but not sufficient conditions for a project to be successful. There has to be something else there. There has to be the "dream" there first.

When we get wrapped up in the persoanl favorite "thing," be it agile software development, earned value, programmatic risk, requirements management. We are operating at the 2nd or 3rd layer of the project.

The success of the "rocket scientist" is to hold in her mind the Dream and then the other 6 elements of success. Only when these elements are in place, can the mechanics of managing the project come into play. If these mechanics are the starting point, the project may or may not be successful. But the outcome that actually changes soemthing in a significant way will be lost.

Performance Measurement Baseline

I got a note today asking about the contents of the Performance Measurement Baseline.

Here's a good starting point

https://acc.dau.mil/CommunityBrowser.aspx?id=243202

If the site asks for a certificate validation, do so. It's a US DoD system. This has pretty much everything there is around Earned Value and related topics. If it's not here directly there are links off site to the what you'll need.

IMP/IMS For Everything

I just read a review of a book on A Girl's Guide to Project Management. There were six bullet points for effective team working:

• Clarity of vision and objective of the project
• Understanding one another’s strengths and weaknesses
• Norms of behavior and communication
• Knowing who is accountable for what and by when
• Recognizing time zones
• Understanding cultural differences

Some are outside my interest, some are core processes in our PP&C world. What struck me though was how we (in our world) have come to speak, act and react through the Integrated Master Plan / Integrated Master Schedule paradigm.

We use words like:

• What does done look like?
• What does done-done look like?
• What are the units of measure of done?
• Heck, what are the units of measure for anything that you describe?
• How would we recognize done when it arrives?
• What's going to prevent us from getting to done on-time, on-schedule?

These are the semantics of the programs we work. Even that phrase "programs we work," is part of the vernacular of our PP&C world. "working the program" means a daily rhythm, a weekly rhythm, and bi-monthly rhythm, and the ubiquitous monthly Contract Performance Report

So What's The Point

When the author (above) speaks of Clarity of Vision, Knowing Who Is Accountable for What, and other unstated process-centric activities on a project - requirements, capabilities, budget, plans, and the like.

These "Artifacts" are used to increase the probability of success for the project. This notion of "Increasing the Probability of Program Success (PoPS)" as crept into our work process in 2009. The Office of Secretary of Defense (OSD) has pushed down onto the services the PoPS.

The IMP/IMS paradigm speaks directly to the bullets listed above and two of the topics of the book. Here's my biased view in light  of the other topics in the book.

They're all important. The people aspects are important. But in the absence of a clear and concise of what done looks like, what the impediments to done and a plan to remove them, mitigate them, ignore them or transfer them to someone else all the people aspects will result in a project failure.

PMI has started to focus in the past few years - at least in our local chapter and annual symposium - on the touchy feely aspects of managing projects. Everyone is self actualized, but no one knows what done look like for the project.

That's a stretch I know, but it is a trend being pushed by those selling touchy feely training.

So What's a PP&C Lead / Program Manager To Do?

Put together a credible Performance Measurement Baseline (PMB). The PMB is the code word for

I know what done looks like, how long it'll take to get to done, how much it will cost, who will have to do what when, how I'm measuring progress toward done, and what the statistical (stochastic) behaviors of the "getting to done" look like and how to work around those unpleasant diversions along the way.

In the end books like the one reviewed are valuable to our profession. But also in the end the project has to show up more or less on time, on budget, and provide the value it promised to the folks paying the bills.

Always ask - "what is the unit of measure shows value for the thing you're trying to sell me?" Be it an idea, a process, a philosophy of living?

Describing What Done Looks Like

David Schmaltz is not a blog I visit too often. Too fluffy for my tastes. But once in awhile he has something that piques my interest. This time it resonates for reasons David may not appreciate. David seems to always be on a tear against scientific management, government programs, the military in general. Usually parroting the NYT misinformation about some defense program. That said, David is starting to echo a message that has been promulgated in the Federal Acquisition Regulations (FAR), Defense FAR (DFAR), and DoD Guidelines for some time. Amazing actually that these words come from his site...paraphrased of course

You must measure progress success in projects by measuring what the project produced - the nouns of the project

Yes, Outcomes versus Output is the language of project success. There's a comment on David's Blog that kinda hints at what is the basis of the Integrated Master Plan / Integrated Master Schedule (IMP/IMS) paradigm used in DoD and FAR acquisition contracts greater than $20M.

Defining what "done" looks like in terms of Accomplishments and their Criteria is the way we speak about  progress. Measuring the progress toward "Done" at "matuity assessment points" (program events) is what IMP/IMS is all about. Adding Earned Value to assess physical percent complete and producing the forecast of actually landing on time, on budget, at the planned degree of maturity - that's what Program Planning and Controls does on those pesky defense system programs.

Why PERT Has Problems

The PERT (Program Evaluation and Review Technique) has been around for some time. ["Application of a technique for Research and Development Program Evaluation," Malcolm, Roseboom, and Fazar, Operations Research, Volume 7, Number 5, Sep-Oct, 1959].

There are some who pride themselves on knowing every intricate detail of the source and evolution of PERT and its partner CPM (Critical Path Method). I'm not one of those. But the problems of scheduling and creating a credible schedule are still with us, even after all the historical assessment of who, why, and how we got to where we are.

The PERT Formula

First let's look at the PERT formula

T = (a + 4b + c) / 6

• T = expected duration (there is no cost in the PERT model, except when resource loading the schedule)
• a = the optimistic duration of the task
• b = the most likely duration of the task, the Mode, the most recurring value
• c = the pessimistic duration of the task

This is the "classical" PERT formula. This is the one implemented by the scheduling tools. Plug in the three value to the Microsoft Project columns and it will tell you the value of T, the expected time. This formula is in the text book, taught by all the teachers and possibly even used. The formula has been reverse engineered thought. The Standard deviation for the probability distribution function for all the possible value of a, b, and c is:

σ = (c - a)/6

This is where the reverse engineering took place. And the source of the problem of using PERT in any credible manner.

There are two fundamental statistical problems with PERT.

1. The estimate of a and c. These are the endpoints of the Beta distribution from which their values are drawn. The probability elicitation literature tells us that is it difficult for a person to accurately estimate the absolute endpoints of a stochastic quantity. A stichastic quantity is a random variable whose underlying probability distributuion changes its shape with time. Stchastic quantities easily represnt realworld processes. Traffic flow, packet flow, bank teller line length, things like that.
2. The estimate of b, the most likely time is also a problem. The original b is the Modal value of all the possible values of the task duration. But since the estimators have large amounts of empirical knowledge in their heads, they tend to estimate the median instead of the mode. The median is the middle most value. The Mode is the most recurring value. The difference between using the median and the mode is a significant asymmetric that PERT and the Beta distributions are not designed to handle. Remember the Beta and the classic formula were reverse engineered from the field data. This is called "curve fitting" and is a powerful way to forecast the future - IF the underlying statistical processes stay the same. Since they don't - they are stochastic - this is a problem.

There are some other limitations to PERT:

1. Activity times are not stochastically independent
2. The critical path comprises fewer activities than reasonable application of the Central Limit Theorem requires to have the modes of the most likely be credible.
3. The approximations of activity duration mean and variance can deviate significantly from reasonable and accurate approximations of the underlying activities.
4. Since only the mean and variance of the Beta distribution on which PERT is built is being used, there is little explicit attention being paid to the shape of the distribution. Mean and Variance do not define the shape of the Beta distribution.

Another Major Source of Bias

• The near critical path activities become critical in every real project. PERT does not make any consideration for these.
• Many parallel activities of roughly the same duration with differing variances produce wide swings in the estimated total duration. This occurs on small projects as well as large.
• PERT makes a very specific estimate of the variance (remember the reverse engineering). The Coon-Donaldson estimator ("Note on William A. Donaldson's 'The Estimation of the Mean and Variance of a PERT Activity Time,'" H. Coon, Operations Research, Volume 13, Number 3, May-June, 1965) creates serious errors that are not accounted for in the normal "teaching" literature.

Actionable Recommendations

1. Exercise great care of the myopic focus on the critical path. Near critical paths are the "show stoppers" of PERT.
2. The difficulties of mis-specifying of activity duration - Mode versus Median
3. Move beyond PERT to Monte Carlo simulation.

System Dynamics (SD) and Design Structure Matrix (DSM)

    There is a very clever theory that was developed at MIT and put to use by Lockheed Martin and similar firms. This is the Design Structure Matrix concept.
    The other concept is System Dynamics. System dynamics is an approach to understanding the behavior of complex systems over time. It deals with internal feedback loops and time delays that affect the behavior of the entire system.
    There are some claiming this is the next big thing in project management. But like all great academic ideas there is a big dose of truth, and the bigger dose of "a simple matter of engineering to make it work." Not that there aren't brilliant minds working the problem. Tyson Browning is one. Tyson worked at Lockheed Martin, went to MIT and developed some very useful concepts around DSM and project scheduling.
    The SD folks have many good ideas as well. But there is still much work to be done. Here's why:

• The concepts of feedback and rework in SD are not represented on the current scheduling paradigms. In fact rework is not allowed to be budgeted in the same work package as development. This may seem illogical to the SD folks. But "them's the rules."
• The DSM optimization processes are powerfully useful for small projects. The free tools from the DSM site - done in excel - can handle a dozen or so dependencies. Our typical program has several 1,000 activities. While we work very hard to minimize the dependencies, there are still 100's 

What needs to happen on the DSM side, is to turn the tool Latix into a project planning tool. It is a wonderful system architecture tool. Focused on software development. This is their sweet spot market of course. They could probably care less about us planners here in the field. There seems to be a tool - PlanWeaver - that fits the bill, but they seem to have moved on to other products and don't answer the mail regarding their product. I saw a demo a few years back and it was very interesting. Just what a planner would want.
    The SD folks need to do the same industrial scaling up.
    I've used both tools and the paradigms around them to advantage for "test projects." Which brings me to my favorite Yogi quote

In theory there is no difference between theory and practice. In practice there is - Yogi Berra

So when the academics and the theorist DSM and SD to the practice program planning and controls in the same way Latix did for software development we'll be on to something unique and powerful.
    Take a look at these concepts. They will trigger new thoughts:

• What are the interdependencies between tasks or deliverables?
• How can these dependencies be reduced. In the software world this is called "reducing coupling" and "increasing cohesion." This is a software architecture paradigm, that is directly transferable to "program architecture."
• The feedback loops and dynamics behavior of SD is a useful paradigm for production. But for discrete development projects this is a bit more problematical since the core planning tools don't support loops.

It's worth the effort to look further, if only to broaden your horizon. But challenge the academics for sure - those with PhD's behind their names - with the question

Have you applied these concepts to working programs. Program with 1,000's possibly 10's or 1,000's of activities and did it improve the end result in any measurable way?

    The academic approach is critically important. It is necessary before any engineering processes can be generally deployed. So we might be able to answer the question above in the affirmative in the future, but for now us poor slobs in the trenches are still working the Work Package sequence tonight to move the deliverables to the left. We could use some help to get our program on baseline by Friday COB. ;>)