Herding Cats

The Naviety of Solutions to Wicked Problems

In this silly season of national elections, the notion of simple solutions to wicked problem is rampant. The same is true for wicked problems in the project and program management space. Government procurement is a wicked problem.

There are always voices out there ranting on about how bad everything is - and there some real examples of how bad things are - without actually providing any credible solutions.

If we first look at what it means to have a Wicked Problem we see, Rittle and Webber's definitions

• There is no definitive formulation of a wicked problem (defining wicked problems is itself a wicked problem) - this makes some sense, but is not too useful for searching for solutions
• Wicked problems have no stopping rule - this means the problem is continuous. Air pollution, green house gases, health care, federal procurement as examples of the "no stopping rule."
  
• Solutions to wicked problems are not true-or-false, but better or worse - even the better or worse may not have units of measure. Better or worse for who, over what time?
• There is no immediate and no ultimate test of a solution to a wicked problem - if we knew that it wouldn't be wicked would it?
• Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly - this doesn't have to literally be the case. For weapons systems acquisition for example, the problem can be "wicked" but still be repeatable if the complexity is high enough. We know what we want in a stealth fighter, but the actual acquisition of a stealth fighter is a "wicked" problem.
• Wicked problems do not have an enumerable (or an exhaustively describable) set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan - this is the basis of Capabilities Based Planning. We need to know a small number of capabilities. When there are 100's or 1,000's of top level requirements, the problem becomes "wicked."
• Every wicked problem is essentially unique. Every wicked problem can be considered to be a symptom of another problem - a colleague at a local defense contractor said “The problem is…that’s not the problem.”
• The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution - explaining the problem, becomes a "wicked" problem.
• The planner has no right to be wrong (planners are liable for the consequences of the actions they generate) - this is common in all "wicked problems" for cost, schedule and technical performance estimates.

Integrating Cost, Schedule, and Technical Performance Measures

In the previous post, the new IPMR DID instructs us to integrate cost, schedule, and technical performance.Here's some more advice.

Now the bigger question. Where is the similar advice for IT projects. Agile starts down this road, but needs a broader capabilities statement. PMBOK based PM doesn't have much to say if anything around this. Home grown PM processes are usually simple and obvious check lists.

What's Missing from Project Management in IT

The management of IT projects is usually based on PMBOK or a similar framework. The projects start with "requirements" and add all the common processes and their outcomes. There may be a business cases, but requirements are the first step in the PMBOK paradigm.

Here's what's missing and what is needed for restore the probability of success - There is no Systems Engineering process. Systems Engineering is mentioned once in PMBOK. Capabilities is only mentioned in the context of people skills.

With both Capabilities and Systems Engineering missing, the seeds of failure are laid before the project starts.

What does this mean for the project?

• Requirements are usually gathered but rarely engineered. Engineering the requirements means
  
  • Performing fact finding.
    
    • Producing an overall statement of the problem in an operational context
    • Developing the overall operational and technical objectives of the target system
    • Define the boundaries and the interfaces of the target system
  • Gather and Classify Requirements
    • Gather required system capabilities, functional, nonfunctional, and environmental requirements, and design constraints
    • Build a Top Down Capabilities and Functional decomposition of the requirements in a flow down tree using some sort of requirements management system
  • Evaluate and Rationalize Requirements
    • Answer the question "why do we need this? in terms of operational benefits
    • Build a cost benefit / model using probabilistic assessment of all variances and dependencies
    • For technical requirements, perform a task assessment to cost and schedule
  • Prioritize Requirements
    • Determine criticality for the functions for the system mission
    • Determine trade off relationships for all requirements to be used when option decisions are made
    • For technical items, prioritize on cost and dependency
  • Integrate and Validate the Requirements
    • Address completeness of requirements by removing the TBD items
    • Validate the requirements agree and are traceable to systems capabilities, goals, and mission outcomes
    • Resolve any requirement inconsistencies and conflicts
• If these seem overly complex consider this - especially when spending other peoples money
  • If you didn't have an answer to each of these, an answer suited to the situation, then what would be missing from your ability to know what DONE looks like?
  • If you didn't have the sources of the items above, what would the customer be able to say about what DONE looks like?

Requirements Elicitation

Several commenters to Herding Cats asks about requirements elicitation. Here's some top level guidance. But first a fundamental change needs to take place in the IT world on how to capture requirements. This change is guided by the processes used in the defense and space business.

It's called Capabilities Based Planning. Identifying System Capabilities is the starting point for any successful program. Systems Capabilities are not direct requirements, but a statements of what the system should provide in terms of “abilities.”

For example, on the Hubble Space Telescope Robotic Service Mission, three capabilities were needed:

1. Do no harm to the telescope.
2. Change the Wide Field Camera.
3. Connect the battery umbilical cable.

Here's a brief presentation on the concepts of Capabilities and Requirements

Capabilities Based Planning

The critical reason for starting with capabilities is to establish a home for all the requirements. To answer the question “why is this requirement present?” “Why is this requirement needed?” “What business or mission value does fulfilling this requirement provide?”

Capabilities statements can then be used to define the units of measure for program progress. Measuring progress with physical percent complete at each level is mandatory for the techncial assessment of the project's progress. But measuring how the Capabilities are being fulfilled is most meaningful to the customer.

The “meaningful to the customer” unit of measure are critical to the success of any program. Without these measures the program may be cost, schedule, and technically successful but fail to fulfill the mission.

The process flow shown on Page 4 of the slideshow is the starting point for Identifying the Needed Capabilities and determining their priorities.

Starting with the Capabilities prevents the “Bottom Up” requirements gathering process from producing a “list” of requirements – all needed – that is missing from a well formed topology. This Requirements Architecture is no different than the Technical or Programmatic architecture of the system. Capabilities Based Planning (CBP) focuses on “outputs” rather than “inputs.” These “outputs” are the mission capabilities that are fulfilled by the program.

In order to fulfill these capabilities, requirements need to be met. But we need the capabilities first. Without the capabilities, it is never clear the mission will be a success, because there is no clear and concise description of what success means.

The concept of CBP recognizes the interdependence of systems, strategy, organization, and support in delivering the capability, and the need to examine options and trade‒offs in terms of performance, cost and risk to identify optimum development investments. CBP relies on scenarios to provide the context to measure the level of capability.

Requirements Elicitation

Requirements are the defined attributes for an item prior to the efforts to develop a design for that item. System requirements analysis is a structured, or organized, methodology for identifying an appropriate set of resources to satisfy a system need (the needed capabilities) and the requirements for those resources that provide a sound basis for the design or selection of those resources.

It acts as the transformation between the customer’s system needs and the design concept implemented by the organization’s engineering resources.

The requirements process decomposes a statement of the customer need through a systematic exposition of what that system must do to satisfy that need. This need is the ultimate system requirement which all other requirements and designs flow.

There are two fundamental classes of requirements:

• Process Performance Requirements.
• Product Performance Requirements.

The Process Performance Requirements define how the work processes are used to produce a beneficial outcome to the customer.

The Product Performance Requirements define the product specifications and how they are related to the process requirements. As well as the product and process requirements, there are functional and non‒functional requirements.

These non‒functional requirements play a significant role in the development of the system. Non‒functional requirements are spread across the entire system or within individual services and cannot be allocated to one specific product artifact (e.g., class, package, component).

This makes them more difficult to handle than functional requirements. The specifics of the system’s architecture, such as highly distributed services bring up additional difficulties.

The distinction between process and product requirements lays the foundation for the Work Breakdown Structure. The WBS, the related Integrated Master Plan (IMP) and Integrated Master Schedule (IMS), also focus on this separation.

The success of the project or program depends on defining both the product and the processes that support or implement the product.

When properly connected, the Requirements Taxonomy, the Work Breakdown Structure, the IMP and the IMS “foot and tied” to the Performance Measurement Baseline (PMB). This provides the traceability of the increasing maturity of the deliverables (vertical) and the physical percent complete of the work efforts (horizontal).

Step-by-Step for Capabilities

1. Determine what the system is supposed to do in terms of Scenarios or Use Cases. This is not a new approach at all. Alistair Cockburn introduced the notion of Use Cases long ago. But they went astray because doing good Use Cases requires understanding what capabilities the customer needs.
   
   • What is the business problem or mission to be accomplished?
   • How would you recognize that this problem has been solved or the mission was accomplished?
   • Measures of Effectiveness are the units needed to confirm this accomplishment.
2. Assemble these Capabilities into a functional architecture, showing how each capability supports the mission or business need. The process flow below is an example.
   
3.  Develop a maturity flow for each capability showing how the presence of this capability allows the business or the mission to do some work. 
   • This of course is simple agile - working software.
   • But agile is now coming to see that bottom up, response to the customer need requires a framework - a Programmatic Architecture to assure that the end is reached as planned.
   • This of course is Stephen Covey's Habit 2 Begin with the End in Mind, and is actually the Integrated Master Plan / Integrated Master Schedule paradigm of DOD 5000.02 procurement. Imagine that 5000.02 and agile on the same page. See Agile+EVM=Success for guidance here.

Here's a picture of the Capabilties Maturity Flow from a real project...

As each capability appears, the project can then start producing useful services - do something useful. But - and this is critical - the business or the mission MUST be capable of receiving this capability. It does no good to have capabilities that cannot be put to use. This is the purpose of this diagram. To show what capabilities are needed, in what order. This is a Top Down process, done by the business or mission owners.

Step-by-Step for Requirements

1. For each defined capability there needs to be a process and product requirements as shown on Page 7 of the slide deck.
2. Each requirement MUST flow from a needed capability. Each requirement needs a reason for being there. It needs a parent. It must fulfill some needed capability.
3. As an aside - all requirements are derived. This means all requirements are derived from the needed capabilities. In some circles this is not the paradigm, but in the complex, software intensive world of space flight and weapons systems - All Requirements Are Derived from the Mission Statement or the Concept of Operations (ConOps).
4. Thesetwo documents are usually missing in the IT world, with the resulting gap of not knowing WHY we're doing something.

Rings of Success

Olympic Rings featured in nearing everywhere these days. Those rings represent the 5  parts of the world which now are won over to Olympism and willing to accept healthy competition. While there is much talk about the projects of the Olympics, some difficulties and many successes, there are other rings that must be connected for project success:

1. Risk - all project elements have risk. Cost and Schedule risk are a starting point. These can be addressed through straight lists of risks, their probability of occurrence, impacts, costs to mitigation and residual impacts after mitigation. Most approaches to risk management assume an independent association between the risks. That is they are decoupled risks, existing on their own. Many times this is not the case. What is Program Controls? speaks to some of the complexity of integrating technical risk with programmatic risk and addressing the coupling issues that are ignored in the Monte Carlo Simulators of project duration and their resulting cost models.
2. Cost - the olde phrase time is money is tossed around without any real consideration for what that means. What is the relationship between time and money. Where is that relationship defined? What are the statistical attributes of that relationship and its impact on the probability of success for the project. This must be addressed through a cost modeling system. SCEA is one place to start.
3. Statement of Work - What Does DONE Look Like? The SOW is one place to look. Assuming it is written properly. But Statement of Objectives, Concept of Operations, Scenarios, Use Cases, and Performance Work Statement are also places to look. Without a clear and concise collection of deliverables
4. Technical Performance Measures - define what DONE looks like in units of measure meaningful to the decision makers. TPMs involve a technique of predicting the future value of a key technical performance parameter of the higher-level end product under development based on current assessments of products lower in the system structure.
5. Work Breakdown Structure - describes the products or services to be delivered by the project, how they are decomposed and related. At the end of the tree of the WBS, is usually the Packages of Work that produce these outcomes. Without a properly formed WBS, you cannot determine what done looks like in any meaningful way. A typical mistake is to develop a WBS based on work types - design, code, test. This simply states the effort not the result.
6. Integrated Master Plan / Integrated Master Schedule - The IMP is the Plan for the delivery of a set of Capabilities from the project and describes the increasing maturity of these capabilities through Significant Accomplishments and their Accomplishment Criteria. The Integrated Master Schedule describes the sequence of work, usually in Work Packages needed to produce the deliverables for the Accomplishment Criteria.
7. Performance Measurement Baseline - brings the cost, schedule, and outcomes together in one place, puts this under change control, and measures progress to plan through assessments of Physical Percent Complete, and forecasts future performance (Estimate At Completion and Estimate to Complete) using Earned Value Management.

Putting This All Together Through the Tangible Evidence

Each of the Rings are represented by tangible evidence in the form of a document. I can hear the agilest now, no documents. Well sorry, in the domain we work, we need evidence that you do in fact have a SOW, a Risk Plan, an IMS, a cost basis of estimate, a Technical Performance Management plan and other items that show the project is planned properly, being managed properly, and measured properly.

Here's the top level connections...

Capabilities Explained

Brad Egeland has a nice post on Project Definition. Brad speaks about project objectives, assumptions, and associated attributes.

While these approaches serve the project well, an "offical" approach used in our defense domain is a Capabilities Statement. Here's our overview on this topic.

And the explaination of this chart that is on the left facing page

If you start with a Capabilities description, you can add Brad's attributes to the pile.

Where Do Requirements Come From

Craig Brown is starting a series written by Nilesh Raje, titled "Who Decides the Project's Requirements?" This is an age old question in the development of software. Nilesh writes for Requirements Networking Group. Like Craig's Blog, RQNG is a must read site.

Before reading the content though, I'd like to establish the notion that requirements only come from one place in enterprise class projects. In the best of worlds, all requirements come from the same place.

The come from the strategy of the business. The strategy tells us "Why" we are doing things. One representation of strategy is the Balanced Scorecard. There ae currently 3 generations of Balanced Scorecard. Many IT books still use the first generation. The one with the foyr quadrants. This is ill suited for the problems of a modern organization. The third generation arrived like this.

With the third generation we can connect strategy with tactics. And with this connection we can define projects and the requirements in those projects that fulfill the business strategy. A full end to end loop to answer the age hold question in software development

Why Are We Doing This?

So here's what those connections look like in the beginning.

What's to the right of this picture is a collection of projects - possible in a portfolio of programs and projects - that fulfill the Vision, Strategic Objective, Performance Goal, Critical Success Factor, and Key Performance Indicators of A business strategy. Note A Business strategy - there are many all defined in the Strategy Map. Here's a Strategy map we used to run a large IT department at a multi-billion dollar Department of Energy site.

For each item in the Strategy Map - P1 for example - there were a set of projects (program and portfolio) to cause the benefits of that strategy to appear at the planned time, for the planned budget and most of all for the planned beneficial outcome to the site.

The overview of this approach can be found in Using Balanced Score to Build a Project Focused Organization.

So we we ask "where do requirements come from," maybe this is the answer

Rules for Requirements Grammars

Requirements elicitation and engineering is a formal discipline in many domains. A recent paper  "A Method for Identifying Requirements Critical to mass Reduction Using DSMS and DMMS," James M. McLellan, Jonathan R. A. Maier, Georges M. Fadel and Gregory M. Mocko Clemson University, 11th International Design Structure Matrix Conference, DSM’09 12 – 13 October 2009, Greenville, South Carolina, USA.

This list is not only useful for formal Requirements Management (RM) environments, but for a broad set of project management domains.

• The subject of the requirement must always be a physical or tangible system, subsystem or component and not a property/attribute of a physical artifact. 
• Requirements with multiple systems must be decomposed into separate requirement statements. 
• Requirements with multiple verbs must be decomposed into separate requirement statements. 
• Requirements with exception clauses must be located at the end of the requirement statement. 
• Requirements with subject description clauses must be located at the end of the statement. 
• Requirements with clauses describing the direct object must be located immediately after the direct object. 
• Requirements with clauses that reference other requirements must be located at the end of the statement. The verb ‘to be’ describes a state of being of the subject not an action.

With this grammar, the result is better requirements. Better requirements have been shown to improve the quality of the project's outcomes.

Requirements Management for Grown Ups

Josh at PMStudent reminded me that DOORS is our "standard" requirements management tool. For those not familiar with DOORS, here some webinars.

DOORS was purchased by IBM awhile back. It's the standard in NASA and DoD ans is now moving into the commercial market place.

The statistics show poor requirements management means poor project performance. But with DOORS you actually have to have "real" requirements management. None of that "make and list and start coding" type of requirements.

With DOORS you can become a "real requirements engineer," and a Systems Engineer as well, since requirements engineering is Systems Engineering.

Baseline Magazine IT Project Survey and Requirements

    I'm never a fan of "project surveys" for all the right statistical analysis reasons. But Baseline Magazine has a nice presentation that goes along with a survey from a consulting company.
    The Truths About Project Planning is worth a look.
    The pitch is generally about managing requirements. The best books I've got on requirements are:

• Requirements Engineering , Methodology for Writing High Quality Requirement Specifications and for Evaluating Existing Ones, Software Assurance Technology Center,Linda Rosenberg, NASA Goddard Space Flight Center Greenbelt, MD
• Systems Requirements Practices,Jeffery Grady, McGraw Hill, 1993
• “Good Practices for Developing User Requirements,” Ellen Gottesdiener CrossTalk, 3/08
• The Requirements Engineering Handbook, Ralph R. Young, Artech House

There are many others of course, but these cover a broad set of domains and contexts.

A Bit About Requirements

• Requirements need a reasons for being - they must be connected to some business reason. This connection needs to be traceable back to a statement in the business case. This traceability is usually held in the document called the Concept of Operations.
• Requirements need to be "engineered," in the same way the system architecture is engineered and in the same way the software is engineered. That is someone has to look at the requirements and make sure they make sense. This is called Requirements Engineering and Requirements Management. 
• Requirements need to be verified and validated among themselves and back to the business case or the Concept of Operations. Then need to be traceable to the code, hardware, operational behaviors. This allows the project to answer the question "who ordered this requirement?," and "if I drop this requirement, what else gets dropped?"

    There is no way around these activities. The implementation of the requirements processes varies from 3x5 cards stuck on the wall for your XP project to a TeleLogic DOORS database with 10,000 requirements for a manned spaceflight program.

Customer Requirements Creep

    Returning from a one week holiday, I found the June 8^th Aviation Week had an editorial titled "VH-71: Spending to Save." The cancellation of the presidential helicopter brought everyone out of the wood work. The article's key point was "failure to manage requirements on the part of the government is a key cause of the cancellation." What started out as a $6.8B program grew to a $13B program. The president was quoted as saying "The helicopter I have now seems perfectly adequate."
    Marine One is actually a fleet of VH-3's and VH-60N's. Emphasis on "Fleet" since more than one is needed to move the president. That aside the management of requirements seems to be a common cause of failure in may programs these. Both in and out of government procurement.
    So why is this? Why with all our fancy dancy project management process ranging from PMBOK based methods to extreme programming, can't we manage requirements?
    I'm not a requirements management specialist. Program Controls is my sweet spot. But I can say by observation where the problem is.

If Requirements Management is NOT a profession, You're going to get unpleasant results

Not a profession in many domains that is. Requirements growth is a key contributor to cost overrun as documented in many GAO, NASA, DoD IG reports as well as the commercial IT and construction press.. Managing requirements is a verb.
    But requirements growth in the absence of budget management is the real problem. Budget authority without requirements management controls leads to OTB. OTB is the code word for "you blew the budget." Over Target Baseline. Here's some quick background.
    This problem is common in commerical IT as well. We simply can't control our urges to add features. What we're really saying is we can't manage the project in a professional manner. One that controls the requirements in a way to assures all stakeholder know what they're getting for the money invested.
    How could the government has not known that adding requirements to the VH-71 woudl not have increased it's cost? How could the government not had a "not to exceed" budget - the Total Allocated Budget shown on the Earned Value Management Gold Card?
    Poor management, that's how. Simple as that. Poor Program Management. Rememeber that the next time someone comes with "magic beans" in an attempt to convince you that the profession of managing projects and programs has been replaced. No Magic Bean is going to fix poor management.