Herding Cats

Danger Will Robinson

There are several fundmental principles of writing software for money. Let's start with the prime directive.

It's not you money, behave apprioriately.

When asked to develop software in exchange for money - this is leaglly called work for hire - we have an obligation to have some notion of the final cost. If not, we're likley working a staff augmentation and not doing work for hire. So let's assume we are on a WFH engagement. Knowing something about the final cost starts with estimating that cost. This estimate says its name - it's an estimate. It's not a firm price. It's not even a firm anything. It's an estimate.

This estimate can come in several forms

• This customer has a budget and would like to  know what can be produced for that budget. This is common in governance based domains. The owner Hubble Space Telescope had a budget to fix HST robotically. He got that budget from the NASA administrator. He released a BAA (Broad Area Announcement) for solicitations of what could be done for his budget. Some for a CIO of a health insurance company. She got a rough budget for upgrading the provider network ERP system and asked vendors to tell her what could be provided for that budget.
• The customer has a set of needed capabilities and needs to know the approximate cost of providing those capabilities and needs to know the cost to provide them on a needed date.

The answer to both of these approaches comes from making estimates. Estimates of cost and schedule. Estimates of capacity for work. The process for making these estimates is called  Basis of Estimate and  usually starts with an  anchoring nbsp;process. 

By anchoring it means making the estimate from something that can be used as a reference. It's been done before, there is a model of what  could  be done. Some reference class. This anchoring process is then followed by an  adjustment process. Adjustments can come in a short time, or over a longer time. But the anchoring and adjustment paradigm is well developed.

One research study has shown

One way to make judgments under uncertainty is to anchor on information that comes to mind and adjust until a plausible estimate is reached. This anchoring-and-adjustment heuristic is assumed to underlie many intuitive judgments, and insufficient adjustment is commonly invoked to explain judgmental biases. However, despite extensive research on anchoring effects, evidence for adjustment-based anchoring biases has only recently been provided, and the causes of insufficient adjustment remain unclear. 

Anchoring and adjustment is well studied in behaviour finance - why people make financiual decsisions that they do. Anchoring and adjustment is also well studied in estimating starting with Oil & Gas estimates of reserves to estimating public works projects. A complete litertaure search can be found from Google of course.

But estimates needed for making business and techncial decisions must take this research into consideration is they are to be successful.  For cost and schedule estimates the best place to start is past performance. Here's an orginal drawing of Flybjerg's Reference Class Forecasting process flow. Google will find you all his work as well. Many about infrastructure and some about IT.

When it is mentioned we haven't done this before so we have no reference classes, we have to pause and think. Is this a trueGreen Field project. Technology that has actually never beed done before is rare in the IT world. If this project is truly without precedent, it's likely an R&D project and need to be planned much differently.

So assuming this is not an R&D project, what can we do about creating estimates when we have little past performance? There are many tools available, some free, to start the estimating process. To produce an anchor estimate, we can start to refine before the project starts, or even after the project starts. 

There are three types of activities that paerticipate in the estimating process

• Stable activities  are  understood and predictable. Estimating these is  straightforward.
• Dependent Activities where the time or effort is  dependent on project attributes or characteristics   not yet known but they are knowable. 
• Uncertain Activities where little data is available to support the estimate with any confidence. 
• Analogous Estimating using the experience from previous projects and extrapolating that onto the current project. 
• Parametric Model Estimating is an  accurate and easy estimating technique developed for estimating the time or resources needed to perform a project activity. 
• Expert Judgment when there is an expert on the project. 
• Published Data Estimating for those activities for which there is published data. The activity is compared to the activities for which data exists and the actual cost or durations of the closest comparable activity is selected from the data and used as the estimate. 
• Reserve Analysis is  fundamental for all estimating and  considers the level of uncertainty and risk in the project and establishes a reserve pool of time, resources, or possibly performance that can be drawn upon to offset the unestimated issues that arise. 
• Bottom Up estimates improves the accuracy of the overall project estimate requiring the project team to decompose the work into very small work packages. 
• Total project budget is many time set even before the scope and schedule are defined. (construction projects funded by bank loans.) A high-level allocation of the budget is created between the likely project deliverables. Each major activity is  estimated and if this estimate is greater than the allocated cost, the timing of resources or scope and deliverables are varied until the project is able to meet the budget goals. This is usually an iterative process. 

So when we hear estimating is the smell of dysfunction and no dysfunctions are listed let alone any credible and in use estimating processes, it's time to question the wisdom is assuming estimates are the problems with software projects.

So let's invert the conversation for a moment.

1. Drip funding is suggested alternative to estimating. But drip funding  just pushes the project estimate back to the project owner, who in turn releases budget. But the capacity for work the productivity of that work capacity still needs to be estimated to determine if there is sufficient budget to complete the needed capabilities on the needed date.
2. Fixed Price funding move the estimating process to the providers of the product. They now need to estimate their capacity for work, risks, margin, and productivity.
3. Decomposing work into  same sized chunks - assuming this is possible - may be useful in some domain. But needs to be tested outside of the anecdotes of those using it.
4. In the end the Return on Investment calculation [ROI=(Value - Cost / Cost)] used by all viable businesses needs to know both cost and value. These are both probabilistic, and the underlying statistical processes driving both these needs to be understood.

In the end it's the business that needs the estimates of the developers have decided they are not useful. It's not the developers money - if it is no one cares how they spend it. The business has a obligation to the shareholders, investors, the funding agency, to have some understanding of what the cost for the products or services will be when they are done.

Related articles

How To and How Not To Make Credible Estimates of Cost and Schedule

Facts and Fallacies of Estimating Software Cost and Schedule

It's an Estimating Problem not A Problem with Estimating

8 Reasons Why Estimates Are Too Low

The Value of Making An Estimate

Answers to Shim's Questions

Shim Marom provides thought provoking posts. The one on Estimates and Not Estimates was a perfect straight line requesting a response.

1. Are cost estimates required in order to manage software projects? - without some forecast of how much the project is going to cost, it's going to be hard for those providing the funds to have any confidence that the project will stay on a budget needed for their target Return on Investment. ROI = (Value - Cost)/Cost. If those funding the project are in the business of making a profit (or operating within the budget they've been assigned), knowing the estimated cost is important to their success.
2. Are cost estimates an effective tool for controlling costs? - controlling costs is a management issue in nearly every domain. But without an estimate of costs we have no target cost for the project management processes. Without a target, we have an open loop control system. No target, no feedback, no corrective actions. With a target we have a closed loop control system, where feedback against the target (the error signal) is used to take corrective actions in support of #1.
3. Do estimates stifle creativity and kill innovation?  - creativity must be driven by the needed capabilities from the customer, the design basis from the requirements derived from those requirements, and the resulting products and services that result from the implementation of the requirements. Estimating the work effort (cost and duration) supports the steering inputs needed for the creativity process. Creativity must support the needed customer capabilities. It's not about the developers, it's about customer need. Without understanding how long and how much, the customer cannot make a decision about the priority of the needed capabilities and the features that implement that capability. It's back to a core concept. You can't know the value of something until you know its cost.
4. Do people need estimates? And if so, why? - this depends on Who the people are. If it's the people providing you money, yes. If you are a consumer of the money, you need some estimate of the planned duration of your work so you can inform the decision makers when you will be delivering the value they have asked for. The purpose of the estimates is to establish a  steering vector for the project's management. Without the steering vector - direction  and speed there is only  open loop control can be applied. You want closed loop control. This is fundamental to the  success of all projects. Using Keny Beck's quote - optimism is the disease of software projects, feedback is the cure. That feedback is used to take corrective action to keep the project headed in the right direction at the right speed to provide the needed (planned) value to the customer on or near the need date at or near the planned cost. The customer is in business to stay in business, and cost control is a crtical success factor for all business - even the business of government.
5. Is the very act of estimation results in the creation of uncertainty? - Yes. An estimate is probabilistic. This understanding is fundamental to the estimating process. Without this understanding the conversation becomes confused. Estimates are not guesses. OK, bad estimates are. Estimates are not point values. They are  Most Likely values with a probabilistic confidence range. Anyone guessing either needs to stop guessing or find someone who know how to estimate to help them.
6. Is estimation a practice still hanging over from the Waterfall era? No, estimating anything in any domain is required to make decisions about the future. How long will it take us to deploy Sharepoint to 11 sites across the nation with 250 total users and 12TB of documents? We need an estimate. Never said what method will be used. Doesn't matter. If we really believe management is about making informed decisions, then management needs to know how much, how long and what. They can't wait till the project is over to find this out. They need an estimate upfront of cost, schedule, and technical performance. 
7. Is No Estimation better than Bad Estimation? This is a nonsense question (no offense Shim). Do we really want to NOT know cost, schedule and technical performance before starting a project and improve that knowledge as the project proceeds? Really?
8. Is  Estimation really just a form of Guessing? Only if you were asleep in the high school statistics class, and didn't pay attention in your analytical methods class in Computer Science school. Anyone suggesting estimates are  guess must have done that. I know that's rude, but come on figure this out. Those suggesting estimates are guess have no idea how the probabilistic and statistical processes of the world work.
9. Are estimates necessary for Governance? Is it reasonable to require estimates for the purpose of pacifying governance needs? - governance is about the proper use of resources for the betterment of the firm. The answer is yes. Legally, public firms must state their obligations for cash flow and commitments to pay. Estimates support this.
10. Is there any point in providing estimates when it is known that many projects fail due to lack of credible estimates? And aren't estimates a tool used to apportion blame afterwards anyway? - the notion of value at risk drives the estimating processes. If these are used to blame than that is simple bad management, not a reason to not estimate. Confusing bad management and bad processes as the reason to change the processes is in itself a waste, since you have not identified the Root Cause of the dysfunction. But first suggesting estimates are the cause of project failure is a red herring. Bad Management is usually the cause in the IT world. But doing the root cause analysis first before stating the cause is hard work. It means not concluding what the solution is, maybe even concluding not even knowing what the real problem is. The somewhat naive approach of five whys is a start. But these is plenty of guidance on the web about Sic Sigma Root Cause Analysis. Using the phrase 5 Whys without referencing some guidance shows that the supposed dysfunctions have yet to be identified.
11. Is costing a necessary tool for determining business value? - You can't know the value of something without knowing its cost. It's that simple and it's that complex. This is fundamnetal to all business processes. Since writing software for money starts and ends with the business process, knowing the cost of that software is needed. Not at the end, but before authorizing the project to spend the companies money for the delivered value. 
12. Does the scope drive the budget or does the budget drive the cost? - this is a contract or business process question. Depends what the customer wants it to be. Internal IT in small firms have more flexibility than larger firms. Governance based firms - insurance, health care, telecommunications, have less flexibility than lawn sprinkler suppliers. Building software for the Air Force has much less flexibilitythan building software for a plastic shoe manufacturer.
13. If estimates are required in order to support decisions, can decisions be made without estimates? Invert the question. What information do you need to make a decision? Do you need to know the cost of the work, the cost of the impact of deciding one way or the other. The cost of the impact of not deciding. 

So some other thoughts:

1. If it's not your money behave appropriately.
2. If you don't write down what DONE looks like in units of measure meanigful to the decision makers, you'll never recognize it until you run out of time and money.
3. Done starts with a set of capabilities the customer neds to put to work to solve a business problem. Knowing the cost of those capabilities is part of the business decision making process.

If No Estimates is about making decisions, then No Estimates better come up with the cost of outcomes of having made that decision or that statement is completely bogus. Time to start looking for  business value from our work efforts in building software for money. That means know the cost of our efforts to some degree of confidence before and during the project. Knowing afterward is simple and worthless, that horse ran away and know we're saying how much it costs.

Related articles

It's an Estimating Problem not A Problem with Estimating

How To Estimate, If You Really Want To

Moving from "Why" to "What" Actions are Needed?

Why Hasn't #NoEstimates Produced Actionable Outcomes Yet?

Book Blog Post

The AMACOM Book Performance-Based Project Management has a blog post from the publisher.

The book can be purchased from the link under the cover to the left. 

Systems Thinking - How To Stop Thinking Small

Much of the discussion on building products and services is focused on eliciting requirements, developing solutions around these requirements. Ignoring for a moment the silliness of not knowing the cost or duration of this work effort, we  still need to address the business side of spending other peoples money in exchange for delivery a known value.

Over the past months I've come to see the world through the eyes of Systems Engineering. I have a MS in Systems Management, but haven't applied it in 25 years.

My current assignment is on a large software intensive system that is a national asset. This is a code word for it has to work as planned or 100's of million of people, sovereigns, and other users will be disappointed at best and possibly be put in harms way at worst.

This systems view means: (Systems Thinking for the Enterprise: New and Energing Perspectives)

• Systems Thinking - seeing the whole, the interrelationships between the parts, and the patterns that emerge from the interactions between the parts.
• Context Awareness - being mindful of the business governance processes, operational, economic, and technical influences and constraints of the domain in which the system operates.
• Accepting Uncertainty - by acknowledging the fundamental principle of all project work - statistical processes drive probabilistic behaviours in ways we may not understand and possibly cannot control. No solution is possible, only risk response in the form of risk retirement or margin.
• Complex Systems Evolve - based on the fundamental principles of the sciences of evolution, ecology, and adaptions, human built systems have the same results. Adaptation, self-organization, and natural selection are in play.

So it all comes down to this:

If you don't know what someting costs - in units of money, schedule, manpower, infrastructure, tools, process risk, delay - you're not going to be able to know what it is worth.

That's it, It's that simple. Anyone suggesting their processes or even a HashTag masquerading as a suggested approach to problems, that this solution will increase the Probability of Project Success (POPS)  or their suggested solution is focused on providing value to the customer, must address how that suggestion will work in the presence of the reality of building products and services in exchange for someone else's money.

What is Earned Value

Earned Value measures the production of tangible outcomes from work efforts on a project using Physical Percent Complete. This P%C is calculated used Quantifiable Backup Data shows what technical or operation goals must be met on a planned date in order to claim some Percent Complete. Measures of this physical percent are NOT how much money you spent or how much time you took to do the work.

They are only measured in unit of tangible technical progress to plan. This tangible technical progress is defined BEFORE the work starts. This progress goal is derived from the time phased Measures of Effectiveness and Measures of Performance goals of the project.

These MOE, MOP, KPP, and TPMs are defined below and their measures - in units meaningful to the decision makers - established before work starts, put on baseline and used to compare progress to plan (time phased) to produce that Physical Percent Complete in the Earned Value (BCWP) formula in the first chart.

 

 

Quote of the Day

This branch of mathematics [probability] is the only one, I believe, in which good writers get results entirely erroneous - Charles Sanders Pierce

When it is said - you can't estimate the future - or we don't know total cost, think of Mr. Pierce. All things project management are probabilistic drive by the underlying statistical processes of irreducible and reducible uncertainty. Rarely, if ever, are these uncertainties Unknowable, in the mathematical sense. 

Performance-Based Project Management(sm) Released

Performance-Based Project Management^(sm) on Amazon

Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.

Performance-Based Project Management(sm) from Glen Alleman

Here's a quick overview of the rest of the book with Principles, Practices, and Processes

5 immutable principles and 5 processes in 60 seconds from Glen Alleman

On a bit more detail about how the parts all come together

Performance based planning in a nut shell (V5) from Glen Alleman

 

 

Related articles

Flaws and Fallacies in Statistical Thinking

What is Project Management About?

Flaws and Fallacies in Statistical Thinking

The processes by which we have discussions about project processes must be based on probability and statisics. Every project is subject to aleatory (irreducible) and epistemic (reducible) uncertainties. When we talk about cost, schedule, and technical performance are all statistical in nature. 

Here's a book that should be read before engaging anyone in a conversation about the processes of project management, estimating, risk, performance management.

Quote of the Day

W. Edwards Deming

 

Showing Up On Time, On Budget, With Needed Capabilities

Software development ranges from straight forward, with small a team with a continuous improvement effort. Say a web site or warehouse management application were the customer is happy to just keep the improvements coming. Every improvement or new feature can be put to use. Along the way new ideas enter into the mix and since there is no real deadline, the features can be deployed when they are ready.

On the other end of this wide spectrum of projects is the mission critical, business critical project. For example a Mergers and Acquisition strategy of two large firms that need to join their ERP systems before any benefits from the merger for the customers, operations, and cash flow can be realized. Below is an example of the Plan for such a project.

The first thing to put to rest is the naive and misinformed notion that planning is somehow not needed. The Plan above is a strategy for the integration of legacy systems with a new ERP system. The Planned date for this system is tied to business success. Long before this Plan was developed, the business strategy identified the needed capabilities of the integrated system, the planned savings from the integrated system, and a customer facing set of abilities of the result. 

This is the fundamental Return on Investment equation. ROI = (Value - Cost)/Cost. We know the value, since the analysis of the value of the integrated system was done before the merger. The cost was estimated at the highest level from experience of integrating ERP in the past. Details come next, usually after the merger. 

Each of the boxes provides a needed capability. End box is an incremental deployment of this capability, with feedback that informs the downstream capabilities. No project can have an end-to-end detailed schedule with any hope of that schedule remaining intact. But the Plan above describes the order of delivery of the needed capabilities. These are not arbitrary, these are not random, these are not subject to change. At least not without understanding the impact on the business merger process and the resulting cost savings and cash flow generation.

So What's The Point

If the value at risk is sufficient to call into question business failure, or at least major issues, if the project fails to meet its goals, then we need a Plan, an estimated cost, and an estimated delivery date. To ignore these business needs is to ignore the obligation to provide that needed information. 

You project may not be a merger of two multi-billion dollar firms. But the question isn't when to estimate but when do we no need to estimate. The asnswer to that starts with the Value at Risk. The business providing the funding gets to say what the value is. 

What is the business willing to Risk on the project without knowing that value before starting?

 

Cost is KING, No Way Out Of It

Cost is KING is a familiar word in business. Overrun our product development cost, ROI on that development is reduced.

So here's an immutable principle of all business:

For a business to stay in business it must manage its costs. Without managing cost there can be no hope of profit. Profit is why a business is in business. Sure there is that "let's change the world" mission statement. Or the "world domination" statement. But without profit those glorious goals cannot be met - unless your a sovereign. So managing cost and knowing cost is a precondition for profit. Profit is a condition for staying in business.

We can know cost from the past - these are called actuals or estimated actuals if the invoice hasn't arrived yet. Knowing costs in the future is called forecasting. Forecasting involves estimating. Estimating future cost is part of the immutable process of staying in business. If we can't forecast future cost, it's going to be hard to hang on to those revenues customers are giving us in exchange for some value we've delivered. 

Enough of Business 101. Now on to project based business processes. Overrun our project cost for an external customer on a non-government job and our profit margin is reduced.

Overrun our baseline cost on a government cost-plus job and you're going to have to explain why we're a bone head to some contracting officer.

Overrun our government contract by 25% or greater and we'll have to explain to congress why you're a bone head that breached the Nunn-McCurdy limit.

Overrun our spend plan for staff, and someone is going to come and ask why we have more people than we planned. Have no plan for that number of people, the balance sheet is going to show the overrun to people you may have never met and they'll be asking us questions we may not have answers for.

Project performance starts with cost management. It may be likely that our salary is a recoverable (allowable) cost. It may be our salary is wrapped into the bill rate to the customer. But no matter the source of funds that pay your salary, the light bill, the really nice capichino maker, the pool table, Income - Cost is retained earnings. If we overrun our Planned cost, we're going to erode what remains from Revenue - Cost = Profit.

Any one not agreeing must be working for a not-for-profit or a non-profit organization. Cost management translates directly into profit margin goals. Profit Margin is revenue minus direct cost plusoverhead, indirects, and all other non-recoverable costs. Profits are what keeps the lights on. Profit is Net Margin.

So Now What?

Cost is one of two numbers in the Return on Investment equation every managerial accounting class taught us. 

ROI = (Value - Cost) / Cost

Unplanned growth in cost reduces ROI. When we hear It's all about delivering customer value, nothing but value, starts and ends with value, ask at what cost? Don't know the cost, you can't determine the NET Value. Simple managerial accounting again. Not developer accounting in story points, not drip accounting in budget fed to your project. Dead Presidents accounting from the bank customers pay into to your payroll account your pay your mortgage from.

Any business not knowing what things cost for the work they are performing, be it internal work or external work, is not going to be in business for long. 

Cost is KING

Once we come to understand that, we can now talk to business people and talk to engineers. Because most engineers work on budget. And budget is not the same as funding. Funding can buy bread at the store, It's real money. Budget is a cell in Excel or a number in SAP, or a number written on the White Board.

So Now What, Really?

If we're on the customer side, or we're in the accounting managers office, or the product managers office, or the account executives office, we'll be having a conversation about the cost of the wonderful work we're doing for our internal or external clients or our customers we're selling our product to. This is how business is run. 

There is a mythical notion that places like Google, Facebook, Twitter don't care about cost. I'd suggest you go look at the 10-K and search for the term cost. You'll see things like cost of revenue. How much do we have to spend to bring in every dollar in revenue. Business runs on cost. It also runs on products, service, cool things no one can produce except for them, stuff we never ever thought about but they did. But when paychecks go out every two weeks to Chase, dead presidents are removed from one account and transferred to another - you're bank account. The cost of building all those cool, unimagined things requires real money. Spend too much - more than planned - profits go down. 

How Can The Business Know What Costs It's Obligated to Pay Next Week, Month, Quarter, Year?

How you ask? They make estimates. Now these estimates may never reach us in the development department. We may only see budget, funding allocation, release of funds, or just a direction to go forth and build cool stuff. But someone, somewhere in the building needs to know the cost of our wonderful work. 

This cost can be flat spread - peanut butter spread as we say where I work. A fixed number of Full Time Equivalents with a base cost and burden (fringe, overhead, and indirects). That's the budget for all the people working here. What happens then is we need to know our capacity for work. What can we produce for that cost? Unless we're building transmissions at Toyota, we'll have a variable cost of production. Somethings we work will cost more per the value produced, some will cost less. Software for example is NOT a production line.  So if we've fixed the cost of production, the value produced will be variable. How variable? Oh I don't know, let's estimate. 

Estimate from past performance. That's the absolute best way. What of there is no past performance, or no applicable past performance? Then estimate anyway. You're seeing a trend here right?

We have to estimate our cost of production in order to know, with any confidence, our future earnings. Where earnings are Revenue minus Cost.

No cost estimating process at the business level is simply nonsense. Now we as a developers may not be involved in cost estimating. But someone is. Let's go find that someone and ask. Do you really not want to know my cost of being here? 

Related articles

Financial Accounting Versus Project Acounting

It's an Estimating Problem not A Problem with Estimating

How To Estimate Almost Anything If You Really Want To

What's Gonna Cost and When Will We Be Done?

Book for Cost and Schedule Forecasting

Quote of the Day

Software intensive project work is indeterminate. Many times, customers don't know what they really want until they see something working. Development's capacity for work is not well understood early in the project. Technology emerges and impacts the effectiveness of the project. Reducible and irreducible uncertainties abound. These conditions are normal, they can't be wished away. 

So what are the options? Let's start with some advice and see what we can do to improve the Probability of Project Success. Wise leaders from the past have words we can use to guide our efforts.

I know one thing: that I know nothing 
— Socrates

Ignorance more frequently begets confidence than does knowledge
— Charles Darwin, The Descent of Man

The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt
— Bertrand Russell

The fool doth think he is wise, but the wise man knows himself to be a fool.
— William Shakespeare, As You Like It

Welcome to Lake Wobegon, where all the women are strong, all the men are good-looking, and all the children are above average.
— Garrison Keillor, A Prairie Home Companion

Knowing nothing about the project is not good for increasing the probability of success. Knowing what capabilities are needed to fulfill a mission or business need is a starting point. Capabilities are not requirements, those come next. Capabilities allow the user to do something of value. This is the value spoken about, most times spoken about in the absence of any units of measure. These measures, the measure of capability is the Measure of Effectiveness. (The tool in the previous link is a systems engineering tool, that provides all project participants with clear, concise, and testable assessments of their understanding of what Done looks like. The notion that tools get in the way of success is ill-informed at best, and just plain wrong at worst).

Measures of Effectiveness are operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in an operational environment, under a specific set of conditions.

They:

• are stated in units meaningful to the buyer or decision maker.
• focus on capabilities independent of any technical implementation.
• are connected to the mission success.

If we don't know what capabilities we want the project to deliver, it's going to be a low probability that the project is a success. We don't know what done looks like. Now we can get paid to discover these capabilities. Capabilities Based Planning is a process to discover these. Someone has to pay for this of course. It is usually the customer that pays. But someone has to pay. 

With the needed capabilities in hand, we can make our first estimate for cost, schedule, and the probability of success. Here's an example. Frank Cepollina was speaking at the BAA (Broad Area Announcement) for the Hubble Robotic Service mission.

He stated his needed capabilities rather than in techncial requirements. Those always come after we know what done looks like. Notice these capabilities are mission capabilities. Get the job done descriptions.

• Do no harm to Hubble, it is very fragile
• Change the wide-field camera, it had failed long after its planned 
• Attach the umbilical battery
• Attach the de-orbit module

How much does that cost and when can you have it ready to fly?

The three vendors needed to develop an estimate. The details of this process can be found in Assessment of the options for extending the life of the Hubble Space Telescope. This example can be extended to IT and software development projects. What do we want the system to do? Don't know? Go find out. Make that the first step in the project. Read Predictabilty: A Simple Approach to Creating Reliable Project Schedules, Steve Bockman for an example of how to think about this in a software development environment. No matter what our domain, size of project, we're going to need to understand something about what capabilities are needed for project success. We might be able just to start work and let these emerge. But our probability of success is unknown and that's not likely a good thing in the end.

So when we know something about the needed capabilities, we can start to discover the requirements. But first we're going to need some kind of measure for those requirements. These are performance measures. 

These are measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. These are Measures of Performance, and are:

• Attributes that assure the system has the capability to perform
• Assessment of the system to assure it meets the design requirements at satisfy the Measures of Effectiveness. 

These are measure of how the requirements are going to perform in pursuit of the mission.

Final we have to discover the Technical Performance Measures. These are attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal. These:

• Assess the effectiveness of the design process.
• Define the compliance to the performance requirements.
• Identify technical risk.
• Are limited to critical thresholds.
• Include projected performance.

Here another need for estimating. How fast do we have to process transactions to meet the needed capability? How much can we weigh and still have the capability to fly? How reliable (mean time between failure, mean time to first failure, mean time to recover from failure), and still meet the needed capabilities?

These estimates are focused on technical outcomes. Thing that fly away, things that provide services in support of the customer. Things. 

Where are we going here?

We must learn to estimate all kinds of things on a successful project. The credibility of our skills as project managers is always tied up with our ability to estimate. Can't estimate, choose not to estimate, see estimating as some kind of dysfunction, it is doubtful those providing us with money are going to have much confidence in our ability to deliver that mystical value we are told is part of all projects.

Estimating is part of project work. To not estimate, means we won't know what done looks like until we arrive at the end. That's unlikley to be acceptable to those paying us to do work. 

The next step is to determine what requirements will be needed to implement the needed capabilities. These are defined as Measures of Performance that characterize physical or functional attributes relating to the system of operation, measured or estimated under specific conditions.

These are:

• Attributes that assure the system has the capability to perform.
• Assessment of the system to assure it meets the design requirements that satisfy the Measures of Effectiveness.

Notice something important here. Software project success always depends on development of working  software. But what software, in what order, with what Technical Performance Measures, Measures of Performance, and Measures of Effectiveness all supporting the needed Capabilities.

Success doesn't start with coding. It's about knowing the answers to the CBP, MOE, MOP, TPM attributes. The notion of emergent anything works only when we can state in some unit of measure what DONE looks like. Without that we're just wasting the customer money exploring, challenging everything, and looking for the mystical Unicorn hiding in the bushes.

So Here's the Punch LIne

When we spend other people's money on our project, private money, public money, government money, they have the expectation of knowing something about how much much and when we will be finsihed with the work in exchange for that money. This is pretty much the case for every project I've ever worked for the past 34 years. It's the basis of every successful business - knowing what your costs are for the operations. So when we hear about developing software  for money and not having any concern about the cost of that effort it simply doesn't make any sense.

Related articles

Elements of Project Success

The "Real" Root Cause of IT Project Failure

Risk Management for Dummies

Requirements Elicitation

Agile Project Management

How Not To Develop What "Done" Look Like

Performance Based Management

Five Core Processes of Project Success

8 Reasons Why Estimates Are Too Low

The post 8 Reasons Estmates are too Low, is one of those pieces of material that on the surface seems plausible but has series flaws. First is the restating bad management practices and then arguing against them. This seems all too common in the Agile domain for some reason.

A poster campaign at Rocky Flats in the late 90's for safety and safeguards, but usable everywhere is...

Don't Do Stupid Things on Purpose

If we ignore the red herring approach of doing stupid things on purpose and then tilting at the resulting windmill, let's look further for each idea in the post. The picture to the right is used when engaging in a conversation about making improvement, but starting with a credible baseline. This is called Dead Horse on a Stick. Thanks to the Master Systems Engineer on our program for this concept. He uses this when he starts a review and the ideas are dead before he got there. It's also appropriate for concepts that are dead on arrival, like suggesting that those paying for products don't have a need to know how much it's going to cost, before they start spending money.

1. Super Hero Estimates - the use of a super hero is a well known flaw on all projects. Don't do this. Just say no to the super hero. Use Reference Class Forecasting. Use past performance. Use reference designs. Find like products. Have a Red Team to assess all estimates. Have the super hero contribute an estimate - the possible anchor. This number can be the most optimal. Then adjust that estimate accordingly. This anchoring and adjustment process is well defined in Tversky and Kahneman's work.
2. Wrong Team - simple - get the right team. Much harder than this of course. But without the right team, the project is headed to the ditch. Knowing its headed to the ditch might provide incentives to get the right people. This is called triage. 
3. Guessing in the Dark - stop guessing in the dark. Do your job as an informed engineer. Start with Reference Classes. Install a Red Team. This is one issue with estimating. But can be fixed with good estimating processes. There are lots of sources for advice. 
4. Forgetting Stuff - this is the role of the WBS. It's the all in product and services for the project. If you forget things, you don't do a good job of the WBS. The WBS is usually derived from the Statement of Work (SOW), Concept of Opeerations (ConOps), Statement of Operations (SOO), or some description of what the customer wants. If you don't have one, go get something that describes enough detail to start the project, then develop that understanding as you go. In the end if you don't know what capabilities the customer wants, your project is not likely to be a success before you even start.
5. Ball of Mud Project - this is bad architecture. It's more common that desired. But it's a bad basis to start a projet. Spend time and money to untangle the project, otherwise you're laying the seeds of disappointment before you start.
6. Multitasking - there should be ZERO doubt that multi-tasking is bad. Don't do it.
7. Mythical Man Month - this is bad management. Don't do it. 
8. Lazy Developer - really? Time to replace that person.

Quick Summary

1. Get a reference class. Don't have one. Get a reference class
2. Get the right people
3. Build a model of the cost and schedule drivers
4. List the deliverables for the project against the needed capabilities. 
5. Unscramble the mess before starting. Can't do that? Don't start
6. Stay focused
7. Plan needed staff
8. Get people who will contribute

So what's the point?

When we look at making improvements to anything from projects and sepnding other people's money, to better peddling of my road bike on century rides - start with

stop doing stupid things on purpose.

Only then can you make real and lasting improvements. If you don't do that, you are beating a dead horse.

Quote of the Day

Such as are your habitual thoughts, such also will be the character of your mind; for the soul is dyed by the thoughts. - Marcus Aurelius

How Not To Develop What "Done" Looks Like

The common picture of requirements elicitation looks like this. Which of course is an example of doing stupid things on purpose. When this picture is used as an example of not doing something because it doesn't turn out right, is a further example of doing stupid things.

Let's see where the gaps appear that results in the outcome in the last panel:

• How the customer explained it - was there a Concept of Operations? How about a Statement of Objectives? Some narrative of what the system would do when it is working? Use Cases? Scenarios? NO? Then what the customer explained cannot be tested in any meaningful way.
• How the project leader understood it - here's where the failure starts. There is no confirmation in units of measure meaningful to the decision makers on what DONE looks like. What are the Measures of Effectiveness that are the goals the customer can assess and agree to?
• How the analyst designed it - what is the baseline description of the system the analyst starts with. What are the Measures of Performance of the system? How will those be measured?
• How the programmer wrote it? - what's the build to plan? How does the developer  know what to build? How will it be tested? 
• etc etc etc

The wheels fall off when there is no description of DONE shared between the customer and the provider. If the customer doesn't know what DONE looks like, who does? The developers? Probably not. Is DONE emerging? Then the customer has to pay for that? 

There is no way out of the need to know what DONE looks like in Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and Technical Performance Measures. Without these the success of the project is in doubt from the start.

The notion that requirements emerge is will established. But the capabilities the customer needs to be stable enough to establish an Estimate At Complete and an Estimated Duration to Complete. Without these the customer has no understanding of the all in cost of the project. These change as the requirements change. That's part of the project management process. This can't be ignored if the customer is to have any confidence in the project providing value at the needed time at needed to be put to use. 

So a reminder one more time...

You can't assess the value of the project outcome without knowing something about the cost to deliver that value and the time frame for that delivery.

No matter what anyone says, this is an immutable business principle. Anything is simply fooling yourself into believing that the customer doesn't care how you spend her money or when you spend it.

Focus on value, that's what the customer bought. Actually they bought a capability to do something to improve their business or mission. But ignoring the cost of delivering that value is ignoring the balance sheet of the business.

 

Project Management and the Three Body Problem

Projects are composed of three fundamental elements. Cost, Schedule, and  Technical outcomes. The Technical Outcomes go far beyond the PMI-style scope terms. In this paradigm, the technical outcomes are at the end of a chain. Here's examples of that chain - Capabilities, Measures of Effectiveness, Measures of Performance, Key Performance Parameters (there are 5 in our domain), and Technical Performance Measures. At the TPM level is where things like quality live, traceable to the KPPs.

These three elements are coupled in dynamic ways. Their connections are springy, in that changes in one has an impact on the other two, But rarely is this impact linear and ridgid. The Iron Triangle notion is really a Three Body problem, in which all three element impact each other and at the same time respond to that impact.

All projects have these three elements, coupled in this way. Changes in one impact the other two. Changes in two impact each other and the third. Without knowing the dynamics of cost, schedule, and technical performance, we can't have any credible understanding of these variables. 

Three Body Problem

The three body problem determines the possible motions of three point masses m₁, m₂, and m₃, which attract each other according to Newton's law of inverse squares. It started with the perturbative studies of Newton himself on the inequalities of the lunar motion. In the 1740s there was a search for solutions (or at least approximate solutions) of a system of ordinary differential equations by the works of Euler, Clairaut and d'Alembert (with an explanation by Clairaut of the motion of the lunar apogee).

Developed further by Lagrange, Laplace, and their followers, the mathematical theory entered a new era at the end of the 19^th century with the works of Poincaré and since the 1950s with the development of computers. While the two-body problem is integrable and its solutions completely understood, solutions of the three-body problem (Java 7 in 64 bit Browser needed) may be of an arbitrary complexity and are very far from being completely understood.

The forces between the bodies can be self attractive or they can be a central force - the restrictive three body problem. Or a combination of the two. This is the basis of complex systems, where multiple forces are applied to  objects, which in turn change the forces. As an aside, the double pendulum and the three body problem are used as examples of complex systems. Without acknowledging that the underlying mathematics is deterministic since the Java example above draws the lines from an algorithm.

This is a common mistake by those unable to do the math, or who want to suggest the problems of the day are beyond solution.

Three Body Problem and Three Elements of Project Management

The three body problem uses gravity as the force between the masses. There is a simpler example of three masses connected with three springs. This model is found in chemistry and biology, at the molecular level. Gravity is not in effect of course, but electromagnetic force.

Consider a simplified model for the vibrations of an ozone molecule consisting of three equal oxygen atoms. The atoms are represented by three equal point masses in  equilibrium positions at the vertices of an equilateral triangle. They are connected by equal springs of constant k that lie along the arcs of the circle circumscribing the triangle. Mass points and springs are constrained to move on the circle, so that, e.g., the potential energy of a spring is determined by the arc length covered.

These class of problems are called soft body dynamics. The visible outcome is the rendering of graphical objects that are flexible in movies and games - like three dimensional garments.

Now to Projects

If we assume for the moment that cost, schedule, and technical performance are dynamic variables, with forces between them described by their functional equations. In our functional equations for the force between them ar not constant, but are relationships like this:

• Cost is a function of schedule - Time is money in this case.
• Schedule is a function of cost - We can buy time, shorten schedule, but at what cost?
• Cost is a function of technical performance - Want to got fast? How much will that cost?
• Cost is a function of schedule - This is not a 1:1 exchange for the schedule question.
• Technical performance as a function of cost - What can we afford to do?
• Technical performance as a function of schedule - When can we have fast?

The interaction between the three core elements (cost, schedule, technical) is a two way interaction, so the spring analogy is not quite correct, since the spring force doesn't know which end it is pushing or pulling.

The Point

It's not the Iron Triangle, it's a springy triangle. The connections are non-linear and most importantly they are probabilistically driven by the underlying statistical processes of the project. Let's start with the picture below.

All project processes are probabilistic. They have behaviours that are not fixed. The notion that you can slice work into same sized chunks and execute these chunks with the same effort would violate the basic aleatory uncertainties of all work processes. With an understanding of the statistical processes, driven by either aleatory or epsitemic uncertainties, is followed by asking probabilistic questions. What's the probability that we'll complete on or before a date or what's the probability we'll compete at or below a cost.

With a probability and statistics foundation, we can now put together a credible plan, driven by the underlying stochastic. All work is connected in dependent ways. The work effort, it's duration, and outcomes is also statistically driven. This picture is typical of such a project.

In The End

• We need to know all three properties.
• We need to know the underlying statistical behaviours. If not go find out or pick a reference class.
• We need to speak about confidence intervals, not deterministic value (single point values).
• We need to know something about cost. If you don't you can't know value or ROI. It's that simple. It's basic financial management. 

So we can:

• Learn to estimate.
• Understand that those with the money need to know how much money and when to release that money.
• Those waiting for the value need to know when they will receive that value.
• Those consuming other peoples money, to produce that value, need to have some notion of how time, money, and production of value take place outside their narrow world of it's all about me. 

Related articles

Why Projects Fail - Top 10 Reasons

The 5 P's of Project Management Success

Why the Triple Constraint Notion is a Bad Idea

A Seat At The Table

In a conversation on Twitter with Sridhar, there was mention that product development software projects are different than internal IT projects. And that how they are run is much different as well. This may be the case. It's not been the case in my experience in Enterprise IT for those internal projects. My experience in product development (FileNet, Triconex, Kontron) and software intensive program delivery for government, defense, space, and energy clients is that projects are expected to show up on or before the planned date, at or below the planned cost and with the agreed set of minimal capabilities to accomplish the business goals or fulfill the needed mission.

When internal IT projects lose their ability to show up on time, on budget, with the needed capabilities - all within the probabilistically defined bounds of those measures - then they lose their seat at the table. Internal IT becomes a cost. Sometimes a sunk cost. An expense. A necessary expense, but an expense all the same. And when you're only an expense, those making decisions tend to treat you differently. You have no tangible way to show your worth. In business worth is defined by revenue or cost savings. In both cases worth is defined in units of money. When you have only cash outflow you're likely not to be in the conversation about how to run the business.

There are endless books, articles, and academic papers on the topic of getting a seat at the table. Successfully delivering internal IT in support of the business. Keeping your seat the table on that business requires very specific behaviours, starting with three simple questions:

• How much will this cost?
• When will you be done?
• Will it work at needed when you're done?

But the bigger question - the killer question is what are we building and why are we building it? Ignoring for the moment the problem of showing up late and over budget dilutes any value produced by the project, some times to ZERO, we must have a notion of what DONE looks like in units of measure meaningful to the decision makers. Those paying the bills for the project.

Below is a briefing of how to answer these governance questions about what and why. With that in hand the how much, when, and working are straight forward. 

Using balanced scorecard to build a project focused org2 from Glen Alleman

Here are two books that have informed my journany on the path to providing value through the management of Enterprise IT projects.

• IT Governance: How Top Performers Manage IT Decisiion Right for Superior Results, Peter Weill and Jeanne W. Ross, HBS Press, 2004
• Reinventing Project Management: The Diamond Approach to Sucessful Growth and Innovation, Aaron J. Shenhar and Dov Dvir, HBS Press, 2007

There are many books on Balanced Scorecard. Some I find best are:

• Strategic Performance Management: Leveraging and measuring your intangible value drivers, Bernard Marr, Elesiver, 2006.
• Balanced Scorecard Diagnostics: Maintaing Maximum Performance, Paul R. Niven, John Wiley and Sons, 2005.
• Strategy Maps: Converting Intangible Assets into Tangible Outcomes, Robert S. Kaplan and David P Norton, HBS Press, 2004.

At The End of the Day

If you can't say, with some degree of confidence, how much you plan to spend, over what period of time, to deliver some tangible value to those paying for your work, you're not in the room when those paying for the work ae deciding what they want you to do. You're considered an expense - possibly a valuable expense. 

Seven Immutable Activities of Project Success

There are Principles, Practices, and Processes of project success. I have a book coming that describes them. These are independent of any method, any domain, any favorite tool, buzz words, or personal paradigm. Much research on Root Causes of project failure have led to these Principles, Practices, and Processes. The Principles don't change, can't change. The Practices need to be tailored to the needs of the project and its paradigm. The Processes are localized.

But around these are 7 Activities of any project, that if not performed in some way, the probability of success is reduced, sometimes to Zero.

1. Plan the scope of work.
2. Break down the project scope of work into finite pieces that can be assigned to a responsible person or an organization for control of the technical, schedule, and cost objectives.
3. Integrated the project work scope, schedule, and cost objectives into an agreed upon plan against which accomplishments can be measured. Control the changes to this plan.
4. Capture the actual costs of doing the work and record the accomplishments produced by this work in units of measure meaningful to the decision makers.
5. Objectively assess the accomplishments at the level the work is performed in those same units of measures
6. Analyze any significant variances from the plan, forecast the impacts of these variances, and prepare an Estimate At Completion based on the performance to date of the work performed. Define corrective actions.
7. Report this information to the project team and customer, so agreement on corrective actions can result.

Domain and Context

No matter if your building bridges across rivers, a flying machine to Mars, the next anti-virus drug, or writing software using Scrum for your internal web site, these activities must take place in some way. 

1. Scope - We can't have a project if we don't have some notion of what DONE looks like. What does the customer want the result to do? What capabilities does the customer want to possess when the project is complete. If the customer doesn't know, then money and time has to be spent to discover this. The Agile development method spends money to discover the requirements. This is many times the right thing to do. But time and money have to be spent.
2. Work Breakdown - As we discover what DONE looks like it may change. This is the nature of projects. As we discover new descriptions of DONE, the breakdown of the work has to change with it, otherwise those performing the work, won't know what the new DONE looks like.
3. Integrating People and Work - this is a straight forward process. Assign people to work. The key is self-organizing, can't be the same as self-directed. Direction comes from external sources. Organization comes from internal sources. 
4. Capture Costs - how much something costs is compared against how much something was planned to cost. This feedback provided steering inputs for future cost forecasting. Without this feedback there is not learning process, some not improvement is possible without feedback learning.
5. Assess accomplishments - the only true assessment of accomplishment comes from customer feedback. Did the delivered product or service fulfil the needed capabilities? Measures of Effectiveness are the assessment of those capabilities.
6. Analyze variances - with a plan and the actuals, you can get feedback from your performance. This is fundamental in all project domains. Kent Beck's quote optimism is the disease of software development, feed back is the cure, is applicable to all project domains, not just software.
7. Report actionable information - with the variances against plan, you can then take corrective actions to get back on plan. 

Without the last 4 activities, you're managing the project open loop. This might be OK if your customer doesn't care how much it will cost or how long it will take to finish the defined work. But of they do, you'll need a target cost and a target completion date, the feedback that you're making progress to  those values. And of course some confidence that those targets are credible, their degree of variance, and how they are changing as the project progresses.

 

So one final comment

When we hear that we don't need to estimate, there is no way to have a closed loop control system. Without on estimate of the cost, schedule, and technical performance goals, the only management control system is open loop. This is unlikley to lead to success for any non-trivial project.

Related articles

Sources of Variance

Evidence of 5 Immutable Principles of Project Success

Risk Management for Dummies

Elements of Project Success

There are three key elements of every project on the planet - Cost, Schedule, and the performance of the product or service produced by the project. Each of these has drivers. The connections between Cost, Schedule, and Technical Performance are not Iron as suggested in the Iron Triangle of a PMI view of the project. Instead the connections are elastic, springy, flexible. But they are connected.

Drivers of the Three Elements

Cost is driven by:

• Labor - how many people do we have on this project?
• Materials - what's or raw material cost and what's the conversation rate for that material?
• When we spend money, how effecient are we? What's our efficacy of funds? Do we get - at a minimum -  $1.00 in return for every $1.00 spent?
• What's our overhead rate, our fully burdened costs of labor and materials?
• What our other indirect costs? Benefits, facilities? 

These costs are themselves variable, as a function of the project phase, external forces for labor, materials, and overhead. But the cost variable starts with these.

Schedule is driven by:

• The irreducible uncertainties of the work being performed. All work durations are random variables. The shape of the Probability Distribution Function can be known, but the exact probability of occurance of any one duration for any one task can not. So to protect from this uncertainty we need margin. Schedule margin, cost margin, techncial margin.
• The reducible uncertainties are event based. There is a probability that the material we order will not arive as planned and we'll have a delay in our project and have to pay for labor waiting to start work. We can spend money to reduce or eliminate reducible uncertainty. There is a probability that when we test the new data based server with the actual data, it will not perform as needed. We can spend money and time to test the scalability of the database.
• The capacity for work impacts schedule. We are not a clever as we thought. We are not as capable of working the planned number of hours as we thought. Our capacity of doing the work is impacted in ways we didn't think of.

Technical Performance is driven by:

• Unrealistic expectations of the Effectiveness and Performance of the resutling technology
• We have unanticipated risk.
• Our solution doesn't scale, isn't as reliable as we need, is harder to repair, doesn't meet the technical, performance, or effectiveness goals.

So What Does All This Mean?

• The three elements of a project are not independent.
• One impacts others.
• Two impact the third.
• There are constant tradeoffs.
• All elements are random processes, possibly known, sometimes unknown.

But for project success we need to have several things in place. The random behavior has to be knowable. It can't be chaos. If it is chaos, the project will fail, because there is no corrective action.

The three elements need to be known to some degree of confidence.

• What's the target schedule.
• When does the customer need to outcomes of the project?
• What are the delivery times for the intermediate outcomes of the project, so the customer can look at them and determine if they are what was needed before its too late to correct them?
• How much is this project going to cost when we're done?
• How much money will we need along the way?
• In exchange for this cost, what is the beneficial outcome, to offset the cost.
• This is the fundamental  Return on Investment calculation ROI = (Benefit - Cost) / Cost. If we don't know the cost we can't know the value.
  
• We may know the cost, because it is fixed, then we need to know the schedule and the value produced in exchange for that fixed cost and schedule in terms of capabilities.
• What are we building?
• Is it the right thing to build? How do we know? Do we have some Concept of Operations, a Statement of Work, or Statement of Objectives, sticky notes on the white, 3x5 cards, something that says what DONE looks like in units of measure meaningful to the decision makers.
• How can we assure that what we're building meets the needs to the customer?
• What the Measures of Effectiveness that state the operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions?
• What are the Measures of Performance that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions?
• What are the Key Performance Parameters that represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program?
• What are the Technical Performance Measures that are the attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal?

Do know these to some degree of confidence, you don't know what DONE looks. The only measure of progress becomes the passage of time and consumption of money. It's unlikely any customer is going to be willing to pay you - at least for very long - to spend their money, without some understand of Cost, Schedule, and the resulting Technical Outcomes.

 

Related articles

The "Real" Root Cause of IT Project Failure

A Fundamental Principle In Managing Uncertainty

The 5 P's of Project Management Success

Both Aleatory and Epistemic Uncertainty Create Risk

Five Core Processes of Project Success

Quote of the Day

Next time someone complains that you have made a mistake, tell them that may be a good thing. Because without imperfection, neither you nor I would exist - Stephen Hawking.

Courtesy Don Yeager's daily posts