Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
Software development ranges from straight forward, with small a team with a continuous improvement effort. Say a web site or warehouse management application were the customer is happy to just keep the improvements coming. Every improvement or new feature can be put to use. Along the way new ideas enter into the mix and since there is no real deadline, the features can be deployed when they are ready.
On the other end of this wide spectrum of projects is the mission critical, business critical project. For example a Mergers and Acquisition strategy of two large firms that need to join their ERP systems before any benefits from the merger for the customers, operations, and cash flow can be realized. Below is an example of the Plan for such a project.
The first thing to put to rest is the naive and misinformed notion that planning is somehow not needed. The Plan above is a strategy for the integration of legacy systems with a new ERP system. The Planned date for this system is tied to business success. Long before this Plan was developed, the business strategy identified the needed capabilities of the integrated system, the planned savings from the integrated system, and a customer facing set of abilities of the result.
This is the fundamental Return on Investment equation. ROI = (Value - Cost)/Cost. We know the value, since the analysis of the value of the integrated system was done before the merger. The cost was estimated at the highest level from experience of integrating ERP in the past. Details come next, usually after the merger.
Each of the boxes provides a needed capability. End box is an incremental deployment of this capability, with feedback that informs the downstream capabilities. No project can have an end-to-end detailed schedule with any hope of that schedule remaining intact. But the Plan above describes the order of delivery of the needed capabilities. These are not arbitrary, these are not random, these are not subject to change. At least not without understanding the impact on the business merger process and the resulting cost savings and cash flow generation.
So What's The Point
If the value at risk is sufficient to call into question business failure, or at least major issues, if the project fails to meet its goals, then we need a Plan, an estimated cost, and an estimated delivery date. To ignore these business needs is to ignore the obligation to provide that needed information.
You project may not be a merger of two multi-billion dollar firms. But the question isn't when to estimate but when do we no need to estimate. The asnswer to that starts with the Value at Risk. The business providing the funding gets to say what the value is.
What is the business willing to Risk on the project without knowing that value before starting?
Software intensive project work is indeterminate. Many times, customers don't know what they really want until they see something working. Development's capacity for work is not well understood early in the project. Technology emerges and impacts the effectiveness of the project. Reducible and irreducible uncertainties abound. These conditions are normal, they can't be wished away.
So what are the options? Let's start with some advice and see what we can do to improve the Probability of Project Success. Wise leaders from the past have words we can use to guide our efforts.
I know one thing: that I know nothing
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt
— Bertrand Russell
The fool doth think he is wise, but the wise man knows himself to be a fool.
— William Shakespeare, As You Like It
Knowing nothing about the project is not good for increasing the probability of success. Knowing what capabilities are needed to fulfill a mission or business need is a starting point. Capabilities are not requirements, those come next. Capabilities allow the user to do something of value. This is the value spoken about, most times spoken about in the absence of any units of measure. These measures, the measure of capability is the Measure of Effectiveness. (The tool in the previous link is a systems engineering tool, that provides all project participants with clear, concise, and testable assessments of their understanding of what Done looks like. The notion that tools get in the way of success is ill-informed at best, and just plain wrong at worst).
Measures of Effectiveness are operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in an operational environment, under a specific set of conditions.
If we don't know what capabilities we want the project to deliver, it's going to be a low probability that the project is a success. We don't know what done looks like. Now we can get paid to discover these capabilities. Capabilities Based Planning is a process to discover these. Someone has to pay for this of course. It is usually the customer that pays. But someone has to pay.
With the needed capabilities in hand, we can make our first estimate for cost, schedule, and the probability of success. Here's an example. Frank Cepollina was speaking at the BAA (Broad Area Announcement) for the Hubble Robotic Service mission.
He stated his needed capabilities rather than in techncial requirements. Those always come after we know what done looks like. Notice these capabilities are mission capabilities. Get the job done descriptions.
How much does that cost and when can you have it ready to fly?
The three vendors needed to develop an estimate. The details of this process can be found in Assessment of the options for extending the life of the Hubble Space Telescope. This example can be extended to IT and software development projects. What do we want the system to do? Don't know? Go find out. Make that the first step in the project. Read Predictabilty: A Simple Approach to Creating Reliable Project Schedules, Steve Bockman for an example of how to think about this in a software development environment. No matter what our domain, size of project, we're going to need to understand something about what capabilities are needed for project success. We might be able just to start work and let these emerge. But our probability of success is unknown and that's not likely a good thing in the end.
So when we know something about the needed capabilities, we can start to discover the requirements. But first we're going to need some kind of measure for those requirements. These are performance measures.
These are measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. These are Measures of Performance, and are:
These are measure of how the requirements are going to perform in pursuit of the mission.
Final we have to discover the Technical Performance Measures. These are attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal. These:
Here another need for estimating. How fast do we have to process transactions to meet the needed capability? How much can we weigh and still have the capability to fly? How reliable (mean time between failure, mean time to first failure, mean time to recover from failure), and still meet the needed capabilities?
These estimates are focused on technical outcomes. Thing that fly away, things that provide services in support of the customer. Things.
Where are we going here?
We must learn to estimate all kinds of things on a successful project. The credibility of our skills as project managers is always tied up with our ability to estimate. Can't estimate, choose not to estimate, see estimating as some kind of dysfunction, it is doubtful those providing us with money are going to have much confidence in our ability to deliver that mystical value we are told is part of all projects.
Estimating is part of project work. To not estimate, means we won't know what done looks like until we arrive at the end. That's unlikley to be acceptable to those paying us to do work.
The next step is to determine what requirements will be needed to implement the needed capabilities. These are defined as Measures of Performance that characterize physical or functional attributes relating to the system of operation, measured or estimated under specific conditions.
Notice something important here. Software project success always depends on development of working software. But what software, in what order, with what Technical Performance Measures, Measures of Performance, and Measures of Effectiveness all supporting the needed Capabilities.
Success doesn't start with coding. It's about knowing the answers to the CBP, MOE, MOP, TPM attributes. The notion of emergent anything works only when we can state in some unit of measure what DONE looks like. Without that we're just wasting the customer money exploring, challenging everything, and looking for the mystical Unicorn hiding in the bushes.
So Here's the Punch LIne
When we spend other people's money on our project, private money, public money, government money, they have the expectation of knowing something about how much much and when we will be finsihed with the work in exchange for that money. This is pretty much the case for every project I've ever worked for the past 34 years. It's the basis of every successful business - knowing what your costs are for the operations. So when we hear about developing software for money and not having any concern about the cost of that effort it simply doesn't make any sense.
The common picture of requirements elicitation looks like this. Which of course is an example of doing stupid things on purpose. When this picture is used as an example of not doing something because it doesn't turn out right, is a further example of doing stupid things.
Let's see where the gaps appear that results in the outcome in the last panel:
The wheels fall off when there is no description of DONE shared between the customer and the provider. If the customer doesn't know what DONE looks like, who does? The developers? Probably not. Is DONE emerging? Then the customer has to pay for that?
There is no way out of the need to know what DONE looks like in Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and Technical Performance Measures. Without these the success of the project is in doubt from the start.
The notion that requirements emerge is will established. But the capabilities the customer needs to be stable enough to establish an Estimate At Complete and an Estimated Duration to Complete. Without these the customer has no understanding of the all in cost of the project. These change as the requirements change. That's part of the project management process. This can't be ignored if the customer is to have any confidence in the project providing value at the needed time at needed to be put to use.
So a reminder one more time...
You can't assess the value of the project outcome without knowing something about the cost to deliver that value and the time frame for that delivery.
No matter what anyone says, this is an immutable business principle. Anything is simply fooling yourself into believing that the customer doesn't care how you spend her money or when you spend it.
Focus on value, that's what the customer bought. Actually they bought a capability to do something to improve their business or mission. But ignoring the cost of delivering that value is ignoring the balance sheet of the business.
Projects are composed of three fundamental elements. Cost, Schedule, and Technical outcomes. The Technical Outcomes go far beyond the PMI-style scope terms. In this paradigm, the technical outcomes are at the end of a chain. Here's examples of that chain - Capabilities, Measures of Effectiveness, Measures of Performance, Key Performance Parameters (there are 5 in our domain), and Technical Performance Measures. At the TPM level is where things like quality live, traceable to the KPPs.
These three elements are coupled in dynamic ways. Their connections are springy, in that changes in one has an impact on the other two, But rarely is this impact linear and ridgid. The Iron Triangle notion is really a Three Body problem, in which all three element impact each other and at the same time respond to that impact.
All projects have these three elements, coupled in this way. Changes in one impact the other two. Changes in two impact each other and the third. Without knowing the dynamics of cost, schedule, and technical performance, we can't have any credible understanding of these variables.
Three Body Problem
The three body problem determines the possible motions of three point masses m1, m2, and m3, which attract each other according to Newton's law of inverse squares. It started with the perturbative studies of Newton himself on the inequalities of the lunar motion. In the 1740s there was a search for solutions (or at least approximate solutions) of a system of ordinary differential equations by the works of Euler, Clairaut and d'Alembert (with an explanation by Clairaut of the motion of the lunar apogee).
Developed further by Lagrange, Laplace, and their followers, the mathematical theory entered a new era at the end of the 19th century with the works of Poincaré and since the 1950s with the development of computers. While the two-body problem is integrable and its solutions completely understood, solutions of the three-body problem (Java 7 in 64 bit Browser needed) may be of an arbitrary complexity and are very far from being completely understood.
The forces between the bodies can be self attractive or they can be a central force - the restrictive three body problem. Or a combination of the two. This is the basis of complex systems, where multiple forces are applied to objects, which in turn change the forces. As an aside, the double pendulum and the three body problem are used as examples of complex systems. Without acknowledging that the underlying mathematics is deterministic since the Java example above draws the lines from an algorithm.
This is a common mistake by those unable to do the math, or who want to suggest the problems of the day are beyond solution.
Three Body Problem and Three Elements of Project Management
The three body problem uses gravity as the force between the masses. There is a simpler example of three masses connected with three springs. This model is found in chemistry and biology, at the molecular level. Gravity is not in effect of course, but electromagnetic force.
Consider a simplified model for the vibrations of an ozone molecule consisting of three equal oxygen atoms. The atoms are represented by three equal point masses in equilibrium positions at the vertices of an equilateral triangle. They are connected by equal springs of constant k that lie along the arcs of the circle circumscribing the triangle. Mass points and springs are constrained to move on the circle, so that, e.g., the potential energy of a spring is determined by the arc length covered.
Now to Projects
If we assume for the moment that cost, schedule, and technical performance are dynamic variables, with forces between them described by their functional equations. In our functional equations for the force between them ar not constant, but are relationships like this:
The interaction between the three core elements (cost, schedule, technical) is a two way interaction, so the spring analogy is not quite correct, since the spring force doesn't know which end it is pushing or pulling.
It's not the Iron Triangle, it's a springy triangle. The connections are non-linear and most importantly they are probabilistically driven by the underlying statistical processes of the project. Let's start with the picture below.
All project processes are probabilistic. They have behaviours that are not fixed. The notion that you can slice work into same sized chunks and execute these chunks with the same effort would violate the basic aleatory uncertainties of all work processes. With an understanding of the statistical processes, driven by either aleatory or epsitemic uncertainties, is followed by asking probabilistic questions. What's the probability that we'll complete on or before a date or what's the probability we'll compete at or below a cost.
With a probability and statistics foundation, we can now put together a credible plan, driven by the underlying stochastic. All work is connected in dependent ways. The work effort, it's duration, and outcomes is also statistically driven. This picture is typical of such a project.
In The End
So we can:
In a conversation on Twitter with Sridhar, there was mention that product development software projects are different than internal IT projects. And that how they are run is much different as well. This may be the case. It's not been the case in my experience in Enterprise IT for those internal projects. My experience in product development (FileNet, Triconex, Kontron) and software intensive program delivery for government, defense, space, and energy clients is that projects are expected to show up on or before the planned date, at or below the planned cost and with the agreed set of minimal capabilities to accomplish the business goals or fulfill the needed mission.
When internal IT projects lose their ability to show up on time, on budget, with the needed capabilities - all within the probabilistically defined bounds of those measures - then they lose their seat at the table. Internal IT becomes a cost. Sometimes a sunk cost. An expense. A necessary expense, but an expense all the same. And when you're only an expense, those making decisions tend to treat you differently. You have no tangible way to show your worth. In business worth is defined by revenue or cost savings. In both cases worth is defined in units of money. When you have only cash outflow you're likely not to be in the conversation about how to run the business.
There are endless books, articles, and academic papers on the topic of getting a seat at the table. Successfully delivering internal IT in support of the business. Keeping your seat the table on that business requires very specific behaviours, starting with three simple questions:
But the bigger question - the killer question is what are we building and why are we building it? Ignoring for the moment the problem of showing up late and over budget dilutes any value produced by the project, some times to ZERO, we must have a notion of what DONE looks like in units of measure meaningful to the decision makers. Those paying the bills for the project.
Below is a briefing of how to answer these governance questions about what and why. With that in hand the how much, when, and working are straight forward.
There are many books on Balanced Scorecard. Some I find best are:
At The End of the Day
If you can't say, with some degree of confidence, how much you plan to spend, over what period of time, to deliver some tangible value to those paying for your work, you're not in the room when those paying for the work ae deciding what they want you to do. You're considered an expense - possibly a valuable expense.
There are Principles, Practices, and Processes of project success. I have a book coming that describes them. These are independent of any method, any domain, any favorite tool, buzz words, or personal paradigm. Much research on Root Causes of project failure have led to these Principles, Practices, and Processes. The Principles don't change, can't change. The Practices need to be tailored to the needs of the project and its paradigm. The Processes are localized.
But around these are 7 Activities of any project, that if not performed in some way, the probability of success is reduced, sometimes to Zero.
Domain and Context
No matter if your building bridges across rivers, a flying machine to Mars, the next anti-virus drug, or writing software using Scrum for your internal web site, these activities must take place in some way.
Without the last 4 activities, you're managing the project open loop. This might be OK if your customer doesn't care how much it will cost or how long it will take to finish the defined work. But of they do, you'll need a target cost and a target completion date, the feedback that you're making progress to those values. And of course some confidence that those targets are credible, their degree of variance, and how they are changing as the project progresses.
So one final comment
When we hear that we don't need to estimate, there is no way to have a closed loop control system. Without on estimate of the cost, schedule, and technical performance goals, the only management control system is open loop. This is unlikley to lead to success for any non-trivial project.
There are three key elements of every project on the planet - Cost, Schedule, and the performance of the product or service produced by the project. Each of these has drivers. The connections between Cost, Schedule, and Technical Performance are not Iron as suggested in the Iron Triangle of a PMI view of the project. Instead the connections are elastic, springy, flexible. But they are connected.
Cost is driven by:
These costs are themselves variable, as a function of the project phase, external forces for labor, materials, and overhead. But the cost variable starts with these.
Schedule is driven by:
Technical Performance is driven by:
So What Does All This Mean?
But for project success we need to have several things in place. The random behavior has to be knowable. It can't be chaos. If it is chaos, the project will fail, because there is no corrective action.
The three elements need to be known to some degree of confidence.
Do know these to some degree of confidence, you don't know what DONE looks. The only measure of progress becomes the passage of time and consumption of money. It's unlikely any customer is going to be willing to pay you - at least for very long - to spend their money, without some understand of Cost, Schedule, and the resulting Technical Outcomes.
Let's Start With End In Mind
If you work on projects and get paid for that work, someone is paying for you. That can be a direct customer if you're working on a external project. Or it can be the firms customers, buying a product or service which provides the money to pay you. No matter, someone pays for you to do what you do best.
Project Accounting is a subset of Finance Accounting, Both Are Important to the Project Manager
As a person working on a project, you may not care about Project Accounting or Finanical Accounting. But it's important to know where your pay check comes from, and it's not the Bank of America.
Project Accounting, sometimes called job cost accounting, creates data that tracks the financial performance of projects. Project Accounting enables the firm providing project resources (labor and material) to monitor the progress of their projects from a financial point of view. This is separate from standard organizational accounting for departments, divisions or the firm.
There are several data values used in project accounting. The project manager and the business manager(s) funding the project expect a certain level of profitability to be maintained in each function of the organization workiing on the project. The cost associated with delivering the project is Budgeted by the firm - usually before the customer of the project receives an invoice for the work and sends Funds in the form of real money. Budget is not dead presidents. Budget is an authorization to spend dead presidents, but you can't take your budget to Star Bucks and buy a Vente Latte.
For external projects the separation of internal costs - both direct and indirect - is many times done by finance. Usually the project costs are wrapped or grossed up for billing purposes to the customer. For internal projects, the fringe, labor overhead, material overhead, and G&A costs are not usually recorded directly to the project, but recorded in the Project Ledger as the wrap rate for your labor. Fringe, labor overhead, material overhead, G&A and other indirects are paid with actual funds - dead presidents - so those costs can be recorded outside the project, since they are not material to the project's performance if managed properly. We see all the time, when those indirect costs get out of hand the firm itself reduces its profit and they come looking for savings from your project.
An important concept of project cost accounting is the ability to provide visibility to targeted revenue against the actual revenue, and the estimated costs to produce that revenue against actual cost to produce that revenue. Many firms separate these two items - revenue and costs. The Project Manager focuses on the cost side - direct labor, materials, services, etc. And the Financial Accounting department focuses the revenue forecast and actuals. Since those direct labor, materials, and services - whether internal or external - have to be paid in real money, someone in Accounts Payable needs to know how much will be coming due in the next period.
This By The Way is the role of Estimating. How much labor will be needed to produce the planned outcomes of the project next quarter, next release, or the next anything? Fixed labor pool? Simply add up the FTE (Full Time Equivalents), multiply by the Fully Burdened labor rate and send that to accounting. But of course that means with a fixed labor pool, you'll need to know what your capacity for work is. Have a fixed commitment to deliver some value? Then you'll need to know how many FTE's that takes. Either way you'll need an estimate for those outside your project paying the bills.
What Does Project Accounting Do Best?
Project cost accounting records the costs associated with a particular project or job. Project accounting collects all cost - invoices, time cards, other direct project charges - and assures they are recorded in the proper charge account. These charges can be direct billed to the customer, or assigned to a Budget Item in the project cost baseline, or recorded as non-billable. When costs are non-billable, they are also considered non-recoverable and are usually captured in some overhead account. It's the cost of business.
It's a simple matter of balancing the books. Money In minus Money Out = Retained Earnings (money left over). It's of course more complex than this, but this will do for now.
This must balance...
All Income in the form of Accounts Receivable = All Outgo in the form of payments in "real money"
For project accounting to add value, care is needed to record and track every project cost. This by the way is one primary role of the Work Breakdown Structure. The WBS defines the budget and captures the cost of each project element. On the project side expenses are represented by direct labor (this is why time cards are used on many external contracts), invoices submitted properly for non-labor project costs.
One important issue in project accounting is who carries the overhead?
Project managers are usually not aware - at least directly - of the overhead, fringe, and other indirect costs to their projects. In some domains these costs are wrapped in a multiplier to the direct labor and recorded on the Project Balance sheet without any detail. This Wrap Rate value is important to the project manager, since this cost is subtracted from the revenue generated from the external customer. For internal projects, this wrapped cost is part of the ROI calculation for the cost of delivering the value from the project to the internal business customer.
In the end someone has to pay the electric bill, those free lunches for the project team, those Star Buck cards we hand out for hard work, and the spot award checks for other actions of the team. Those really nice 1080P monitors sitting on the engineers desks? Someone has to pay for those. The Aeron chairs? Someone has to pay. In the end someone has to pay.
Our daughter (now grown an gone) came home one day from her High School Economics class and announced, Dad we learned today there is no such thing as free. Yep, dear, welcome to the real world. People need to know what the real cost is. When they say free checking, someone is paying for that, probably you.
On projects, all that infrastructure you enjoy - laptops, lunchroom, parking, etc. is paid for by someone. If not directly, than indirectly. That's accounted for in the Financial Accounting System, the one the CFO is interested in. As a project manager you may or may not know about the wrap rate or even care about the wrap rate. But those who pay you do. And since they do, you may want to have some concern about the wrap rate as well. Fringe benefits, labor overhead, material overhead, G&A, etc. are real costs and impact that ellusive bottom line for your project and your firm.
So In the End
If you work on projects and are not concerned with the wrap rate either directly or indirectly (a pun for all us project planning and controls geeks), then you're probably direct labor. Wonderful direct labor, hired for your irreplaceable skills, but accounted for as labor all the same. Recorded on the books by your direct rate (annual or hourly), the fringe, overhead and other indirects.
So if you don't care for the Overhead discussion, or don't want to perform your role as a project manager using Overhead, just remember, those who pay you do care. And since they care, you may want to care, or at least pretend you care, you're job may depend on it.
The separation of Project Accounting from Financial Accounting is common in many firms. Finanical Acounting is the domain of the CFO or equivalent and the finance and accounting department, where payments of invoices to customers and from suppliers ae processed.
Project Accounting, sometimes called Job Cost Accounting, is usually the domain of the Project Manager. Where budgets are assigned and records of work peformed and cost of that work reported. But actual money doesn't change hands.
In the Financial Accounting domain, all costs and all income are recorded as transactions on the General Ledger. These transctions are rolled out through accounts to produce a Balance Sheet of the cost versus value in some form. Sometimes that value is revenue. Sometimes, it's business value recorded as intangible asset. Managing of the intangible assets of a firm is the primary role of business leaders. Teh tangible assets are usualy managed in the accounting, by the depaertment with the same name.
I'm making this simple, and likely simple minded for the point that is coming. Who owns the overhead costs that are present in every single business? For larger complex projects here's one framework for doing that, but we'll keep it simple for now.
But before going simple here's a quick looks at FASB 86, the guidance for accouting for the development and use of software. But one quick notion that is missed by many in the agile world not working the arcane processes of public accounting ...
The Agile Alliance is working this issue.
So Back To Project Accounting Processes
Projects spend budget. Businesses spend Funds. If you work on a project, rarely does someone on the business side come to you with an envelope full of money - dead presidents - and say go hire some developers to write code for this project. Those authorizing you to hire people, or buy materials, do so with a Purchase Order of some sort (we have PCARDS that skip all the paper work), or some authorization from HR to hire. The employee or contractor then charges to a charge number or Employee Number to capture their expenses (labor or material).
That expense is then wrapped with fringe, overhead, and other indirects and recorded on the books as the Fully Burdened cost to the project.
Someone Has to Pay
No matter where you work in the organization, the all in cost for the project has to be paid. This includes your direct labor. What you get in your pay check, those fringe benefits - health insurance, matching 401(k), AMC movie passes, spot awards. Your firm has to pay. If you're a 1099 supplier, then it's you that has to pay. As a 1099, if you're not billing your customer for all that overhead than those costs are reducing your revenue. Same for a firm.
So if you're not concerned about overhead, don't manage the project knowing the overhead. Not a problem in principle. In practice though someone does, and if they care, you may want to care as well. If not, then you're likley working as labor on the project. That's in NO way a problem. We're all labor in some form to some one. But when it comes to managing projects and managing the business that projects use or produce, these indirect costs can make or break the project and or the business. Knowing about them, managing in their presence is simply the responsible thing to do. You know likey knowing what your project will cost when it's done.
Here's some background that have served me well over time as a Program Planning and Control manager.
Todd Little and Steve McConnell use a charting method that collects data from projects and then plots it in the following way. For Little's data its the initial estimated duration versus the actual duration.
and for McConnell's data it's the estimated completon date versus the actual completion date.
So Where's the Rub?
These charts show that project estimates exceed some ideal estimate on a number of projects - the sampled projects. If we were sitting in the statistics class in an engineering, physics, chemistry, biology course, here's some questions that need answers.
What's missing are several things.
The Core Issue with Using Past Numbers
What To Do Next?
The first thing to do is go out to the book store and get a book on statistical forecasting or statistical estimating that has actual math in the book. Next is to ask some hard questions?
Then read all you can find on reference class forecasting and statistical inference. Data is not information. Cause is not correlation.
There's really no way out of this. Spending other peoples money, at least money they are no willing lose, means having some process of estimating the probability of success.
The Final Thought
Plot cost and schedule for your projects asa Joint Probability. Below is a Monte Carlo Simulation of the Joint Cost and Schedule for a program. A similar chart is needed but using a collection of projects. Take Little's and McConnell's sample projects and plot both cost and schedule. There may be correlations between original cost and actual cost, versus original schedule and actual schedule. Big projects have higher risk - restating the obvious by the way. Higher risk project may have wider variances in performance - also restating the obvious.
But these one dimension - one independent variable - plots of cost overrun versus original cost estimates just show the uncalibrated, un-normalized, non-root-cause data. It's just a chart. Of little value for taking corrective action.
It's that easy and it's that hard. If you don't have a handle on what risks are going to impact your project, those risks will still be there, you just won't know it.
The first step in increasing the probability of project success is to have some notion of what is going to prevent that success. This means asking what can go wrong, rather than what can go right. In order to answer the question what can go wrong we need to know what we are doing. What is the project about? What are we trying to produce? When do we need to produce it? How much money will we need to spend to produce this things call DONE?
Let's start with some obvious risks that we have to handle for any hope of success of the project. These are obvious because they occur on every project, in every domain, using any project management method.
"Planning constantly peers into the future for indications as to where a solution may emerge. A Plan is a complex situation, adapting to an emerging solution."
- Mike Dwyer, Big Visible Charts
When we hear about unchangeable requirements, fixed budgets, fixed anything on a project. We should think about it again and say to ourselves. The person saying those things doesn't actually have a clue about how actual project success is delivered.
Proper Preparation Prevents Poor Performance
This simple phrase describes the core behavior of all project related work. For any platitude to survive contact with reality, it must be true, actionable, and applicable in your own domain.
The notion that “emergence” is the driver for the participants in a project requires careful consideration. The technical or business requirements of the outcomes of the project are always “emergent” in some form. To be otherwise would require a preset group of activities, materials, technology, and personnel.
Failing to understand the subtleties of the continuous emergence of requirements, that enable the capabilities to be delivered, means failing to understand any project requires planning to deal with these emerging requirements. This emergence also pertains to the tools and processes used to manage and deliver the project. emergence is applicable to all elements.
Preparing for Emergence is a critical success factor in project work. Proper preparation is the foundation of programmatic and technical risk management. This means asking and answering the “if – than” question, rather than the “what – if” question.
Managing in the presence of emergence requires directed decision making. Just letting things happen disconnects the outcomes of the project from the neeed capabilities produced by the project. These capabilities are the immutable part of the business process. They can only change, when the strategy for the success of the business enabled by the project changes. To do otherwise, would be to disconnect between the investment in the project and the value produced by the project.
“If – Than” means knowing what can go wrong and how to respond to the following:
We hear all the time, pithy statements about teams, teaming, team building, enabling team that provide innovation, and most every soft skill ever thought of, applied to managing projects. At the project below, where I was one of many program managers, we came to realize one fundamental truth about teams -
If you have a team who is your opponent?
Having played team sports at the High School, College, and competitive adult level, our teams always played against an opposing team. Much of the team building platitudes seem to ignore that teams are formed to play other teams, score points, win games, overcome obstacles, and participate in competition.
Who is the other team if we're on a project team?
See if the notions below resonant when you hear short, cleaver words about the role of teams on projects?
It's not clear how this notion came about, since I haven't talked directly with those making statement like that. I'll have to estimate myself why this might be true.
One reason might be they haven't been exposed to the statistical processes used to make estimates in a variety of domains, not just project management. When we encounter the need for estimates, we need to come in contact with probability and statistics. All processes on projects are statistical in nature. This produces uncertainty and uncertainty results in probabilistic outcomes.
Estimating means searching in this probabilistic "space" for a number that is close enough to answer a question. How many people are in line for movie tickets inside? Should I buy my tickets at the kiosk outside? Or maybe, how long will it take me to drive from my office to the airport parking garage for my flight on Wednesday morning? These are not exact numbers, these ae quick estimates. The answer to the first question can be made with a quick look inside the theater. The second comes from the experience of driving to the airport several times a month over the past 20 years. One is informed by observeation of the current situation, one informed by past experience under a variety of situations.
Let's start with a simple definition of an estimate - it is an approximation of some value for some purpose, even if the data is incomplete, uncertain, or even unstable. Typically the estimate is derived from a statistical source of data - either observed or referenced in some way.
Making Simple Estimates
When we don't have the exact answer of some value, we can make an estimate of that value. The result is a number and a confidence on that value. Say we need an estimate of how long it will take to drive to the airport. It's 52 miles. Some on surface streets, some on open 2 lane roads, some on freeways. Knowing the speed limit for each of these segments can get us an approximate time portal-to-portal. We can improve the estimate, knowing the weather conditions and the traffic flow on the surface streets.
Let's look at five easy steps for making estimates (abstracted from The "Mother of All Guesses" - A User-Friendly Guide to Statistical Estimation, by Francois Melese and David Rose COPYRIGHT ARMED FORCES COMPTROLLER, 1998)
Making Estimates for Project Work
With the steps above we can start to make estimates of our project work.
But first let's kill some myths used to not to make estimates.
We've heard them all and more. There is an endless list of reasons for not doing most anything on a project, but that doesn't remove our obligation to be stewards of other people's money we are spending with expectation (estimate) of some value in return.
Let's start with a core principle of all project work, and possibly a principle of all life's work when we encounter money.
We can't determine what something is worth until we know what it costs.
This is a crass capitalist point of view I know. But we live in a crass capitalist society and them's the rules. Don't like the rules, it might be better to look somewhere else for work other than spending other peoples money without knowing how much it will cost in the end and when we'll be done.
We all know and maybe even experience a Dilbert's paradigm when it comes to projects. But that doesn't make it right. We hear this some times from management. But we also hear it as an excuse for not making estimates from those who should be looking after the cost needed to compute the Return on Investment.
A Few More Myths
Below are some links to other Blogs on this topic, explore, think about our role in managing other peoples money.
When we hear about software development methods, it's critical to understand in what domain they are applicable. Here's an analogy for that question. Without an answer to this question there is no basis for comparing units of measure. What might be a wonderful approach in one domain would be unusable in another - this is the case for all the examples.
A solo or 3 team project with a customer in the same building would never apply DOD 5000.02, FAR 34.2 Earned Value flow downs. Building the flight avionics to fly to ISS would never use #NoEstimates and emergent requirements on a weekly basis for a 200 person development team spread across the United States.
So before any conversation - beyond shared ancedotes and asking audiences to think about it, because I can't tell you what to do - can take palce, we need to establish the domain, the value at risk, and the governance model applied to our work. Then we can talk about shared ideas.
Matt Heusser's article brings up some interesting points. Let's look to see if there are any limitations from a domain or context point of view. By domain I mean, in what taxonomy are you writing software for money. By context I mean what are the constraints or governance guidelines in that domain.
1. Make the amount of money small
This is a version of time boxing. It limits the value at risk for the development process. This bounds the risk process. In exchange for the total loss of doing the wrong thing information can be found. This is also called tuition cost.
Issue: We may not have all we need to forecast the total cost and schedule. Projects in many domains aren't made up of small chunks of themselves. So we'll need to confirm that the sum of the parts results in the whole. Integration, test, verification, architecture, interface management, and many other Systems Engineering aspects need to be involved in some way.
2. Fund a pilot that delivers working software and use this model to forecast schedule
This is buying a reference class. With the reference class - all be it a class of 1 - a forecast of future cost, schedule, and technical performance. We need all three in the reference class.
Issue: this is a larger issue of Number 1. With the pilot can we be assured that work can be scaled? Verification of that will have to be part of the pilot or a follow on. Then comes the confidence intervals for how that scaling will interact with the other - yet to be developed - components of the system. Is the scaling linear, nonlinear, stochastic interactions, and a whole raft of other discoverable processes. Each needs to be planned and budgeted.The ACA web site is a recent example. A UAV I worked where the engine didn't have enough thrust after the final integration. Etc. etc.
3. Move from contract negoiation to partnership
You've simply transferred to responsibility to estimate the cost and schedule to someone else. A single example - in the article - is not the basis of a syndicatable process. So this example, while interesting, probably isn't going to go too far with someout means to address the Estimate at Complete needed in most non-trivial software development projects.
Issue: Still don't know how much the project will cost in the end.
4. Employ Start Stop Heuristics
Seems like just another version of time boxed budget and schedule.
Issue: still doesn't address the Estimate at Complete before actually reaching the end or near the end of the project.
5. Drop Estimation From Your Estimation Process All Together
Another version of time boxed budget. Someone has to know how much money is needed to run the business on an annual basis. Or how much money will be allocated to do some work on an annual basis. This is called Level of Effort. Work until the money runs out, we give you more money, or tell you to stop. PayPal works in this way - sorta. The prioritization of the work is the responsibility asking for the outcomes. They have a budget. Give that budget (not funds, budget) to the development organization in exchange for delivered code.
Issue: The project will cost a know amount. We don't know what we'll get for that amount. But that might be OK. Once work is been going for awhile, a Reference Class can be built to allow that question to be answered, assuming the requested software fits inside the reference class in some way.
So In The End
The 5 suggestions don't have a domain beyond the single examples. And the suggestions don't seem to have a way to forecast the bounds of the project with an Estimate at Completion beyond the use of the reference class of the project itself. This self referencing reference class seems a bit sporty.
So yes, there are some ways to develop software in the absence of formal estimating. Although 2 of the 5 are actually using reference classes to forecast.
Those paying for the work to be done, still have to come up with some upper bound on cost, schedule, and technical capabilities for that cost and schedule. These 5 suggestions are a start. But we don't yet know where they can be applied. If they have been applied outside of specific anecdotes, and if they are scalable beyond the personal anecdotes.
No problem. This notion of not estimating is still evolving. At some point, answers need to be forth coming and the Yoda-style conversation replaced by business conversation - what did you do with my money?
In a meeting today on the difficulties of increasing the probability of project success in our domain (DOD/NASA). There were three sources of uncertainty around cost, schedule, and technical forecasting
The program runs into things that cause cost and schdeule overruns:
We really shouldn't be doing #1 and #3. #1 means we didn't look hard enough. #3 means we can't handle the truth. #2 is the definition of uncertainty. But is it uncertainty because the project, didn't know or didn't want to know.
Doing stupid things in purpose
When managing projects that are funded by other people's money, we are obligated to know something about the probabilistic outcomes of our work. Without the ability to make forecasts and estimates of these outcomes , we are relegated to the category of labor.
As Project Managers, we must learn to estimate and forecast to some level of confidence. This is not guessing, that is child's play. Choosing not to estimate is also child's play. We must learn to make estimates based in sound statistical and probabilistic principles. Without that, the very notion of Return on Investment is intentionally ignored.
Return on Investment = (Value produced - Cost to Produce) / (Cost to Produce)
It's that simple and it's that hard. The cost and value variables are probabilistic estimates and must be treated in that way. But without knowing either of these, to some degree of confidence, we cannot make informed management decisions.
That is the primary role of project management or engineering development -
Make Decisions based on evidence in the presence of uncertainty