Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
Software development ranges from straight forward, with small a team with a continuous improvement effort. Say a web site or warehouse management application were the customer is happy to just keep the improvements coming. Every improvement or new feature can be put to use. Along the way new ideas enter into the mix and since there is no real deadline, the features can be deployed when they are ready.
On the other end of this wide spectrum of projects is the mission critical, business critical project. For example a Mergers and Acquisition strategy of two large firms that need to join their ERP systems before any benefits from the merger for the customers, operations, and cash flow can be realized. Below is an example of the Plan for such a project.
The first thing to put to rest is the naive and misinformed notion that planning is somehow not needed. The Plan above is a strategy for the integration of legacy systems with a new ERP system. The Planned date for this system is tied to business success. Long before this Plan was developed, the business strategy identified the needed capabilities of the integrated system, the planned savings from the integrated system, and a customer facing set of abilities of the result.
This is the fundamental Return on Investment equation. ROI = (Value - Cost)/Cost. We know the value, since the analysis of the value of the integrated system was done before the merger. The cost was estimated at the highest level from experience of integrating ERP in the past. Details come next, usually after the merger.
Each of the boxes provides a needed capability. End box is an incremental deployment of this capability, with feedback that informs the downstream capabilities. No project can have an end-to-end detailed schedule with any hope of that schedule remaining intact. But the Plan above describes the order of delivery of the needed capabilities. These are not arbitrary, these are not random, these are not subject to change. At least not without understanding the impact on the business merger process and the resulting cost savings and cash flow generation.
So What's The Point
If the value at risk is sufficient to call into question business failure, or at least major issues, if the project fails to meet its goals, then we need a Plan, an estimated cost, and an estimated delivery date. To ignore these business needs is to ignore the obligation to provide that needed information.
You project may not be a merger of two multi-billion dollar firms. But the question isn't when to estimate but when do we no need to estimate. The asnswer to that starts with the Value at Risk. The business providing the funding gets to say what the value is.
What is the business willing to Risk on the project without knowing that value before starting?
Software intensive project work is indeterminate. Many times, customers don't know what they really want until they see something working. Development's capacity for work is not well understood early in the project. Technology emerges and impacts the effectiveness of the project. Reducible and irreducible uncertainties abound. These conditions are normal, they can't be wished away.
So what are the options? Let's start with some advice and see what we can do to improve the Probability of Project Success. Wise leaders from the past have words we can use to guide our efforts.
I know one thing: that I know nothing
The trouble with the world is that the stupid are cocksure and the intelligent are full of doubt
— Bertrand Russell
The fool doth think he is wise, but the wise man knows himself to be a fool.
— William Shakespeare, As You Like It
Knowing nothing about the project is not good for increasing the probability of success. Knowing what capabilities are needed to fulfill a mission or business need is a starting point. Capabilities are not requirements, those come next. Capabilities allow the user to do something of value. This is the value spoken about, most times spoken about in the absence of any units of measure. These measures, the measure of capability is the Measure of Effectiveness. (The tool in the previous link is a systems engineering tool, that provides all project participants with clear, concise, and testable assessments of their understanding of what Done looks like. The notion that tools get in the way of success is ill-informed at best, and just plain wrong at worst).
Measures of Effectiveness are operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in an operational environment, under a specific set of conditions.
If we don't know what capabilities we want the project to deliver, it's going to be a low probability that the project is a success. We don't know what done looks like. Now we can get paid to discover these capabilities. Capabilities Based Planning is a process to discover these. Someone has to pay for this of course. It is usually the customer that pays. But someone has to pay.
With the needed capabilities in hand, we can make our first estimate for cost, schedule, and the probability of success. Here's an example. Frank Cepollina was speaking at the BAA (Broad Area Announcement) for the Hubble Robotic Service mission.
He stated his needed capabilities rather than in techncial requirements. Those always come after we know what done looks like. Notice these capabilities are mission capabilities. Get the job done descriptions.
How much does that cost and when can you have it ready to fly?
The three vendors needed to develop an estimate. The details of this process can be found in Assessment of the options for extending the life of the Hubble Space Telescope. This example can be extended to IT and software development projects. What do we want the system to do? Don't know? Go find out. Make that the first step in the project. Read Predictabilty: A Simple Approach to Creating Reliable Project Schedules, Steve Bockman for an example of how to think about this in a software development environment. No matter what our domain, size of project, we're going to need to understand something about what capabilities are needed for project success. We might be able just to start work and let these emerge. But our probability of success is unknown and that's not likely a good thing in the end.
So when we know something about the needed capabilities, we can start to discover the requirements. But first we're going to need some kind of measure for those requirements. These are performance measures.
These are measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. These are Measures of Performance, and are:
These are measure of how the requirements are going to perform in pursuit of the mission.
Final we have to discover the Technical Performance Measures. These are attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal. These:
Here another need for estimating. How fast do we have to process transactions to meet the needed capability? How much can we weigh and still have the capability to fly? How reliable (mean time between failure, mean time to first failure, mean time to recover from failure), and still meet the needed capabilities?
These estimates are focused on technical outcomes. Thing that fly away, things that provide services in support of the customer. Things.
Where are we going here?
We must learn to estimate all kinds of things on a successful project. The credibility of our skills as project managers is always tied up with our ability to estimate. Can't estimate, choose not to estimate, see estimating as some kind of dysfunction, it is doubtful those providing us with money are going to have much confidence in our ability to deliver that mystical value we are told is part of all projects.
Estimating is part of project work. To not estimate, means we won't know what done looks like until we arrive at the end. That's unlikley to be acceptable to those paying us to do work.
The next step is to determine what requirements will be needed to implement the needed capabilities. These are defined as Measures of Performance that characterize physical or functional attributes relating to the system of operation, measured or estimated under specific conditions.
Notice something important here. Software project success always depends on development of working software. But what software, in what order, with what Technical Performance Measures, Measures of Performance, and Measures of Effectiveness all supporting the needed Capabilities.
Success doesn't start with coding. It's about knowing the answers to the CBP, MOE, MOP, TPM attributes. The notion of emergent anything works only when we can state in some unit of measure what DONE looks like. Without that we're just wasting the customer money exploring, challenging everything, and looking for the mystical Unicorn hiding in the bushes.
So Here's the Punch LIne
When we spend other people's money on our project, private money, public money, government money, they have the expectation of knowing something about how much much and when we will be finsihed with the work in exchange for that money. This is pretty much the case for every project I've ever worked for the past 34 years. It's the basis of every successful business - knowing what your costs are for the operations. So when we hear about developing software for money and not having any concern about the cost of that effort it simply doesn't make any sense.
The common picture of requirements elicitation looks like this. Which of course is an example of doing stupid things on purpose. When this picture is used as an example of not doing something because it doesn't turn out right, is a further example of doing stupid things.
Let's see where the gaps appear that results in the outcome in the last panel:
The wheels fall off when there is no description of DONE shared between the customer and the provider. If the customer doesn't know what DONE looks like, who does? The developers? Probably not. Is DONE emerging? Then the customer has to pay for that?
There is no way out of the need to know what DONE looks like in Measures of Effectiveness, Measures of Performance, Key Performance Parameters, and Technical Performance Measures. Without these the success of the project is in doubt from the start.
The notion that requirements emerge is will established. But the capabilities the customer needs to be stable enough to establish an Estimate At Complete and an Estimated Duration to Complete. Without these the customer has no understanding of the all in cost of the project. These change as the requirements change. That's part of the project management process. This can't be ignored if the customer is to have any confidence in the project providing value at the needed time at needed to be put to use.
So a reminder one more time...
You can't assess the value of the project outcome without knowing something about the cost to deliver that value and the time frame for that delivery.
No matter what anyone says, this is an immutable business principle. Anything is simply fooling yourself into believing that the customer doesn't care how you spend her money or when you spend it.
Focus on value, that's what the customer bought. Actually they bought a capability to do something to improve their business or mission. But ignoring the cost of delivering that value is ignoring the balance sheet of the business.
There are Principles, Practices, and Processes of project success. I have a book coming that describes them. These are independent of any method, any domain, any favorite tool, buzz words, or personal paradigm. Much research on Root Causes of project failure have led to these Principles, Practices, and Processes. The Principles don't change, can't change. The Practices need to be tailored to the needs of the project and its paradigm. The Processes are localized.
But around these are 7 Activities of any project, that if not performed in some way, the probability of success is reduced, sometimes to Zero.
Domain and Context
No matter if your building bridges across rivers, a flying machine to Mars, the next anti-virus drug, or writing software using Scrum for your internal web site, these activities must take place in some way.
Without the last 4 activities, you're managing the project open loop. This might be OK if your customer doesn't care how much it will cost or how long it will take to finish the defined work. But of they do, you'll need a target cost and a target completion date, the feedback that you're making progress to those values. And of course some confidence that those targets are credible, their degree of variance, and how they are changing as the project progresses.
So one final comment
When we hear that we don't need to estimate, there is no way to have a closed loop control system. Without on estimate of the cost, schedule, and technical performance goals, the only management control system is open loop. This is unlikley to lead to success for any non-trivial project.
There are three key elements of every project on the planet - Cost, Schedule, and the performance of the product or service produced by the project. Each of these has drivers. The connections between Cost, Schedule, and Technical Performance are not Iron as suggested in the Iron Triangle of a PMI view of the project. Instead the connections are elastic, springy, flexible. But they are connected.
Cost is driven by:
These costs are themselves variable, as a function of the project phase, external forces for labor, materials, and overhead. But the cost variable starts with these.
Schedule is driven by:
Technical Performance is driven by:
So What Does All This Mean?
But for project success we need to have several things in place. The random behavior has to be knowable. It can't be chaos. If it is chaos, the project will fail, because there is no corrective action.
The three elements need to be known to some degree of confidence.
Do know these to some degree of confidence, you don't know what DONE looks. The only measure of progress becomes the passage of time and consumption of money. It's unlikely any customer is going to be willing to pay you - at least for very long - to spend their money, without some understand of Cost, Schedule, and the resulting Technical Outcomes.
Let's Start With End In Mind
If you work on projects and get paid for that work, someone is paying for you. That can be a direct customer if you're working on a external project. Or it can be the firms customers, buying a product or service which provides the money to pay you. No matter, someone pays for you to do what you do best.
Project Accounting is a subset of Finance Accounting, Both Are Important to the Project Manager
As a person working on a project, you may not care about Project Accounting or Finanical Accounting. But it's important to know where your pay check comes from, and it's not the Bank of America.
Project Accounting, sometimes called job cost accounting, creates data that tracks the financial performance of projects. Project Accounting enables the firm providing project resources (labor and material) to monitor the progress of their projects from a financial point of view. This is separate from standard organizational accounting for departments, divisions or the firm.
There are several data values used in project accounting. The project manager and the business manager(s) funding the project expect a certain level of profitability to be maintained in each function of the organization workiing on the project. The cost associated with delivering the project is Budgeted by the firm - usually before the customer of the project receives an invoice for the work and sends Funds in the form of real money. Budget is not dead presidents. Budget is an authorization to spend dead presidents, but you can't take your budget to Star Bucks and buy a Vente Latte.
For external projects the separation of internal costs - both direct and indirect - is many times done by finance. Usually the project costs are wrapped or grossed up for billing purposes to the customer. For internal projects, the fringe, labor overhead, material overhead, and G&A costs are not usually recorded directly to the project, but recorded in the Project Ledger as the wrap rate for your labor. Fringe, labor overhead, material overhead, G&A and other indirects are paid with actual funds - dead presidents - so those costs can be recorded outside the project, since they are not material to the project's performance if managed properly. We see all the time, when those indirect costs get out of hand the firm itself reduces its profit and they come looking for savings from your project.
An important concept of project cost accounting is the ability to provide visibility to targeted revenue against the actual revenue, and the estimated costs to produce that revenue against actual cost to produce that revenue. Many firms separate these two items - revenue and costs. The Project Manager focuses on the cost side - direct labor, materials, services, etc. And the Financial Accounting department focuses the revenue forecast and actuals. Since those direct labor, materials, and services - whether internal or external - have to be paid in real money, someone in Accounts Payable needs to know how much will be coming due in the next period.
This By The Way is the role of Estimating. How much labor will be needed to produce the planned outcomes of the project next quarter, next release, or the next anything? Fixed labor pool? Simply add up the FTE (Full Time Equivalents), multiply by the Fully Burdened labor rate and send that to accounting. But of course that means with a fixed labor pool, you'll need to know what your capacity for work is. Have a fixed commitment to deliver some value? Then you'll need to know how many FTE's that takes. Either way you'll need an estimate for those outside your project paying the bills.
What Does Project Accounting Do Best?
Project cost accounting records the costs associated with a particular project or job. Project accounting collects all cost - invoices, time cards, other direct project charges - and assures they are recorded in the proper charge account. These charges can be direct billed to the customer, or assigned to a Budget Item in the project cost baseline, or recorded as non-billable. When costs are non-billable, they are also considered non-recoverable and are usually captured in some overhead account. It's the cost of business.
It's a simple matter of balancing the books. Money In minus Money Out = Retained Earnings (money left over). It's of course more complex than this, but this will do for now.
This must balance...
All Income in the form of Accounts Receivable = All Outgo in the form of payments in "real money"
For project accounting to add value, care is needed to record and track every project cost. This by the way is one primary role of the Work Breakdown Structure. The WBS defines the budget and captures the cost of each project element. On the project side expenses are represented by direct labor (this is why time cards are used on many external contracts), invoices submitted properly for non-labor project costs.
One important issue in project accounting is who carries the overhead?
Project managers are usually not aware - at least directly - of the overhead, fringe, and other indirect costs to their projects. In some domains these costs are wrapped in a multiplier to the direct labor and recorded on the Project Balance sheet without any detail. This Wrap Rate value is important to the project manager, since this cost is subtracted from the revenue generated from the external customer. For internal projects, this wrapped cost is part of the ROI calculation for the cost of delivering the value from the project to the internal business customer.
In the end someone has to pay the electric bill, those free lunches for the project team, those Star Buck cards we hand out for hard work, and the spot award checks for other actions of the team. Those really nice 1080P monitors sitting on the engineers desks? Someone has to pay for those. The Aeron chairs? Someone has to pay. In the end someone has to pay.
Our daughter (now grown an gone) came home one day from her High School Economics class and announced, Dad we learned today there is no such thing as free. Yep, dear, welcome to the real world. People need to know what the real cost is. When they say free checking, someone is paying for that, probably you.
On projects, all that infrastructure you enjoy - laptops, lunchroom, parking, etc. is paid for by someone. If not directly, than indirectly. That's accounted for in the Financial Accounting System, the one the CFO is interested in. As a project manager you may or may not know about the wrap rate or even care about the wrap rate. But those who pay you do. And since they do, you may want to have some concern about the wrap rate as well. Fringe benefits, labor overhead, material overhead, G&A, etc. are real costs and impact that ellusive bottom line for your project and your firm.
So In the End
If you work on projects and are not concerned with the wrap rate either directly or indirectly (a pun for all us project planning and controls geeks), then you're probably direct labor. Wonderful direct labor, hired for your irreplaceable skills, but accounted for as labor all the same. Recorded on the books by your direct rate (annual or hourly), the fringe, overhead and other indirects.
So if you don't care for the Overhead discussion, or don't want to perform your role as a project manager using Overhead, just remember, those who pay you do care. And since they care, you may want to care, or at least pretend you care, you're job may depend on it.
The separation of Project Accounting from Financial Accounting is common in many firms. Finanical Acounting is the domain of the CFO or equivalent and the finance and accounting department, where payments of invoices to customers and from suppliers ae processed.
Project Accounting, sometimes called Job Cost Accounting, is usually the domain of the Project Manager. Where budgets are assigned and records of work peformed and cost of that work reported. But actual money doesn't change hands.
In the Financial Accounting domain, all costs and all income are recorded as transactions on the General Ledger. These transctions are rolled out through accounts to produce a Balance Sheet of the cost versus value in some form. Sometimes that value is revenue. Sometimes, it's business value recorded as intangible asset. Managing of the intangible assets of a firm is the primary role of business leaders. Teh tangible assets are usualy managed in the accounting, by the depaertment with the same name.
I'm making this simple, and likely simple minded for the point that is coming. Who owns the overhead costs that are present in every single business? For larger complex projects here's one framework for doing that, but we'll keep it simple for now.
But before going simple here's a quick looks at FASB 86, the guidance for accouting for the development and use of software. But one quick notion that is missed by many in the agile world not working the arcane processes of public accounting ...
The Agile Alliance is working this issue.
So Back To Project Accounting Processes
Projects spend budget. Businesses spend Funds. If you work on a project, rarely does someone on the business side come to you with an envelope full of money - dead presidents - and say go hire some developers to write code for this project. Those authorizing you to hire people, or buy materials, do so with a Purchase Order of some sort (we have PCARDS that skip all the paper work), or some authorization from HR to hire. The employee or contractor then charges to a charge number or Employee Number to capture their expenses (labor or material).
That expense is then wrapped with fringe, overhead, and other indirects and recorded on the books as the Fully Burdened cost to the project.
Someone Has to Pay
No matter where you work in the organization, the all in cost for the project has to be paid. This includes your direct labor. What you get in your pay check, those fringe benefits - health insurance, matching 401(k), AMC movie passes, spot awards. Your firm has to pay. If you're a 1099 supplier, then it's you that has to pay. As a 1099, if you're not billing your customer for all that overhead than those costs are reducing your revenue. Same for a firm.
So if you're not concerned about overhead, don't manage the project knowing the overhead. Not a problem in principle. In practice though someone does, and if they care, you may want to care as well. If not, then you're likley working as labor on the project. That's in NO way a problem. We're all labor in some form to some one. But when it comes to managing projects and managing the business that projects use or produce, these indirect costs can make or break the project and or the business. Knowing about them, managing in their presence is simply the responsible thing to do. You know likey knowing what your project will cost when it's done.
Here's some background that have served me well over time as a Program Planning and Control manager.
It is popular to make a list of maxims for developing products, managing projects, or managing business processes. Some are based on experience, some based on surveys, some based on principles and practices of a profession.
Here's mine based on counter examples of the sole contributor paradigm. The sole (or small group) contributor paradigm means maxims gathered from personal experience from a person's engagement on the job. One example for the sole contributor, used without permission and with full attribution is Woody Zuill's list. There are others Five Project Maxims, 18 Maxims of Successful IT Consulting and other. But I like Woody's framework best, because his topics fit best with our processes on complex, mission critical, software intensive programs and the hands on integration with process. Although Woody would likely not agree, both technical skills and formal process frameworks are critical success factors in any sufficiently complex domain - both are needed.
Doing the work is guided by the Strategy and Performance Goals of the needed Capabilities.
Without a clear and concise understanding of what DONE looks like in Measures of Effectiveness for the needed capabilities, all the project work has no home. It's just a list of features or functions captured by the development team from the customer or product owner.
It's the capability to accomplish a business strategy that defines the mission and vision of the project. Why are we doing this project? How will we recognize we've accomplished our mission? The capabilities delivered by the project starts with the fulfillment of Critical Success Factors. Which in turn implements a Performance Goal in support of a Strategic Objective that measurably benefits the business or supports a mission.
Responding to Change is impossible unless the system is easy to change, easy to maintain, easy to fix, and easy to enhance.
The ability to easily change a product or a process starts and ends with the architecture. This understanding began with Notes on the Synthesis of Form, Christopher Alexander, 1964. It's the architecture that enables the change, assuring that coupling among the components is minimized, cohesion between the static and dynamic processes is maximized, separation of concerns is traceable to all architecture decisions. If you're developing these as you go - allowing them to emerge - you're going to be disappointed when you discover your product is coupled in ways you didn't know, has weak cohesion among it's parts, and has cross cutting concerns which result in a tangled mess when you start to make changes.
The notion that the best architectures emerge are suggested by those not working on complex systems interdependent components, but on systems with lower levels of compelxity between the components. Imagine an enterprise ERP system, a software intensive manufacturing system, the 32 flight and weapons computers on the F-35, the multiple levels of interaction of Future Combat System (I worked the rebaslining of the IMP/IMS for Class I), and process control system found in a nuclear power station.
Now ask, would you like the architecture of these software systems to emerge as the development takes place?
Here's where to start for architecture in the enterprise IT domain. There are architectures for realtime embedded systems as well. For defense systems DoDAF is the architecture framework. So when you here responding to change ask - what's the mechanism that allows you to do that, when the system you're working on is complex, high risk, critically important - say banking, navigation and control, oil & gas supply chain, electric power generation and delivery, health care, drug development, retail, transportation? You get the idea
The notion of Question Everything ignores the fundamentals of every professional process improvement paradigm.
Working on projects is not about the needs of the individual. It's about the needs of the whole. Personal desires must be subordinate to the needs of the mission success. It's not about you. It's about the customer and the governance framework in which the customer operates her business or fulfills her mission.
Questions are great, you can learn at lot from questions. But questions asked without doing your homework are a waste of your time and those you are asking the question to. Go do your homework. Learn about ITIL, COBIT, INCOSE Systems Engineering, SEI, and other professional frameworks first. Then you'll have a basis for your questions. Then start with the root cause test for your questions. When someone says those haven't worked in their experience, don't just ask the 5 whys, seek the root cause.
The Why's approach may be able to reveal the symptoms. But to get at the root cause a deeper assessment is needed. One based on a process framework. A place to the Whys to land. Why didn't the work team follow the established test procedures? Why didn't the customer establish a set of needed capabilities before we started developing stories for the software development effort? These whys then reveal the root cause. The whys need to have actionable outcomes, not just the question. 1st graders can ask why.
Process improvement needs to ask why, but it can only deliver value when there is an actionable answer. No actionable answer in units of measure meaningful to the decision makers? The question everything paradigm is Muda (waste).
Working Product is product that meets the Technical Performance Measures (TPM), the Measure of Performance (MoP), and Measure of Effectiveness (MoE) as defined by the Concept of Operations (ConOps), Statement of Objectives (SOO), and Statement of Work (SOW).
Without stating these attributes of the working product there is no way to tell if it is the right working product. Right for the needed capabilities. Right for the strategy. Right for the technical, operational, and performance requirements. Simply saying working product in the absence of these measures is ignoring the large context of effectiveness and strategic value. When we hear many software features have little value, we can only determine that if the planned strategic value is defined and tested along the way. This is NOT Big Design Up Front. It is the core of strategy making.
But the notion of having working software be put to immediate use needs a domain and context for it to be useful. Otherwise it's just another platitude of the agile vocabulary. Working on orbit for a Navigation and Guidance computer may not happen for 9 months. That's the time it takes to get to Mars. So working needs an operational definition. Working in the full fidelity emulator of the space craft. Working in the complete Verification and Validation (100% thread coverage) of the emergency shutdown system. (I was one of gthe orginal architects of this system). Working in the full transaction processes system test bed.
Crunch-time is a symptom of harmful and counter-productive attitudes.
It's got nothing to do with attitudes, and everything to do with competent and mature business management and processes. Newspapers have crunch time every day, sometime twice or three times a day. Banks have crunch time every month. Surgeons have crunch time once they make their fist cut. 3 miles out onto a hot LZ in I-Corp, 1969, is crunch time delivering critical supplies to Fire Base Rip Cord. Flying to New York City has crunch time every time the 777 pushes back from the gate at LAX. It's not attitude, it's competency to manage in the presence of uncertainty and deliver as promised because you've been trained, have experience and a support system. But in the end it's the process. Process rules.
Knowing the capacity for work starts with knowing the demand for work. Throughput can only be determined if you know both demand and capacity. Then and only then, can you add margin for the irreducible uncertainties. And reserve for the reducible uncertainties.
We are only innovators of our process if we are capable of providing the innovative solution within the governance framework of our business.
If it ain't broke don't fix it. If it's broken first find the root cause and fix that cause. Rarely in modern business is the a broken process that didn't work right at one time. A critical success factor for all process improvement is to determine the root cause of the failure. Then and only then examine if there is a process problem. If so, fix the process. If not, fix the application of the process. Stop wasting time looking for solutions to the wrong problem.
The object of all projects is to deliver value to those paying you to do the work.
Writing software for money is not the same as producing art for money. If you're producing art for money, you're probably not a very good artist. If you're treating your job of producing value for money as art, you're probably not getting a lot of repeat customers.
Customers bought the capability to do something, they only care if you're self-actualized if they are a relative. Customers are happy when you've fulfilled their need to possess a capability for the expected cost on the expected day. There must lots of opportunities for participants on a project to receive personal satisfaction, grow together as a team, increase their skills, and even be innovative - but the customer rarely is willing to pay for that directly. It better be wrapped in the overhead rate. Good artists copy, great artist steal - Pablo Picasso. Good firms hire people already prepared to succeed. Read Making the Impossible Possible: Leading Extraordinary Performance: The Story of Rocky Flats for specific actgionable advice on doing all the things needed for success, including all the people processes. The abstract is here.
The more we understand that improvement is hard work. There is no free ride.
Nobody Ever Gets Credit for Fixing Problems that Never Happened: Creating and Sustaining Process Improvement is a start. They suggest less than 10% of the firms adopting Toyota's TQM actually apply it properly. This loops back to the question everything nonsense, when the questioning is uninformed by the missing root cause analysis of the dysfunction. The source of Dysfunction in the workplace must be determined before any suggestions for improvement can be made. Stating something is the smell of dysfunction is like stating what's that rotten smell as we drive by the recycling center/ Look out the window and see the source. Find the source before doing anything.
At the end of the day successfully managing projects is hard work. But there is plenty of advice. This is one of my favorites.
Twas the Night Before Baseline
(With apologies to Clement Clarke Moore)
Twas the eve for the holidays,
And all through the shop,
Our consultants were working,
Would they ever get to stop?
All the CAMs had corrected
their IMSs with care,
With hopes that the IBR interviews
Would not be nightmares.
When all of a sudden
There arose such email chatter,
IMS updates were piling up,
So which ones would matter?
And what did I see
To my eye’s disbelief?
But even more IMS updates
In BoxNet motif!!
But jolly fat fingers
flew over keyboards,
And corrections were entered
Errors vanquished evermore!
Now CAs! Now WADs!
Now WPs! Now RAMS!
On CARs! On Baselines!
On Schedules! On CAMs!
To the TMT briefings,
To the top TMT Generals,
Now CAM away, CAM away, CAM away all!
Courtesy of Jay Charleston, CAM on a DTRA Anti-Virus Program. Our team was literally working the baseline submittal in late December for an IBR in mid January. Only the strong survive the 72 hours straight of re-baselining $350M a bio-pharma drug development program.
There is a post that references a concept I've come to use that puts uncertainty into three classes. This post it not exactly what I said, so let me clarify it is bit.
First some background. I work on an engagement that provides advice to an office inside the Office of Secretary of Defense (OSD). This office, the inside, is responsible for determining the Root Cause of program performance for ACAT1 (Acquisition Category 1) programs.
These are large programs. Larger than $5B. In most domains outside the ACAT1's this numer is ridiculously large. But inside the circle of large defense programs, $5B is really not that much money. Joint Strike in a Congressional Quarterly and the Government Accountability Office indicated a "Total estimated program cost now $400B," nearly twice initial cost. DDG-1000 is $21,214 Million, yes that $21,214,000,000.
No IT or software development project would come within a millionth of that. If you're interested there are reports at Rand and IDA for the current issues. There are certaintly multi-million dollar IT projects. The ACA web site is probably going to be in the range of $85M to several 100 million. The facts are still coming in. So anyone who says they know and doesn't work directly in the program, proably doesn't know and is making up numbers. GAO will get to the real numbers soon we hope.
Principles Rule, Practices Follow, Everything Else is BS
The principles of cost and schedule estimating, assessment of the related technical and programmatic gaps are the same in all domains for every scale. From small to billion. Why? Because it's the same problem no matter the scale.
The soliloquy in the movie makes a good point -handling the truth is actually very difficult for almost everyone outside the domain - in many instances.
We want the simple answer. We want it all to be fine. We really don't want to do the heavy lifting needed to come up with an answer. We want the simple answer. Many times we don't want an answer at all, we want to just do our job and ignore the fuduciary responsibility to tell others what the cost and schedule impacts are, or even to do our job of discovering that DONE looks like before we start spending other peoples money.
So here's the way out of the trap of at least (1) and (2)
But the words used in the original post that referenced my post are not my intent, nor are they part of any process I work in.
Here's a list of other posts on this topic. It's a crtically important topic. One that deserves deatiled analysis. One that we're obligated to know and use when it's not our money we're spending. It's called Governance.
Here's some more discussion on Estimating for fun and profit.
It's that easy and it's that hard. If you don't have a handle on what risks are going to impact your project, those risks will still be there, you just won't know it.
The first step in increasing the probability of project success is to have some notion of what is going to prevent that success. This means asking what can go wrong, rather than what can go right. In order to answer the question what can go wrong we need to know what we are doing. What is the project about? What are we trying to produce? When do we need to produce it? How much money will we need to spend to produce this things call DONE?
Let's start with some obvious risks that we have to handle for any hope of success of the project. These are obvious because they occur on every project, in every domain, using any project management method.
In Concepts of Mathematics, Ian Stewart, there is a math joke that goes like this...
An astronomer, a physicist, and a mathematician (it is said) were holidaying in Scotland. Glancing from a train window, they observed a black sheep in the middle of the field.
How interesting observed the astronomer, all Scottish sheep are black!
To which the physicist responded, No, no some Scottish sheep are black!
The mathematician gazed heavenward in supplication, and then intoned, In Scotland there exists at least one field, containing at least one sheep, at least one side of which is black.
Pick your role here. When I hear words like this can't be done, this has never been done, doing this is evil, doing this is a waste, this is always done, or any other absolute statement that contains never or always, in the absence of a domain, a context in that domain, tangible evidence that the statement is effective outside of a single person's observation, the insistence from the speaker that I've told you this many times over, some evidence from somewhere else, untested beyond opinion, or worse just stated because it sounds like a good idea - as Dilbert has mentioned in the past it looks like it's going to be a long day.
It has been suggested that context is somehow irrelevant. This notion seems to start around the criticism of the MoSCoW requirements method where requirements that implement the needed capabilities of a system are categorized.
MoSCoW means: (how it got the moniker I have no idea) and is a well developed method of eliciting requirements in a paradigm where the customer has some notion of what done looks like. If this is not the case, no method will help and the norms of project management are no longer effective - it's a Chaos mode. So in fact MoSCoW has no value and the suggestion may be true - in Chaos we don't need requirements elicitation methods.
But let's assume we're not in chaos and we actually would like to know something about what Done looks like, how much it might cost to get to done, how long it might take, and what of value will be produced when we reach done.
But for projects operating in the other three uncertainty modes - variaton, foreseen uncertainty, and unforeseen uncertainty, the first question is actually the inverse question ...
Why would we NOT prioritize the requirements using this or a similar mechanism?
Let's look at some content in the notion that prioritizing requierements should be ignored. A requirements Trade Space is mandatory on any non-trivial project for some very simple reasons:
Below is an example (at the end) of a program that has a minimum set of capabilities fulfilled by the requirements, where the Program Manager and the Customer managed the success with ruthless adherence to a schedule and the needed capabilities.
This example might serve as a paradigm for other mission critical projects.
Now the question is can this paradigm or context be applicable to small group, agile software projects? Turns out Star Dust had small team software intensive software components. As Shim says think about it.
Proper Preparation Prevents Poor Performance
This simple phrase describes the core behavior of all project related work. For any platitude to survive contact with reality, it must be true, actionable, and applicable in your own domain.
The notion that “emergence” is the driver for the participants in a project requires careful consideration. The technical or business requirements of the outcomes of the project are always “emergent” in some form. To be otherwise would require a preset group of activities, materials, technology, and personnel.
Failing to understand the subtleties of the continuous emergence of requirements, that enable the capabilities to be delivered, means failing to understand any project requires planning to deal with these emerging requirements. This emergence also pertains to the tools and processes used to manage and deliver the project. emergence is applicable to all elements.
Preparing for Emergence is a critical success factor in project work. Proper preparation is the foundation of programmatic and technical risk management. This means asking and answering the “if – than” question, rather than the “what – if” question.
Managing in the presence of emergence requires directed decision making. Just letting things happen disconnects the outcomes of the project from the neeed capabilities produced by the project. These capabilities are the immutable part of the business process. They can only change, when the strategy for the success of the business enabled by the project changes. To do otherwise, would be to disconnect between the investment in the project and the value produced by the project.
“If – Than” means knowing what can go wrong and how to respond to the following:
It's not clear how this notion came about, since I haven't talked directly with those making statement like that. I'll have to estimate myself why this might be true.
One reason might be they haven't been exposed to the statistical processes used to make estimates in a variety of domains, not just project management. When we encounter the need for estimates, we need to come in contact with probability and statistics. All processes on projects are statistical in nature. This produces uncertainty and uncertainty results in probabilistic outcomes.
Estimating means searching in this probabilistic "space" for a number that is close enough to answer a question. How many people are in line for movie tickets inside? Should I buy my tickets at the kiosk outside? Or maybe, how long will it take me to drive from my office to the airport parking garage for my flight on Wednesday morning? These are not exact numbers, these ae quick estimates. The answer to the first question can be made with a quick look inside the theater. The second comes from the experience of driving to the airport several times a month over the past 20 years. One is informed by observeation of the current situation, one informed by past experience under a variety of situations.
Let's start with a simple definition of an estimate - it is an approximation of some value for some purpose, even if the data is incomplete, uncertain, or even unstable. Typically the estimate is derived from a statistical source of data - either observed or referenced in some way.
Making Simple Estimates
When we don't have the exact answer of some value, we can make an estimate of that value. The result is a number and a confidence on that value. Say we need an estimate of how long it will take to drive to the airport. It's 52 miles. Some on surface streets, some on open 2 lane roads, some on freeways. Knowing the speed limit for each of these segments can get us an approximate time portal-to-portal. We can improve the estimate, knowing the weather conditions and the traffic flow on the surface streets.
Let's look at five easy steps for making estimates (abstracted from The "Mother of All Guesses" - A User-Friendly Guide to Statistical Estimation, by Francois Melese and David Rose COPYRIGHT ARMED FORCES COMPTROLLER, 1998)
Making Estimates for Project Work
With the steps above we can start to make estimates of our project work.
But first let's kill some myths used to not to make estimates.
We've heard them all and more. There is an endless list of reasons for not doing most anything on a project, but that doesn't remove our obligation to be stewards of other people's money we are spending with expectation (estimate) of some value in return.
Let's start with a core principle of all project work, and possibly a principle of all life's work when we encounter money.
We can't determine what something is worth until we know what it costs.
This is a crass capitalist point of view I know. But we live in a crass capitalist society and them's the rules. Don't like the rules, it might be better to look somewhere else for work other than spending other peoples money without knowing how much it will cost in the end and when we'll be done.
We all know and maybe even experience a Dilbert's paradigm when it comes to projects. But that doesn't make it right. We hear this some times from management. But we also hear it as an excuse for not making estimates from those who should be looking after the cost needed to compute the Return on Investment.
A Few More Myths
Below are some links to other Blogs on this topic, explore, think about our role in managing other peoples money.
When we hear about software development methods, it's critical to understand in what domain they are applicable. Here's an analogy for that question. Without an answer to this question there is no basis for comparing units of measure. What might be a wonderful approach in one domain would be unusable in another - this is the case for all the examples.
A solo or 3 team project with a customer in the same building would never apply DOD 5000.02, FAR 34.2 Earned Value flow downs. Building the flight avionics to fly to ISS would never use #NoEstimates and emergent requirements on a weekly basis for a 200 person development team spread across the United States.
So before any conversation - beyond shared ancedotes and asking audiences to think about it, because I can't tell you what to do - can take palce, we need to establish the domain, the value at risk, and the governance model applied to our work. Then we can talk about shared ideas.
Matt Heusser's article brings up some interesting points. Let's look to see if there are any limitations from a domain or context point of view. By domain I mean, in what taxonomy are you writing software for money. By context I mean what are the constraints or governance guidelines in that domain.
1. Make the amount of money small
This is a version of time boxing. It limits the value at risk for the development process. This bounds the risk process. In exchange for the total loss of doing the wrong thing information can be found. This is also called tuition cost.
Issue: We may not have all we need to forecast the total cost and schedule. Projects in many domains aren't made up of small chunks of themselves. So we'll need to confirm that the sum of the parts results in the whole. Integration, test, verification, architecture, interface management, and many other Systems Engineering aspects need to be involved in some way.
2. Fund a pilot that delivers working software and use this model to forecast schedule
This is buying a reference class. With the reference class - all be it a class of 1 - a forecast of future cost, schedule, and technical performance. We need all three in the reference class.
Issue: this is a larger issue of Number 1. With the pilot can we be assured that work can be scaled? Verification of that will have to be part of the pilot or a follow on. Then comes the confidence intervals for how that scaling will interact with the other - yet to be developed - components of the system. Is the scaling linear, nonlinear, stochastic interactions, and a whole raft of other discoverable processes. Each needs to be planned and budgeted.The ACA web site is a recent example. A UAV I worked where the engine didn't have enough thrust after the final integration. Etc. etc.
3. Move from contract negoiation to partnership
You've simply transferred to responsibility to estimate the cost and schedule to someone else. A single example - in the article - is not the basis of a syndicatable process. So this example, while interesting, probably isn't going to go too far with someout means to address the Estimate at Complete needed in most non-trivial software development projects.
Issue: Still don't know how much the project will cost in the end.
4. Employ Start Stop Heuristics
Seems like just another version of time boxed budget and schedule.
Issue: still doesn't address the Estimate at Complete before actually reaching the end or near the end of the project.
5. Drop Estimation From Your Estimation Process All Together
Another version of time boxed budget. Someone has to know how much money is needed to run the business on an annual basis. Or how much money will be allocated to do some work on an annual basis. This is called Level of Effort. Work until the money runs out, we give you more money, or tell you to stop. PayPal works in this way - sorta. The prioritization of the work is the responsibility asking for the outcomes. They have a budget. Give that budget (not funds, budget) to the development organization in exchange for delivered code.
Issue: The project will cost a know amount. We don't know what we'll get for that amount. But that might be OK. Once work is been going for awhile, a Reference Class can be built to allow that question to be answered, assuming the requested software fits inside the reference class in some way.
So In The End
The 5 suggestions don't have a domain beyond the single examples. And the suggestions don't seem to have a way to forecast the bounds of the project with an Estimate at Completion beyond the use of the reference class of the project itself. This self referencing reference class seems a bit sporty.
So yes, there are some ways to develop software in the absence of formal estimating. Although 2 of the 5 are actually using reference classes to forecast.
Those paying for the work to be done, still have to come up with some upper bound on cost, schedule, and technical capabilities for that cost and schedule. These 5 suggestions are a start. But we don't yet know where they can be applied. If they have been applied outside of specific anecdotes, and if they are scalable beyond the personal anecdotes.
No problem. This notion of not estimating is still evolving. At some point, answers need to be forth coming and the Yoda-style conversation replaced by business conversation - what did you do with my money?
In a meeting today on the difficulties of increasing the probability of project success in our domain (DOD/NASA). There were three sources of uncertainty around cost, schedule, and technical forecasting
The program runs into things that cause cost and schdeule overruns:
We really shouldn't be doing #1 and #3. #1 means we didn't look hard enough. #3 means we can't handle the truth. #2 is the definition of uncertainty. But is it uncertainty because the project, didn't know or didn't want to know.
Doing stupid things in purpose
When managing projects that are funded by other people's money, we are obligated to know something about the probabilistic outcomes of our work. Without the ability to make forecasts and estimates of these outcomes , we are relegated to the category of labor.
As Project Managers, we must learn to estimate and forecast to some level of confidence. This is not guessing, that is child's play. Choosing not to estimate is also child's play. We must learn to make estimates based in sound statistical and probabilistic principles. Without that, the very notion of Return on Investment is intentionally ignored.
Return on Investment = (Value produced - Cost to Produce) / (Cost to Produce)
It's that simple and it's that hard. The cost and value variables are probabilistic estimates and must be treated in that way. But without knowing either of these, to some degree of confidence, we cannot make informed management decisions.
That is the primary role of project management or engineering development -
Make Decisions based on evidence in the presence of uncertainty