The Five principles of risk management, includes the core concept that cost, schedule, and technical performance are inseparable. There are some in the software business that conjecture software is some how immune from that principle.
These risk principles live in a large context of Five Principles, Five Practices, and Five Processes of project management in general. Project management applicable to all project no matter the domain, technology, tools, methods, business strategy, or those applying these.
There was mention on a Twitter conversation that software projects are somehow unique and the statement #3 is not correct. In fact it is, and here's why.
But first let's start with the Principles, Practices and Processes needed for the success for all projects.
So when it is mentioned that software projects are different and most importantly that somehow cost, schedule and technical performance can be separated, let's consider a few of the mechanics of software projects:
So here we are again back at the foundation of all project work. Cost, Schedule, and Technical Performance (Scope included in this along with everything else not cost and schedule) are inseparable. In order to increase the probability of success for the project the coupling between these three project drivers needs to have margin. Otherwise the project will be whipped by a change in one of the variables
You think because you understood one you must understand two because one and one makes two. But you forgot you must also understand and - Sufi wisdom saying
Much has been written and spoken about how to develop and deploy projects using people and their ideas. The FAA's National Airspace System, Systems Engineering Manaul has something to say about this as well
Three steps are needed for success, with the last step being the most critical.
It starts with Brainstorming. Getting ideas out on the table so we can discuss them
Then comes Brainwriting, where the ideas are built into testable concepts. This iterative process of peer review, peer criticism, and peer contribution is very powerful. It is NOT a love fest where everyone is supportive of any idea. But a serious consideration of ideas amoung peers, professionals, and everyone acknowldeging the importance of the mission. This next step is not for wall flowers or the hurt puppies we sometimes encounter in meetings or other channels of communication.
Then the most important process of all, actually tesing the ideas to see if they will withstand the rigor of actual use. This is where many in the softer side of project management, or those who really like to just explore come face-to-face with the reality of spending other peoples money, on a planned time line, with a commited set of capabilities.
The notion of an adversarial group review process is missing from most IT shops, sometimes for the right reason, but sometimes for the wrong reason.
At the bar during a recent DOD Program Management Conference, a colleage (former cost director of a major government agency) came to the conclusion there are three sources of project failure do to unforseen events.
Don't be victum of #2 and #3. Go find out all the problems, all the impediments, all the unplanned impacts including cost and schedule that will derail your project.
A colleague sent some books and others arrived that are must reads for any project management looking for guidance in the modern world of complex, integrated, emergent systems in any domain.
This book caught my eye after exchanging some Twitter conversation with Troy about the notion of No Estimates. The real interests was not so much the Kanban or even the Scrum - which we do practice - but the Monte Carlo Simulation.
We live MCS on a weekly basis in our project domain. The Schedule Risk Analysis (SRA) is mandated by the acquisition processes used to manage our programs. MCS a be applied in any project domain. In Chapter 4 estimating using MCS is introduced. While Troy presents numbers we would not consider in our modeling - 90% confidence levels - the concepts are all solid. This brings up the important notion that if you're going to model anything - using MCS or Method of Moments or any algorithm - you must know the model of the project. This in itself is a major contributor to project success.
When you hear people quote George Box All Models are Wrong. Some Models are Useful. Ask first can they produce the paper in which he made that statement. The answer is "Science and Statistics," George E. P. Box, Journal of American Statistical Association, December 1976, Volume 71, Number 356, Applications Sections. And when you read that paper and then hear his quote used as an excuse for not modeling your project or system, you'll know that excuse is nonsense. All models are wrong because they are models, not the real thing. But good models are just as good as the real thing in many instances.
Visualizing Project Management, Kevin Forsberg, Hal Mooz, Howard Cotterman, Center for Systems Management (CSM). This book is the basis of a method for managing projects based on Systems Engineering. Systems Engineering is a discpline found on every defense and space project, and in many other domains where complex systems are the heart of project, user communities, and the processes they use to conduct their work.
It is rarely found in commercial software and IT shops. I'd say many of the failures found in IT and software development result directly from the absence of systems engineering principles.
This books shows you how to manage projects, develop products and services, deploy these, and operate them using Systems Engineering Principles. As stated in the forward of the 2nd edition there are 1000's of reasons for failure, bit there is not a single excuse - Mike Reid (Penn State Football).
A Primer for Model-Based Systems Engineering, David Long and Zane Scott is a book about systems engineering, the product developed and sold by the authors at Vitech, and the critically important concepts of applying Systems Engineering to projects.
Like the other two books, this book is a practitioner book. It tells you how to do things. It tells what when and where to apply the principles, practices, and processes of modeling systems using the sysML notation. This notation, the models it produces, and the elements of the model are in use on every program and project I work on. The Vitech tool is very nice. And like other system modeling tools is the vehicle for communication between team members, suppliers, and customers.
When you hear that tools get in the way of communication, ask of those making that statement have ever worked on complex, mission critical, must work first time, high risk projects? No? Like the George Box quote - please be quiet.
These books and the others in this blog under the BOOKS index are used in our practice of Performance-Based Project Management(sm). Books, papers, conferences, professional organizations are places to meet others that share our ideas and many times challange our ideas. Both are needed for improvement in our profession of increasing the probability of project success.
If this is how you see your management, your interaction with them, or how your firm works, then here's some advice
Here's what you do as soon as you have the chance.
If you don't Run Away, I guess you can sit around and grouse about how bad your management is, how to do your job without having to be accountable for how much money you are spending, or pretend that someone else, somewhere in the firm will figure how to estimate before you RUN AWAY.
But in the end if the Monte Python gang is the view of the world of your management, ask yourself if you've hitched up with the wrong crowd but don't know it yet.
Woody asks some important questions that have straight foreword answers in the software for money domain. These answers come for the business side of a software development organization. It's the business of software that needs estimates. Given a choice no developer really wants to estimate her work. I never did, I just wanted to code. But once there is an understanding that my paycheck didn't come from the Bank of America, even thought that's what it said at the top (above TRW), then I slowly learned our customers paid of some value (in that first job a missile defense radar system) they needed to know how much it was going to cost to produce the needed capabilities.
So here's some good questions that have answers from the point of view of those paying for the cost to produce the software for the customer.
So some further questions?
Knowing what it costs to produce a specific value is a critical success factor in any business. Writing software for money is no different.
The discussion (of sorts) on Twitter around "no estimates" - what ever that actually means, since there is no definitive description other than exploring - brings me back to my core program management, project management, writing software for money, designing algorithms for identifying moving targets in radar systems, and other software engineering experiences.
Let's start with a fundamental pricniple of all product or service development, either internal or external. While leading a couple hands full of project managers at a large Department of Energy environmental cleanup site, where software development was a critical success factor - and by the way we introduced eXtreme Programming into an ANSI validated Earned Value Management System - our external consulting firm gave us some good advice. We were bidding our technology and services at another DOE site, with similar cleanup problems. We were working on strategies, balanced scorecards, systems architectures, and the like.
That's all nice boys and girls, but here's some fundamental advice - our customer has money and we want it. What's it gonna cost and when will you be done providing the capabilities to close this site?
That's it, that's the winning strategy. The customer has a need, we want to providea solution to it. How much will it cost and when will we have it. If we can answer those three questions - cost, schedule, delivered capabilities, with attached unassailable beneficial outcomes - we will win. This is a business strategy. All the Balanced Scorecard presentations, examples of past performance, deep references of success, are all for naught if the customer can't afford our solution. It comes down to this - and this is where I learned this from the Managing Partner of the Big 6 (at that time) consulting firm.
You can't know the value of something until you know its cost
That's a fundamental principle of all business transactions. Value is always exchanged for cost. We do this when we buy a Venti Nonfat Latte at Star Bucks. We do this when we pay the lawn care company to mow and trim weekly. We do this when we buy anything, including software or the services that produce the software.
This is an immutable principle of commerce
So when we hear, there are alternative ways of writing software for money that don't involves estimating the cost of doing that work, think again. How did you get around the immutable principle of commerce? Now notice I used the word estimate in the same post as know. Yes, estimating allows us to know something to some level of confidence.
I'll estimate that my 1 hour drive to work everyday, will be extended to 1 hour 20 minutes when the snow storm arrives tomorrow night.
I know I'd better have margin in my drive schedule, if I want to attend the 8:30 stand up.
I estimate that it will take 3 days to install and verify the database for the system, given the historical data from the last 3 times we did this.
This knowledge can then be used to plan the access to the server room, arrange for all the verification and validation data we'll need to certify the contents are ready for use by the customer.
We estimate to a degree of confidence, things (time, cost, performance) we'll need to know about to do our job.
So How Can We Learn To Estimate?
Here's where we start. We start with what has taken place in the past. We've never done this before you say. I'd suggest, working literally in the rocket science world, there is very little in the commercial software world that hasn't been done by someone, somewhere in the past. You may not know these people, but it's been done. And more importantly it's the people issues that muck up the project most of the time, not the technical, unless of course it actually is rocket science, or stealth fighter science, or bioscience.
So with the second best basis of estimate approach - What is this like? We've done similar things in the past, how is this problem like those solutions? Next comes the 10 questions approach. The Planning Game. Then a parametric approach. Function Points, COCOMO, SEER, Price-S, SLIM, CoStar, and a long list of other basis of estimate tools, some free can guide you. So when you hear software can't be estimated, change the phrase to I don't know how to estimate software projects, but I can sure look into learning how.
Finally the least desirable way to estimate is to ask the expert. This only works if the expert has been calibrated with a reference class, has her optimism bias in check, knows all about anchoring and adjustment, and has a track record for producing credible estimates. If not, you're going to be disappointed in the result.
But our management uses our estimates against us. Our management doesn't understand the notion of probability and statistics. Our management behaves like Dilbert's boss. This has nothing to do with the need to estimate the cost, schedule, and technical performance of the product and service needed by your customer. It has everyone to do with managing up. And if that's not possible, producing a credible estimate with those risks baked in to protect yourself. And if that's not possible start looking for a better manager or even a better job because your company is going to be in the ditch before long.
So when we hear that estimating is the smell of dysfunction - without ever listing one single dyfunction - remember there are lots of dysfunctions in business. This is normal, because humans are involved. But that dysfunction is not caused by the need to estimate. The need to estimate is a core business process. Doing bad estimates, doing estimates for the wrong reasons, doing estimates wrong - that's a dysfunction that is universal.
In the end you need to either nut up or shut up as Woody says. Yes, that Woody. Learn to estimate for all the right reasons, then when there is an opportunity to have an enlightened manager at your current firm or a new firm, you'll be prepared to contribute value to the business process in ways that benefit the top line.
Since that top line, minus the costs to produce the goods or services is the bottom line (in it's simplest form) is what writing software for money is all about. Knowing the middle line - costs of goods sold (COGS) is critical to actually staying in business.
There was a question posed on LinkedIn
How do you get Project Manager understand the importance of Earned Value Management?
It's been shown over and again when we make EV a compliance process and hold managers accountable for compliance it is a necessary condition for success, but far from sufficient.
Until EVM starts providing actionable information to the program management staff in the form of "leading indicators" it will always be one of those compliance processes. This information must reveal where in the program, technical and programmatic changes can be made, the testable outcomes of those changes on program performance, and the impacts on cost and schedule forecasts and the EAC against the BAC. Starting at the program level, but going down to the Control Account and Work Package.
In other words...
Nice data there in your Formats 1-5 and really nice IMS, what should I do next?
EV numbers themselves are lagging indicators, non-statistically adjusted, non risk adjusted, not connected to the effectiveness and performance measures of the project - unless enlightened users do so - and not connected to the needed capabilities of the customer as delivered by the program. There is no DID for the IMP, so Systems Engineering rarely flows down MOEs and MOPs - except where they do, because of the knowledge of the power of this approach.
Next is a real problem. 748-C has a loop hole big enough to fly a 787 through. In section 3.8 "Earned value is a direct measurement of the quantity of work accomplished. The quality and technical content of work performed is controlled by other processes." Measuring quantity is a construction centric view of producing value. Not a value centric view. Our defense, space, biopharma, software intensive programs that apply EVM are not pouring concrete or welding pipe. (I used to work in that domain as a SW developers on piping design systems). If we see EV as measuring quantity we ignore all the concerns Paul Solomon has brought forward, starting with the missing Technical Performance Measures for the product.
Sure we have exit criteria for the Work Packages. But these need to trace vertically through the IMP to the Capability to accomplish the mission or provide a solution to the business need. In other do something with the resulting product or service. It has to be the right thing of course. But EV - as stated in 748 - only measures to quantity of parts needed to delivery a capability, not the ability of the system to fulfill the needed capabilities. This by the way is essentially a Systems Engineering issue. With the missing IMP, most of the motivation for connecting the dots between EV and mission success is simply not there. It's back to the immutable principle
We don't know what done looks like in units of measure meaningful to the decision makers
"Earning value" means assessing the efficacy of the BCWS. This means assessing the Technical Performance Measures of the work be delivered. Many implementations assign TPMs and QBD's this work. But this is not called out in 748-C nor any formal EV guidance. It is called out in the DAG and the old SEMP DID. No SEMP DID is in place.
So as EV practitioners we need to make the business case of EVM, not just the compliance case. When we add "leading indicators" that are statistically sound (Eric Druker has written about this as have others, including me), forecasting of EAC based on ARIMA processes rather than our linear, non-statistical, non risk adjusted CPI/SPI measures. As well those measure "roll up" the variances of all the past time series data, just like Darrell Huff tells us to do in "How To Lie With Statistics".
All the data is available in the EV engine and in the CR at PARCA for DOD jobs. What's next is to start using EVM in the same way supply chain managers, quality control managers, systems engineers, and every design and manufacturing engineer does - compare statistically adjusted performance against the probabilistic plan to see leading indicators emerge of where we are going to go into the ditch in the future, show the PM, that corrective action "before the fact" rather than after.
This will be a topic discussed at EVM World and ISCEA conferences coming soon.
Project success is elusive in all business and technical domains, including software development, construction, new drug development, any project involving multiple participants, irregular funding, and emerging requirements and risks that haven't yet been identified and handled.
To increase the probability of project success we must start with principles and apply practices implemented by processes known to produce benefical outcomes. Without these principles the practices and processes have no home.
These principles provide the framework for practice and process. Let's establish them first:
Identify the Technical and Operational Capabilities
The capabilities needed for success of the project, mission, or business are defined by the customer. Through a strategy development process or some other process that states why these are needed in units of measure meanigful to the decision maker. These measures are stated as effectiveness and performance goals.
Construct the Sequence of Capabilities
The capabilities must be delivered in an order that maximizes business value, minimizes risk, and maximizes opportunities along the way to make improvements for the customer.
With the sequence of work in place, we can now look for risks to our success and opportunities for improvement. Risk is created by uncertainty. Uncertainty comes in two flavors:
With sequenced capabilities, risk handling plans, and opportunity plans we are now able determine the cost and schedule of the work needed to deliver the capabilities. This work starts with packages of work holding the budget for the work and describing the period of performance. This result shows us the cost needed to produce each capability. This cost can be compared to the beneficial outcomes from the capability to confirm the business case or mission strategy if viable from a business point of view.
For each chunk of work in the plan to implement the needed capabilities we need some method to measure progress to our plan. These measures must be based on tangible evidence of physical percent complete. This can be done through three basic approaches.
Each of these assessments of progress to plan is based on a pre-defined unit of measure. This avoids the opinion of progress we often see on projects stuck at 80% to 90% complete. It also prevents measuring progress with the passage of time and consumption of money. Even the notion of working software has to have a tangible measure of working. Passing a test is fine. Is it the right test to confirm that a requirements is fulfilled to deliver a capability?
Each capability of the insurance provider network ERP system above is developed in a planned sequence to provide value in support of a business strategy. This order includes minimizing risk and maximizing opportunities. Each point where capabilities join the business can put these to work in generating value.
We need to know what DONE looks like, how we’re going to arrive at DONE on-time, on-budget, with the planned capabilities, what resources, funding, and work sequences we need, what risk handling and opportunity management can be performed, and how we’re going to measure tangible physical percent complete along the way.
These Principles enable 5 Practices and 5 Process to be implemented to increase the probability of project success. The details of all these can be found in Performance-Based Project Management(sm).
It's not you money, behave apprioriately.
When asked to develop software in exchange for money - this is leaglly called work for hire - we have an obligation to have some notion of the final cost. If not, we're likley working a staff augmentation and not doing work for hire. So let's assume we are on a WFH engagement. Knowing something about the final cost starts with estimating that cost. This estimate says its name - it's an estimate. It's not a firm price. It's not even a firm anything. It's an estimate.
This estimate can come in several forms
The answer to both of these approaches comes from making estimates. Estimates of cost and schedule. Estimates of capacity for work. The process for making these estimates is called Basis of Estimate and usually starts with an anchoring nbsp;process.
By anchoring it means making the estimate from something that can be used as a reference. It's been done before, there is a model of what could be done. Some reference class. This anchoring process is then followed by an adjustment process. Adjustments can come in a short time, or over a longer time. But the anchoring and adjustment paradigm is well developed.
One research study has shown
One way to make judgments under uncertainty is to anchor on information that comes to mind and adjust until a plausible estimate is reached. This anchoring-and-adjustment heuristic is assumed to underlie many intuitive judgments, and insufficient adjustment is commonly invoked to explain judgmental biases. However, despite extensive research on anchoring effects, evidence for adjustment-based anchoring biases has only recently been provided, and the causes of insufficient adjustment remain unclear.
Anchoring and adjustment is well studied in behaviour finance - why people make financiual decsisions that they do. Anchoring and adjustment is also well studied in estimating starting with Oil & Gas estimates of reserves to estimating public works projects. A complete litertaure search can be found from Google of course.
But estimates needed for making business and techncial decisions must take this research into consideration is they are to be successful. For cost and schedule estimates the best place to start is past performance. Here's an orginal drawing of Flybjerg's Reference Class Forecasting process flow. Google will find you all his work as well. Many about infrastructure and some about IT.
When it is mentioned we haven't done this before so we have no reference classes, we have to pause and think. Is this a trueGreen Field project. Technology that has actually never beed done before is rare in the IT world. If this project is truly without precedent, it's likely an R&D project and need to be planned much differently.
So assuming this is not an R&D project, what can we do about creating estimates when we have little past performance? There are many tools available, some free, to start the estimating process. To produce an anchor estimate, we can start to refine before the project starts, or even after the project starts.
There are three types of activities that paerticipate in the estimating process
So when we hear estimating is the smell of dysfunction and no dysfunctions are listed let alone any credible and in use estimating processes, it's time to question the wisdom is assuming estimates are the problems with software projects.
So let's invert the conversation for a moment.
In the end it's the business that needs the estimates of the developers have decided they are not useful. It's not the developers money - if it is no one cares how they spend it. The business has a obligation to the shareholders, investors, the funding agency, to have some understanding of what the cost for the products or services will be when they are done.
Shim Marom provides thought provoking posts. The one on Estimates and Not Estimates was a perfect straight line requesting a response.
So some other thoughts:
If No Estimates is about making decisions, then No Estimates better come up with the cost of outcomes of having made that decision or that statement is completely bogus. Time to start looking for business value from our work efforts in building software for money. That means know the cost of our efforts to some degree of confidence before and during the project. Knowing afterward is simple and worthless, that horse ran away and know we're saying how much it costs.
Much of the discussion on building products and services is focused on eliciting requirements, developing solutions around these requirements. Ignoring for a moment the silliness of not knowing the cost or duration of this work effort, we still need to address the business side of spending other peoples money in exchange for delivery a known value.
Over the past months I've come to see the world through the eyes of Systems Engineering. I have a MS in Systems Management, but haven't applied it in 25 years.
My current assignment is on a large software intensive system that is a national asset. This is a code word for it has to work as planned or 100's of million of people, sovereigns, and other users will be disappointed at best and possibly be put in harms way at worst.
This systems view means: (Systems Thinking for the Enterprise: New and Energing Perspectives)
So it all comes down to this:
If you don't know what someting costs - in units of money, schedule, manpower, infrastructure, tools, process risk, delay - you're not going to be able to know what it is worth.
That's it, It's that simple. Anyone suggesting their processes or even a HashTag masquerading as a suggested approach to problems, that this solution will increase the Probability of Project Success (POPS) or their suggested solution is focused on providing value to the customer, must address how that suggestion will work in the presence of the reality of building products and services in exchange for someone else's money.
Earned Value measures the production of tangible outcomes from work efforts on a project using Physical Percent Complete. This P%C is calculated used Quantifiable Backup Data shows what technical or operation goals must be met on a planned date in order to claim some Percent Complete. Measures of this physical percent are NOT how much money you spent or how much time you took to do the work.
They are only measured in unit of tangible technical progress to plan. This tangible technical progress is defined BEFORE the work starts. This progress goal is derived from the time phased Measures of Effectiveness and Measures of Performance goals of the project.
These MOE, MOP, KPP, and TPMs are defined below and their measures - in units meaningful to the decision makers - established before work starts, put on baseline and used to compare progress to plan (time phased) to produce that Physical Percent Complete in the Earned Value (BCWP) formula in the first chart.
This branch of mathematics [probability] is the only one, I believe, in which good writers get results entirely erroneous - Charles Sanders Pierce
When it is said - you can't estimate the future - or we don't know total cost, think of Mr. Pierce. All things project management are probabilistic drive by the underlying statistical processes of irreducible and reducible uncertainty. Rarely, if ever, are these uncertainties Unknowable, in the mathematical sense.
Performance-Based Project Management(sm) on Amazon
Here's the Table of Contents and Chapter 2. This chapter is the anchor for the Principles, Practices, and Processes of increasing the probability of project success.
The processes by which we have discussions about project processes must be based on probability and statisics. Every project is subject to aleatory (irreducible) and epistemic (reducible) uncertainties. When we talk about cost, schedule, and technical performance are all statistical in nature.
Here's a book that should be read before engaging anyone in a conversation about the processes of project management, estimating, risk, performance management.