Principles, Practices, and Processes to Increase Probability of Project Success
It's popular in some agile circles to question everything. This begs the question - is there any governance process in place? No? Then you're pretty much free to do whatever you want with the money provided to you to build software. If there is a Governance process in place, then that means there are decision rights in place as well. This decision rights almost always belong to the people providing the money for you to do your work.
Questioning those governance processes and questioning the principles, processes, and procedure that implement the governance processes usually starts with the owners of the governance process. If there is a mechanism for assessing the efficacy of the governance, that's where the questioning starts. Go find that place, put in your suggestions for improvement, become engaged with the Decision Rights Owners and then provide your input.
Standing outside the governance process shouting challenge everything is tilting at windmills.
So when you hear that phrase, ask do you have the right to challenge the governance process?
All successful projects adhere to these five immutable principles during the lifecycle of the design, development, deployment, and operation. These principles are independent of any project or program domain or context in that domain. They are also independent of any project management method or product development method, including agile.
They ask five questions that must have credible answers that establish the foundation for success. Without credible answers to these 5 questions, the project has little hope of success.
So if you hear some unsubstantiated conjecture like ... decisions can be made without estimating ask how any or all of the 5 immutable principles can be met?
If your business is not subject to any external governance process, you’re free to spend your money as you please. But you’re not free to suggest your approach is applicable to those who are governed by external frameworks of spending and accountability for that spend, without a testable confirmation this idea doesn’t violate those governance principles.
Governance includes: Responsibility for a specific duty, task, or decision. Authority to influence behaviours. Communication of decisions. Empowerment to give individual's authority to act.
The governing of IT systems has two distinct components.
All businesses that operate inside governance frameworks, which address:
make use of estimates in their decision support processes. To suggest Not Estimating can be the basis of those decision making process is to willfully ignore these principles.
If it’s not your money, you don’t get to do as you please. If it’s your money, do as you please no one really cares. If it’s your customer’s money, confirm with them how they expect you to behave when spending that money.
There's a meme going around in some parts of agile that management is inhumane. This is an extreme view of course, likely informed by anecdotal experience with Bad Management, or worse lack of actual management experience.
Managing in the presence of Agile is not the same as managing in traditional domains. The platitude is Stewardship, but that has little actionable outcomes needed to move the work efforts along toward getting value delivered to customers in exchange for money and time.
One view of management in Agile can be informed by Governance of the work efforts. Here's a version of Governance, from "Agile Governance Theory: conceptual development,” Alexandre J. H. de O. Luna, Philippe Kruchten , and Hermano Moura.
There a popular notions in the agile development world that authors like Hayek and Taleb speak to how software development works. Let's look at one example. You Can’t Understand Agile Without Understanding Hayek. Let's look at Hayek's paper. "The Use of Knowledge in Society," F. A. Hayek, The American Economic Review, Vol. 35, No. 4, September 1945, pp. 519-530.
Let's look at the thesis of Hayek in light of software development and the decisions that must be made when spending other people's money in the presence of uncertainty. Below is from the paper, but download the paper and read the whole thesis. Hayek was a social scientist. He was not a program manager of engineered to order software intensive system of systems. His ideas have not held up well. Hayek and Modern Macroeconomics. So I'd suggest not spending your customers money before doing some more research.
What is the problem we wish to solve when we try to construct a rational economic order?
On certain familiar assumptions the answer is simple enough. If we possess all the relevant information, if we can start out from a given system of preferences and if we command complete knowledge of available means, the problem which remains is purely one of logic. That is, the answer to the question of what is the best use of the available means is implicit in our assumptions. The conditions which the solution of this optimum problem must satisfy have been fully worked out and can be stated best in mathematical form: put at their briefest, they are that the marginal rates of substitution between any two commodities or factors must be the same in all their different uses.
This, however, is emphatically not the economic problem which society faces. And the economic calculus which we have developed to solve this logical problem, though an important step toward the solution of the economic problem of society, does not yet provide an answer to it. The reason for this is that the "data" from which the economic calculus starts are never for the whole society "given" to a single mind which could work out the implications, and can never be so given.
The peculiar character of the problem of a rational economic order is determined precisely by the fact that the knowledge of the circumstances of which we must make use never exists in concentrated or integrated form, but solely as the dispersed bits of incomplete and frequently contradictory knowledge which all the separate individuals possess. The economic problem of society is thus not merely a problem of how to allocate "given" resources-if "given" is taken to mean given to a single mind which deliberately solves the problem set by these "data." It is rather a problem of how to secure the best use of resources known to any of the members of society, for ends whose relative importance only these individuals know. Or, to put it briefly, it is a problem of the utilization of knowledge not given to anyone in its totality.
First some questions:
Is there vague and unfolding knowledge, driven by externalities, of what the project should be accomplishing, with uncertain and variable funding? With members of the project team having differing views of the outcomes from those paying for the project?
If so, the project has already failed, the members of the project team are on a death march (Yourdan) and should start looking for work elsewhere, before it gets canceled.
So like the other misrepresentations of seminal works, the agile community adopts this attractive ideas with little or no consideration or understanding of their principles. And most of all it those principles are applicable to software development or if those principles are even applicable in today's understanding of who the world works. Macroeconomics is the dismal science - treat it as such when developing software.
So whenever there is some conjecture that someone outside the software development domain has a principle that can be applied to Agile, test that idea first before spending any time and money acting on it. Especially if that time and money is not yours.
Risk Management is How Adults Manage Projects - Tim Lister
Tim's quote offends some people. Here's a collection of his presentations. But this not really the point of this blog. I'm working two programs where Agile (Scrum and SAFe) is integrated into the Earned Value Management System, complaint with EIA-748-C and the guidelines for deploying, using, and reporting progress to plan with that system. This integration is directed by FAR 34.2 and DFARS 234.2 on Software Intensive System of Systems programs.
These programs are similar to Enterprise IT programs, they have mission critical outcomes, they are expensive, they have high risk, they are not simple structure - by definition, and they have deadlines and budget goals set of the enterprise for maintenance of corporate performance.
Outside of the programs we work, we encounter much confusion around risk management, the source of risk, and the separation of the effective and non-effective process of managing risk in the presence of uncertainty. Let's start with a framework for making decisions in the presence of uncertainty. This briefing has been shown before here, many times. But it can't be shown too many times, because there is much misinformation about how to manage in the presence of uncertainty. This briefing is part of a much large set of charts developed over several years for an Office of Secretary of Defense for Acquisition, Technology and Logistics. These are the people that buy things for the warfighter and those who support them.
It may not appear to be applicable in all domains, but I'd strongly suggest it is. Because uncertainty is present on all projects. And the risk this uncertainty creates is present on all projects. Uncertainty and the resulting risk cannot be avoided it can only be managed.
When we hear Agile is Risk Management it's not true. Agile Software Development in the form of Scrum, XP, DSDM, Crystal, and now even Agile Prince2 is just that - a Software Development Method using the principles of agile.
In some domains by the way a few of those principles are violations of governance of other people's money. Many business people have day jobs and aren't going to be available on demand to work with the developers. standards - DODAF, ToGAF, 1553 Bus, ISO 12207 etc. define architectural processes.
But that aside for now, Agile provides information for risk management through frequent feedback and adaptability. But Agile is NOT risk management for a simple reason ...
Agile does not model the uncertainties that underly all project work.
Agile has no notion of margin. Agile has an informal notion of risk reduction of Reducible Risks (Epistemic uncertainties if you read the briefing). But those irreducible uncertainties (Aleatory) are not addressed by Agile. Research shows the irreducible uncertainties are the killer problems on all project work. These come from the natural variances of humans, processes, and machines.
Agile in the form of methods has no process model for addressing the reducible and irreducible uncertainties other than to provide data on performance to date to make decisions about mitigations, margin, and management reserve needed to MANAGE Risk in the presence of uncertainty.
And of course the final tag line
To Manage Risk in the presence of uncertainty we must Estimate not only the probability of occurrence of an Event based risk, but also Estimate the statistical processes of naturally occurring variances and then estimate the impact of those uncertainties creating risk, Estimate the cost and schedule to mitigate those risk, and finally Estimate the residual uncertainty after mitigation.
Yes, Risk Management is How Adults Manage Projects and of course Estimating is how Adults Manage Projects
There a misinformed notions floating around the agile community that products and better than projects. That some how - unspecified and unsubstantiated as usual - that projects are undesirable and focusing on products and their value - again unspecified how to measure that value and unsubstantiated that value measurement isn't what projects do as well.
A Project is a temporary endeavor undertaken to create a unique product or service. There is a definition (possibly emerging) of what needs to be delivered and a target date (with a probabilistic confidence) when it needs to be delivered. A project is unique in that it is not a routine operation (Server Ops is not a project, it's operations), A Project is a specific set of operations designed to accomplish a stated goal. The project team often includes people who don’t usually work together – sometimes from different organizations and across multiple geographies. Projects for example:
Project management is the application of the knowledge, skills, tools, and techniques to the Project's activities to meet the project requirements.
A Product is anything that can be offered to a market to solve a problem or satisfy a want or need. It has a life cycle with multiple stages. A product is conceived, developed, introduced and managed in the market, and retired when the need for it diminishes. A product can only be developed within the context of a project, and multiple projects can occur within a product’s life cycle.
There are multiple roles in the Product development
Product managers and project managers work closely together in high-performance organizations. And both work with the broader product team and executive leaders.
When I hear ...
Project: laced with untested assumptions, based on industrial thinking. Product: cont. tests assumptions, based on lean/agile good practice.
I'm breath taken at the serious lack of understanding of the roles, principles, processes, procedures, and governance frameworks for spending other peoples money to produce products.
A simple summary. A Product is a entity provided to a user. It can be anything: a physical product that you hold in your hands, a software application, or a service that you are delivering. A Project is the series of activities to produce the defined outcome that turns into theProduct.
† I used to work in the Product Lifecycle Management business. PLM integrates people, data, processes and business systems.
In Agile development the quest for Value has become an obsession in the absence of other factors. The first missing attribute is what does it cost to produce that mythical Value. We focus on value over focusing on cost, is a nonsense statement in any Managerial Finance paradigm.
We cannot assess the value of a product or service until we know the cost to acquire that product or service
In the value definition parlance we need to use units of measure that are meaningful to the decisions makers
These are connected as
The next level of detail of these measures can be described below from "Review of Value and Lean in Complex Product Development," Ghadir I. Siyam, David C. Wynn, and P. John Clarkson in Systems Engineering, Vol. 18, No. 2, 2015.
In the end the define of Value is a Systems Engineering process.
So when we hear we focus on value ask how much does that value cost? What are the units of measure of that value? Are those units of measure meaningful to the decision makers?
No answers, then value is a meaningless phrase, just a platitude.
All project work is random work. There are three core random variables on all projects, shown below. There are sub-variables as well as all the ...ilities involved in project work, but let's start with the major three.
Fixing, 1, 2, or all 3 of these random variables does NOT make the randomness go away.
These variables are random and all variables on projects are random because of uncertainty. This uncertainty (as mentioned on many other blogs) comes from two sources. Aleatory uncertainty that is the underlying natural randomness of all project activities. This is called irreducible uncertainty. It can't be reduced. Nothing you can do will reduce it. It's there and will always be there. This is a statistical process. The only way to work in the presence of irreducible uncertainty is to have margin. Cost margin, schedule margin, technical margin.
The second is epistemic uncertainty. This is uncertainty that is event based. It's there but can be handled in some ways. Those ways can include buying two of everything in case one breaks, having redundancy in other forms - a backup site for the data center, testing, prototypes, and other activities that provide a Plan B when the probability that something will go wrong becomes true and that thing that went wrong is no longer a probability but has turned into an Issue.
So Here's the Real Problem
When we hear, we don't need to estimate, I can fix time and budget, that doesn't make the randomness go away. It just sets an upper bound on what you CAN spend and when you HAVE TO BE DONE. Those uncertainties that create the randomness are still there. Then fixed time and fixed budget plans, leave open the technical randomness as well. The time and budget are still random inside the constraints set by the project.
There's no getting around this. No matter how often someone says you can. Those someones were asleep in the engineering probability and statistic class. Here's the classic engineering course we were all forced to take as physics grad-student Probability and Statistics. †
This is basic probability and statistics of project work. The probability that something will turn out unfavorable is created by epistemic uncertainty. The statistical variances of everyday life are created by aleatory uncertainty.
Ignoring these uncertainties means it's going to turn out bad for those paying for your work
You need margin to protect from irreducible uncertainty. You need specific actions to protect from reducible uncertainty. So you can in fact fix the cost and schedule IF AND ONLY IF (IFF) you have margin and risk buy down plans. When someone says we've fixed the duration and the budget. two things come to mind.
A third notion is the killer notion
When you fix time and cost, have sufficient risk buy down activities to reduce the epistemic uncertainty that creates the probability of something going wrong to an acceptable level, and have sufficient margin to cover the expected overruns in duration, you still have the technical reducible and irreducible uncertainties that the things you building won't work, won't be what the customer wants, will cause other issues - these are called externalities in the economics of software development, and other unknowns, possibly unknowable at the beginning of the project.
When you fix time and or budget, and don't have protections for reducible and irreducible, you're going to be late and over budget and you have willfully ignored those outcomes. Oh and by the way, there is a probability your little gadget is not likely to meet the needs of those paying you either.
These immutable condition (aleatory and epistemic uncertainty) are completely ignored in agile development. Agile provides rapid feed back to the risk management processes of software development. But agile is NOT a risk management process in and of itself. That's a topic for another time.
If you think you have no uncertainties - reducible or irreducible, and have fixed the budget and duration and maybe even the outcomes. You're likely on a de minimis project. Good luck with that.
† We had to take a few courses outside our major, and this was another. Classical Electrodynamics. This was an engineering course. We had a foundation of electrodynamics from the physics point of view. In that view everything can be solved through Maxwell's equations. A simple set of partial differential equation describing how electromagnetism works. When asked to give a talk on antenna theory in the engineering course, a friend (I was too afraid at that time) went to the chalk (yes no white boards) one wrote done maxwell's equations for the reciprocity theorem of antennas in free space. The Professor at the back of the told him (Steve) to sit down. We're engineers not physicists we want to know HOW things work not WHY things work
There is Nothing so Practical as a Good Theory
This quote comes from the German-American social psychologist Kurt Lewin (1890-1947), who was pointing out that psychology was thin on good theories, but benefited greatly when there was one. A good theory simplifies explanations and makes them more coherent, robust, objective, and even allows better predictions of behavior.
So what's the theory - Principle - of making decisions in the presence of uncertainty without estimating the impact of those decisions? So far haven't heard one that could be tested outside of personal anecdotes 0f it works for me, so it must work for you.
The real problem that was brought to light by Woody's Zuill's original post way back when is quite simple
Estimates are misused by bad management to hold people accountable for things can never be accountable for. But it's not the estimate or the estimating process that is the root cause. It's the bad management.
But conjecturing - and it's pure conjecture - that Not Estimating is the corrective action for that Root Cause of dysfunctional management is essentially nonsense. First the dysfunction is behavioral - Bad Managers. Not mathematical - estimates made in the presence of underlying certainty.
Yes estimates are sometimes harder. Much too hard for those who have no experience making estimates. Even much harder when the customer is clueless about what she actually wants to spend her money on. So as Capers Jones says below, if our customer can't come up with some form of needed capabilities in exchange for the money being spent - we don't know how much or when we'll be done.
And if the customer puts an upper bound - a Not To Exceed contract we call that - on the spend, it doesn't remove the other two random variables from the work effort - Time and Capabilities. If the customer doesn't have some notion of What Done Looks Like - and not the lame definition of done found in the agile literature - but the real definition of done in units of measure of effectiveness, measures of performance, and all the ...ilities associated with the work outcomes - then you're on a DEATH MARCH project and estimating or not estimating isn't going to add one iota of increased probability of success.
Here's an example of a VERY software intensive system of systems A consistent multi-user, multi-goal framework for assessing system performance with application to a sonar system. This is not likely a system you will have worked on. But it is similar most all other Software Intensive System of Systems found in Enterprise IT.
But Jones's quote again fits a very broad set of domains - all domains I'd suggest.
When we mean to build, we first survey the plot, then draw the model, and we see the figure of the house. Then we must rate the cost of erection, which if do find outweighs ability. What do we then, but draw a new model in fewer offices, or at least detest to build at all?
- Bardolph, Henry IV, Part II, Act I
The goal of managing other people's money when build a product or providing a service is to plan and coordinate the needed work activities to deliver a satisfactory outcome, or complete enterprise endeavor within the constraints of schedule, budget, resources, infrastructure, and available technology.
The intellectual content of the discipline of engineering, business and technical management, risk management, and program controls, are oriented on the components and are value neutral.  Not matter the outcome the processes are the same or similar. The value produced by these efforts is independent of the means to produce them. Once delivered he consumer of this value cares little how they arrived. That consumer didn't buy the process to produce that value, they bought the value. When these are confused the notion of focusing on value is perverted to focused on those spending the money rather than on those providing the money.
The underlying principles of these disciplines are focused inside the boundaries of that system. The resulting value is focused outside the system of it production.
Project success depends on the integration of the activities below. The primary role of the processes below guides the value producing activities to …
Design the Programmatic Process to support the Technical Project Engineering activities to Increase the Probability of Project Success.
 Systems Engineering Body of Knowledge, V0l.5,
The Investors concern about returns over a limited horizon is a pervasive feature of all business management decision making.
This is a core concept when the business invests to produce some value. When this investment takes place in the presence of uncertainty, the expected return from that investment is uncertain as well. When there are multiple choices to be made with regards to multiple investments and multiple returns, the decision making process falls into the realm of MicroEconomics and decision making, Real Options Theory, and Managerial Finance.
Applying these principles when spending other peoples money is the basis of good management. This does npt prevent things from going wrong. Nothing will prevent things from going wrong. But when managing in the presence of uncertainty, willfully ignoring the principles of Microeconomics and Managerial Finance, will certainty set you on the path to disappointment in the outcomes when the time and money run out.
A 2013 webinar at Cyber Security & Information Systems Information Analysis Center, presented some Immutable Laws of Software Development. These are worth repeating every time there is a suggestion hat some method or another, or some new and untested idea is put forth that will increase productivity by 10X or increase your profitability by NOT doing core business processes.
Here's the list presented in the webinar and is dedicated to Watts Humphrey who said all these in the past. For each Immutable Law, I've made a suggestion on how to avoid the undesirable outcome.
One of the primary responsibilities of management is to make decisions during the execution of projects so that gains are maximized and losses are minimized. Decision analysis is critical for projects with built-in flexibility, or options. when these choices operate in the presence of uncertainty. 
The notion that we can spend other peoples money in the absence of the immutable principles of Managerial Finance and Microeconomics of decision making pervades the #Noestimates community. Here's a few of those principles to counter that fallacy.
We can't know the benefit until we know the cost to achieve that benefit
Microeconomics of decision making in the presence of uncertainty is based on Opportunity costs
Uncertainty is a central fact of life on most large IT capital investments. Along with uncertainty comes managerial flexibility. A real option refers to the right to obtain the benefits of owning a real world asset without the obligation to exercise that right.
So in the end, when making a decision - that is selecting from more than one choice - there are several methods listed here.
Each of these methods of making choices in the presence of uncertainty requires making an estimate of the impact of that choice, an estimate of the cost to acquire the value for the choice, and an estimate of that value.
 Real Options: The Value Added through Optimal Decision Making, Graziadio Business Review,
It's common these days to re-purpose a quote or a platitude from one domain into another and assume it's applicable to the second domain. My favorite recent one is
"Layers of redundancy are the central risk management property of natural systems” - Taleb
Taleb is the author of Black Swan, about long tailed statistical processes in the financial domain. These Black Swans tend to bite you when you least expect it. Are there Black Swans in the software development domain? Only if you're not looking. Financial systems are rarely engineered to perform in specific ways. Software systems are, st where I work and I suspect everywhere someone is paying money for the system to be developed or acquired.
So let's look at the Taleb quote that is often re-quoted by agile people and especially those advocating no estimates.
First some full disclosure. One of my graduate degrees is in Systems Management, which is a combination of Systems Engineering and Finance. As well I work with systems engineers and support systems engineering processes in the aerospace and defense domain. So I'll predisposed to view the work through the eyes of Systems Engineering. Everything is a System is a good starting point for what we do.
Now let's look at the Taleb quote through the eyes of Systems Engineering and the software systems that are engineered in the domain we work. There are many kinds of redundancy found in our systems. To avoid falling victim to platitudes that abound in the agile and No estimates domains, let's start with a framing assumption.
Redundancy provides resiliency to the system to withstand disruption within acceptable degradation parameters and to recover within an acceptable time and composite costs and risks.
In Taleb's (financial trading systems) domain resilience is desirable as it is in software intensive systems. Software systems that fly the airliner you ride on, manage the trains, process credit card transactions, control air traffic, manage the moving parts of your car. Any system where software is the dominate component for the proper functioning of the product or service also require resiliency.
But redundancy is not the only way to do this. And many times redundancy is very expensive, and creates less resiliency. - Fool Proof: Why Safety Can Be Dangerous and How Dangerous Makes Us Safe, Greg Ip, Little Brown, 2015
There are rules for assessing the resiliency that results from approaches beyond just redundancy. There are many other system design aspects that provide resiliency.
This notion of margin is absent from Agile development. And the result is when things go wrong, you're late, over budget and the product doesn't work. To have margin we must be able to estimate how much margin. Too much margin is a waste. Too little margin will not protect the system from disruption.
So when we hear a platitude like Layers of redundancy are the central risk management property of natural systems ask what kind of redundancy, what kind of fault handling and response processes. In fact ask first is that quote used as a platitude even applicable in the domain of interest? Or is it just a phrase picked up and repeated with little or no understanding of the principles, practices, or processes to which it CAN be applied.
 The Theory and Practice of Reliable System Design, Daniel Siewiorek and Robert Swarz
Phillip Armour has a classic article in CACM titled "Ten Unmyths of Project Estimation," Communications of the ACM (CACM), November 2002, Vol 45, No 11. Several of these Unmyths are applicable to the current #NoEstimates concept. Much of the misinformation about how estimating is the smell of dysfunction can be traced to these unmyths.
Mythology is not a lie ... it is metaphorical. It has been well said that mythology is the penultimate truth - Joseph Campbell, The Power of Myth
Using Campbell's quote, myths are not untrue. They are an essential truth, but wrapped in anecdotes that are not literally true. In our software development domain a myth is a truth that seems to be untrue. This is Armour's origin of the unmyth.
The unmyth is something that seems to be true but is actually false.
Let's look at the three core conjectures of the #Noestimates paradigm:
The Accuracy Myth
Estimates are not numeric values. they are probability distributions. If the Probability Distribution below represents the probability of the duration of a project, there is a finite minim - some time where the project cannot be completed in less time.
There is the highest probability, or the Most Likely duration for the project. This is the Mode of the distribution. There is a mid point in the distribution, the Median. This is the value between the highest and the lowest possible completion times. Then there is the Mean of the distribution. This is the average of all the possible completion times. And of course The Flaw of Averages is in effect for any decisions being made on this average value †
“It is moronic to predict without first establishing an error rate for a prediction and keeping track of one’s past record of accuracy” — Nassim Nicholas Taleb, Fooled By Randomness
If we want to answer the question What is the probability of completing ON OR BEFORE a specific date, we can look at the Cumulative Distribution Function (CDF) of the Probability Distribution Function (PDF). In the chart below the PDF has the earliest finish in mid-September 2014 and the latest finish early November 2014.
The 50% probability is 23 September 2014. In most of our work, we seek an 80% confidence level of completing ON OR BEFORE the need date.
The project then MUST have schedule, cost, and technical margin to protect that probabilistic date.
How much margin is another topic.
But projects without margin are late, over budget, and likely don't work on day one. Can't be complaining about poor project performance if you don't have margin, risk management, and a plan for managing both as well as the technical processes.
So what we need is not Accurate estimates, we need Useful estimates. The usefulness of the estimate is the degree to which it helps make optimal business decisions. The process of estimating is Buying Information. The Value of the estimates, like all value is determined by the cost to obtain that information. The value of the estimate of the opportunity cost, which is the different between the business decision made with the estimate and the business decision made without the estimate. ‡
Anyone suggesting that simple serial work streams can accurately forecast - an estimate of the completion time - MUST read Forecasting and Simulating Software Development Projects: Effective Modeling of Kanban & Scrum Projects using Monte Carlo Simulation, Troy Magennis.
In this book are the answers to all the questions those in the #NoEstimates camp say can't be answered.
The Accuracy Answer
But remember, making estimates is how you make business decisions with opportunity costs. Those opportunity costs are the basis of Microeconomics and Managerial Finance.
Cone of Uncertainty and Accuracy of Estimating
There is a popular myth that the Cone of Uncertainty prevents us from making accurate estimates. We now know we need useful estimates, but those are not prevented by in the cone of uncertainty. Here's the guidance we use on our Software Intensive Systems projects.
Finally in the estimate accuracy discussion comes the cost estimate. The chart below shows how cost is driven by the probabilistic elements of the project. Which brings us back to the fundamental principle that all project work is probabilistic. Modeling the cost, schedule, and probability of technical success is mandatory in any non-trivial project. By non-trivial I mean a de minimis project, one that if we're off by a lot it doesn't really matter to those paying.
The Commitment Unmyth
So now to the big bug a boo of #NoEstimates. Estimates are evil, because they are taken as commitments by management. They're taken as commitment by Bad Management, uninformed management., management that was asleep in the High School Probability and Statistics class, management that claims to have a Business degree, but never took the Business Statistics class.
So let's clear something up,
Commitment is how Business Works
Here's an example taken directly from ‡
Estimation is a technical activity of assembling technical information about a specific situation to create hypothetical scenarios that (we hope) support a business decision. Making a commitment based on these scenarios is a business function.
The Technical “Estimation” decisions include:
This kind of information allows us to calculate the amount of time we should allow to get there.
The Business “Commitment” and Risk decisions include:
These are the business consequences that determine how much risk we can afford to take.
Along with these of course is the risk associated with the uncertainty in the decisions. So estimating is also Risk Management and Risk Management is management in the presence of uncertainty. And the now familiar presentation from this blog.
Risk Management is how Adults manage projects - Tim Lister. Risk management is managing in the presence of uncertainty. All project work is probabilistic and creates uncertainty. Making decisions in the presence of uncertainty requires - mandates actually - making estimates (otherwise you're guess your pulling numbers from the rectal database). So if we're going to have an Adult conversation about managing in the presence of uncertainty, it's going to be around estimating. Making estimates. improving estimates, making estimates valuable to the decision makers.
Estimates are how business works - exploring for alternatives means willfully ignoring the needs of business. Proceed at your own risk
† This average notion is common in the No estimates community. Take all the past stories or story points and find the average value and use that for the future values. That is a serious error in statistical thinking, since without the variance being acceptable, that average can be wildly off form the actual future outcomes of the project
‡ Unmythology and the Science of Estimation, Corvus International, Inc., Chicago Software Process Improvement Network, C-Spin, October 23, 2013
As far as hypothesis are concerned, let no one expect anything certain from astronomy, which cannot furnish it, lest he accept as the truth ideas conceived for another purpose, and depart from this study a greater fool than when he entered it. Andreas Osiander's (editor) preface to De Revolutionbus, Copernicus, in To Explain the World: The Discovery of Modern Science, Steven Weinberg
In the realm of project, product, and business management we come across nearly endless ideas conjecturing to solve some problem or another.
Replace the word Astronomy with what ever word those conjecturing a solution will fix some unnamed problem.
From removing the smell of dysfunction, to increasing productivity by 10 times, to removing the need to have any governance frameworks, to making decisions in the presence of uncertainty without the need to know the impacts of those decisions.
In the absence of any hypothesis by which to test those conjectures, leaving a greater fool than when entering is the likely result. In the absence of a testable hypothesis, any conjecture is an unsubstantiated anecdotal opinion.
An anecdote is a sample of one from an unknown population
And that makes those conjectures doubly useless, because they can not only not be tested, they are likely applicable only the those making the conjectures.
If we are ever to discover new and innovative ways to increase the probability of success for our project work, we need to move far away from conjecture, anecdote, and untestable ideas and toward evidence based assessment of the problem, the proposed solutions and the evidence that the propsed correction will in fact result in improvement.
One Final Note
As a first year Grad student in Physics I learned a critical concept that is missing from much of the conversation around process improvement. When an idea is put forward in the science and engineering world, the very first thing is to do a literature search.
Without some way to assess the credibility of any idea, either through replication, assessment against a baseline (governance framework, accounting rules, regulations), the idea is just an opinion. And like Daniel Moynihan says:
Everyone is entitled to his own opinion, but not his own facts.
and of course my favorite
Again and again and again — what are the facts? Shun wishful thinking, ignore divine revelation, forget what "the stars foretell," avoid opinion, care not what the neighbors think, never mind the unguessable "verdict of history" — what are the facts, and to how many decimal places? You pilot always into an unknown future; facts are your single clue. Get the facts! - Robert Heinlein (1978)
There's a common notion in some agile circles the projects aren't the right vehicle for developing products. This is usually expressed by Agile Coaches. As a business manager, applying Agile to develop products as well as delivering Operational Services based on those products, projects are how we account for the expenditures of those outcomes, manage the resources and coordinate the needed resources to produce products as planned.
In our software product business, we use both a Product Manager and a Project Manager. These roles are separate and at the same time overlapping.
Product Managers focus on Markets. What features are the market segments demanding? What features Must Ship and what featues can we drop? What is the Sales impacts of any slipped dates?
Project Managers are inward focused to the resource allocation and management of the development teams. How can we get the work done to meet the market demand? When can we ship the product to maintain the sales forecast?
In very small companies and startups these roles are usually performed by the same person.
Once we move beyond the sole proprietor and his friends, separation of concerns takes over. These roles become distinct.
Products are about What and Why. Projects are about Who, How, When, and Where. From Rudyard Kipling's Six Trusted Friends)
Product Management focuses on the overall product vision - usually documented in a Product Roadmap, showing the release cycles of capabilities and features as a function of time. Project Management is about logistics, schedule, planning, staffing, and work management to produce products in accordance with the Road Map.
When agile says it's customer focused, this is true only when there is One customer for the Product, rather than a Market for the Product and that customer is on site. That'd not be a very robust product company if they had only one customer.
When we hear Products are not Projects, ask in what domain, business size, and value at risk is it possible not to separate these concerns between Products and Projects?