There's a never-ending opportunity to learn how to estimate in the presence of uncertainty. Here's some resources for informing that learning process.
Don't need estimates? - the project is de minimis
And the Principles, Practices, and Processes to Increase Probability of Success
There's a never-ending opportunity to learn how to estimate in the presence of uncertainty. Here's some resources for informing that learning process.
Don't need estimates? - the project is de minimis
The presentation "Quantifying the Impact of Agile Practices," Larry MacCherone at the RallyOn 2013 Conference, presents some results on estimating impacts. The chart below shows 4 estimating types, including No Estimates, the sample sizes for each type and the components that make up the estimating types.
The Software Development Performance Index (SDPI) scale on the left ranges - by eyeball measurement - from 46 to 55.
The Higher the number the better the performance of the process. The presentation speaks to the components of the index further.
But first another piece of information ...
Teams doing Full Scrum have 250% better Quality than teams doing No Estimating
But are these differences meaningful statistically?
Let's start with several reading assignments, before answering
Let's start with the numbers from the chart
Since the raw underlying data is not available, we can't do any p-Factor assessment from the population samples, but there is a simple question that can be asked.
Are there any statistical differences between the 4 SDPI's? If you look below at the quick and dirty assessment of the only data available, it looks like all 4 approaches are within a single digit variances of each other. Not that useful actually.
So the critical question still remains
How can you make a decision in the presence of uncertainty without estimating the impact of that decision?
The essence of mathematics is to not make simple things complicated, but to make complicated things simple - Stan Gudder
This notion that estimating is hard, estimates are a waste because they are always wrong, willfully ignores the basic mathematics of making decisions of in presence of uncertainty. The foundation of all decision-making, Probability and Statistics. Without this understanding there can be no credible information provided to the decision makers.
“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to,” said the Cat.
“I don’t much care where–” said Alice.
“Then it doesn’t matter which way you go,” said the Cat.
“–so long as I get SOMEWHERE,” Alice added as an explanation.
“Oh, you’re sure to do that,” said the Cat, “if you only walk long enough.”
(Alice’s Adventures in Wonderland, Chapter 6)
When we hear
The idea behind the #NoEstimates approach to software development isn't to eliminate estimates but, rather, to explore other ways to solve problems without specifically asking, 'How long will it take?'
We first need to ask by what principle of decision making in the presence of uncertainty, what kind of business project has no interest in how long will it take? The answer seems to be a de minimis project.
Because in the real world not Wonderland with Unicorns, those paying have some sense of where they want to go, how much they are willing to pay, how long they are willing to wait to get there, and what value will be produced by their investment. De Minimis projects have no concern for those answers.
And those spending that customers money appear not very interested in applying known solutions, instead are answering with the Cheshire Cat's words, since the No Estimates advocates appear to live in Wonderland.
For those interested in learning how to produce credible Estimates and make decisions in the presence of uncertainty, here's some starting places that have served me well:
There are many dozens of other books, and 100's of papers describing how to make estimates in the presence of uncertainty.
After you do some reading and you hear someone say, estimates are hard, estimates are guesses, estimates are always wrong. estimates are a waste, we can decide with estimates, you'll know they didn't read any of these, haven't a clue what they're talking about, and just making things up as the go. Just like the cat
Mike Cottmeyer posted
This is an interesting article that misses the point of what's going on. Companies have too much management because they are poorly architected and not organized around value producing assets. When you don't have sufficient encapsulation, you end up with too much orchestration. Management is not the problem. It's a symptom of the problem. Firing, or redeploying managers, without fixing the underlying organizational issues will fail cause you to faster and harder. The solution is to refactor the organizational architecture, increasing encapsulation, and then redeploy managers as you are able to reduce the need for them.
The root cause of this dysfunction is the lack of line of sight traceability for Why the firm is in business to how the firm fulfills the strategy needed to deliver on that WHY.
Here's one way that has been shown over the years to connect the dots between Why and How and critically importantly, how the projectize the work efforts needed to implement the strategy.
While this quote appears inverted in our self-centered world, the expert has eliminated the nonsensical, naive, amateur, simple minded options and narrowed the choices based principles, practices, and processes of her profession. It is trivially easy to assert I'm exploring new ways of doing things when there is no bounding governance principles to guide your exploration. If the exploration takes place in a de minimis world, the options are boundless.
The SAFe Reference Guide is out.
The website has most everything you need, but it is hard to use and when printed in simplified mode doesn't contain the embedded pictures.
In our local Agile Meetup at Rally, Charles Bradley went through the 3 major methods for agile at scale - SAFe, LeSS, and Nexus
This book is a very useful even if you're not using SAFe, there is invaluable content in the book, including many pages on estimating.
So get the book SAFe , it's the latest in how to manage software development At Scale. LeSS has similar advice, Nexus not so more, you have to attend their classes.
Thanks to Sean Craig's Live Sketch Note for capturing concepts directly from Woody Zuill's talk. This is a good starting point for answering the mail on the notion that decisions can be made in the presence of uncertainty without estimating the impact of those decisions.
Let's look at these Notes and plausible responses to the conjectures. But first, let's look at the motivation for a blog post like this one from Bono's admonition ...
There’s nothing more dangerous than an idea – or a person – that’s almost right. – Bono
I attended our Agile Meetup last night, where the speaker walked through the three current Agile at Scale development methods, all based on Scrum - SAFe, LeSS, and Nexus. He had an interesting statement.
Each of the authors have a different, sometimes slight, sometimes major, approach to producing software using their methods. But each author (or currator of the mehod) has extensive experience testing that method in the field against a broad range of domain and environments. 1,000 to 10's of 1,000's. There is no sure basis of credibilityy for the No Estimates conjecture that decision can be made in the presence of uncertainty without first estimating the impact of the decision.
When we hear a suggestion about a process that inverts the logic of normal business activities based on a governance framework - say Microeconomics of Software Development, we need to ask who benefits? How would that suggestion be tangibly beneficial to the recipient that is now inverted?
Estimates are for the business, why would the business no longer want an estimate of cost, schedule, or technical performance for the provided capabilities they are paying to have developed?
Turns out of course, it's not the business that has lost the need to estimate, it's those spending the money provided by the business that are suggesting estimates are not needed. This conjecture started long ago when an original post, from his chief coder position at the lawn sprinkler company. (I have those sprinkler heads on my lawn, and they're pretty nice). But his conjecture starts with estimates are a waste, not saying for whom they are a waste for.
Anyone in the finance department, paying his salary, paying the developers has a fiduciary need to know what something will cost when it is done. And since ALL and I Mean ALL software development operates in the presence of uncertainty, the only when to have knowledge of that cost, schedule, and technical content is to make estimates. The further conjecture that estimates are hard, can't be made, and are always wrong, simply affirms that those making those claims don't know how to estimate. That is clear, but that has nothing to do with the principles of estimating in the presence of uncertainty.
I'm no longer a developer, having left development in the early 80's after a career of writing FORTRAN 77 code for missile defense systems that is still in place today. But that has nothing to do with the principles, practices, and processes of writing code using current languages and tools. If you find it hard, but go find someone who doesn't find it hard.
So let's look at what's being said here about managing software development in the presence of uncertainty using other people's money
Transformation comes from pursuing profound questions than seeking practical answers
It's not your money
Here's an extract from a Deloitte handbook about business decision making
Effective governance and decision making establishes a framework for transformation and can improve the odds of solidifying change and realizing the benefits of transformation.
Control and Certainty about What?
If you're not managing risk, if you're not making estimates about these risk, you're not managing the project as an adult
Control our Urge to Control Things
If the types of decisions we make requires estimates, then are there better types of decisions?
If you accept that project have uncertainty, then making decisions in the presence of this uncertainty requires you estimate the impact of that decisions. #NoEstimates has yet to state how, in the presence of uncertainty, that a decision can be made with NO Estimates. Only that they are exploring for ways to to this. Been exploring for going on 4 years now, seems to have found nothing. Which would support the notion that the microeconomics of decision making is still in place and any decisions in the presence of uncertainty requires making an estimate.
The Fear of Losing Control is a Big Barrier to Change. Control is Illusion. Fear is Real
Just another platitude without any basis in principles of closed loop business and technical management
The cycle of non-improvement
There is no evidence that No estimates improves anything for those paying for the software development. This is a consistent issue with the conjecture that No Estimates fixes problems, without idnetifying the root cause of the problem, testing the hypothesis that No Estimates fixies the problem, and that this fix can be syndicated beyound the personal ancedote of the orginal conjetcure
What is an Estimate?
This question has a simple answer. It is not a Guess. It is not a promise. This question and the answers used by No Estimates advocates is used to manipulate the situation. Bad Managers use estimates as promises. Those with no experience, skill, or knowledge of estimating consider estimates as Guess's. Both are wrong, both are behaving as if they are in a Dilbert carton, following the script of a dysfunctional set of characters.
There endless numbers of books, papers, tools, classes on how to estimate software projects. These have all been provided in this blog before and can be found again. But here's a starting place - Estimating Software Intensive Systems. Read this and you'll have the start of all you need to break the cycle of uniformed, unsubstantiated states about what is an estimate, how are they made, and how can they be used to make decisions in the presence of uncertainty
Most of the things that matter can't be measured
Let's establish a baseline here. We're writing software for money, usually other people's money. In this context - a business context, Lord Kelvin has good advice for us. Writing software for money is not about the self-actualization of the individuals on the team. That may an outcome, but those paying ae not paying to improve the staff's psychological wellbeing.
When you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind. …It may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the stage of science. - Lord Kelvin (1824—1907)
So what is the quest of the advocates of No Estimates? It's not actually clear.
I can only conjecture that the purpose of Not Estimating is to return control of the work processes to those doing the work by removing the decision making processes from those owning the money. It's the anarchistic approach to software development.
I, the one providing the labor in exchange for payment, am the one who will be telling you, the paying, when I'll be done, how much it will cost when I'm done, and what you'll be getting for that time and money. I am the one in charge of the effort to spend your money.
Thought interferes with the probability of events, and in the long run therefore, with entropy - David L. Waston (1930)
All project work is probabilistic, driven by underlying uncertainties, both reducible and irreducible. Willing an outcome in the presence of these uncertainties does not make is so. The only useful process to manage in the presence of uncertainty is to estimate the outcome of any decisions made by the participants of the processes by which the project operates
The mantra of Agile is Trust the team. In some domains, that is an admirable goal. In other domains it's a naïve path to disaster. I work in the latter domain, on mission critical, sometimes national asset programs, but always mission critical - can't fail, must work, must provide proper information when called upon to do so.
When we are called on to perform a Root Cause Analysis of why the system failed to do what it was suppose to be, we find the same thing that Dr. Bill Corcoran suggest is found on all root cause analysis processes.
An inescapable fact is that the competent investigation of every harmful event reveals that the causation of the harm includes the mistaken/ naïve/ unwarranted/ gullible/ imprudent trust and confidence in one or more erroneous/ untrustworthy theories, assumptions, standards, devices, procedures, processes, programs, people, institutions, agencies, contractors, and/or conditions. The functional alternatives include monitoring, curiosity, skepticism, and the “questioning attitude.”
Here's some quotes to apply when you hear agile means trust of the team, why are you questioning our processes, our methods, our organizational models. (Thanks to Dr. Corcorn's news feed today):
So saying it again for clarity - you can't make a decsion in the presence of uncertaty without estimating the outcome of that decision. When you do, be preared to conduct a Root Cause Analysis of why your project went in the ditch. Trust is necessary but far from sufficient when spending other people's money on non-trivial development efforts.
Much of the discussion around project management processes, especially around agile, and most especially around the misconceptions of Estimating as espoused by the #NoEstimates advocates, starts with the misuse of reductive reasons based on single factor analysis.
Here's how it goes.
When we see these two concepts used together we get things like as the cartoon of the Reductionist view of a single concept - If you did it this way, you'd be from 3X to 10X faster.
So here's the problem and the solution. Complex systems are part of the solution to all complex problems. Anyone claim complex problems can be solved with simple systems, needs to have a testable working system, in that complex problem space. I work in a complex problem space - literally space flight, aircraft flight, the ground systems that enable those systems to Fly. As well as biopharma, electric utilities (nuclear and conventional fired), complex enterprise IT systems (dozens to many dozens of interacting systems).
When you hear a simple and many time simple-minded solution to a complex problem - Applying No Estimates will remove the dysfunction on software projects (this is the ontological inverse of the statement estimates are the smell of dysfunction). We can be reminded by H L Menken's quote:
For every complex problem there is an answer that is clear, simple, and wrong.
Crede Sed Proba
Trust is a core attribute of Agile Software Development. But any non-trivial business venture will also what to verify that to outcomes of the work effort are going to meet the needs of those paying, at the time those capabilities are needed, for the expected cost.
When you hear agile is all about trust, make sure you have a appropriate verification process in place as well. This is called governance where we work. It's risk management and Risk Management is ow Adults Management Projects (Tim Lister).
Many want us to trust but any credible governance process requires a verification process. With that verification, that trust cannot be trusted.
I forgot something important that you must remember until you go six feet under…. There are only two kinds of people in the whole wide world, grifters and suckers…. [With suckers,] let their stupid brains stay asleep in their chump world. Keep your own brain honed to razor sharpness in the secret world of con. –Iceberg Slim
I came across a couple of interesting tweets over the weekend
(Scrum is) more defined rather than empirical process. Agile in name only. scrum, not agile. not as it was prior to scrum. lost art.
Scrum is prescriptive and defined rather than empirical. it seeks to be guidance for work it's too far removed from to understand.
This took me back since Scrum is derived from an empirical closed loop control system developed in the USAF by Col. John Boyd. There's a recent paper "What Lessons Can the Agile Community Learn from A Maverick Fighter Pilot?" by Steve Adolph. I was puzzled by the tweet, asked why that was the opinion, but didn't get a response. As a start, here's a resource for Col. Boyds papers.
No need to explain why OODA is the basis of Scrum, here's a much better post - OODA: The Mindset of Scrum.
The OODA concept is very simple:
Boyd's presentation "A Discourse on Winning and Losing Introducing core ideas & themes Of Boyd’s ‘Theory of intellectual evolution and growth,"was presented at “Patterns of Conflict” at the U.S. Marine Amphibious Warfare School in June 1980. Here's some more Boyd from the Boyd Library. A good book on Boyd is Boyd: The Fighter Pilot Who Changed the Art of War
In our Software Intensive System of Systems domain, this is the definition of Agile we use from the now-Secretary of Defense Dr. Carter
So when we hear anything about the current state of Agile or even that Scrum is NOT empirical and needs to be replaced, ask what new process is being suggested that will meet Dr. Carter's definition?
…the OODA loop sketch and related insights represent an evolving, open-ended, far from equilibrium process of self-organization, emergence and natural selection. This is a good starting point for not only Scrum and other agile methods, that can be used to test the validity of any method suggested to replace or augment the current agile software development processes.
Assessing the outcomes of the feedforward and feedback loops is the basis of all Closed Loop control system. (See link below on Closed Loop Control). Making decisions in the presence of this emerging uncertainty requires making estimates of the impacts or outcomes of any decision. There's simply no way out of this principle when making decisions in an emerging domain with uncertainty (reducible or irreducible). Anyone suggesting otherwise either hasn't the knowledge of closed loop control systems or is try to sell you something that violates the principles of closed loop control, microeconomics of decision making, or managerial finance. Don't buy it. Learn to manage in the presence of uncertainty using principles not persoanl ancedotes.
All project work is uncertain. This uncertainty comes in two forms - Aleatory and Epistemic. Aleatory Uncertainty is irreducible, Epistemic uncertainty is reducible. More can be found on this in Both Aleatory and Epistemic Uncertainty Create Risk.
But now what to do about it. Here's a collection of papers that has served me well in defining the processes needed to make decisions in the presence of uncertainty when managing projects using other people's money:
So when you hear about making decisions in the presence of uncertainty without estimating the outcomes of those decisions, you'll have a basis of knowledge to realize that is completely bogus, a violation of the principles of microeconomics, and a violation of the principles of managerial finance.
A nice conversation on twitter about estimates on software brought up the topic of estimates as commitments. The #NoEstimates advocates see estimates as making commitments. Yes, commitments are made when we estimate. But that commitment is not a promise. A Promise is a guarantee, with 100% confidence it will be net. Our wedding vows are a promise to each other. A commitment must have a probabilistic confidence. Just for the reference,
A commitment must have a probabilistic confidence. Just for the reference, a commitment is the state or quality of being dedicated to a cause, activity, etc.
"the company's commitment to quality or an engagement or obligation that restricts freedom of action. Take for example Boulder's commitment for 100% recycling of consumer products inside the city limits. Is that commitment are guarantee every single consumer waste product collected at our curb will be recycled? That is a tall order and not likely to ever be the case.
I have an 80% confidence (an estimate) I can deliver what you need on or before September 15 (an estimate), at or below $15,000.oo (an estimate) with a 15% error band (an estimate).
The misuse of that commitment is a real problem in all domains. But that's got NOTHING to do with the need for the estimate and EVERYTHING to do with bad management. No Estimating isn't going to make the need for estimates go away or fix bad management, no matter how many Dilbert cartoons are used to illustrate that.
In any non-trivial domain, there are larger business processes in play that is looking for value in exchange for money. When business people think about that exchange, they think about some agreement between those paying for the value and those providing the value . This agreement can be formal or it can be informal. But that agreement is always in place.
In our domain of software intensive system of systems, we have formal agreements at the top of the engagement and less formal agreements lower down based on agile software development processes. Here's how it is done where we work.
So when you hear Estimates can't be done, estimates are a waste, estimates are the smell of dysfunction, estimates are evil, this may actually be true for those working on de minimis projects. Projects where those paying for the work have NO concern about how much it will cost to produce the value, NO concern when that value will be available for use, NO concern if the capabilities that produce that value actually work and actually do produce the value for the needed cost on the needed date.