Get the truth & print it - John S. Knight.
Get the truth & print it - John S. Knight.
All decision making starts with a domain and context. Without domain and context we have no way to understand the environment in which we operate and the well defined objectives needed to assess our decision making process. We have no way to address the natural uncertainty of our project work. We can not make a decision in the absence of alternatives.
We can only address uncertainties - both reducible and irreducible - by understanding the probabilistic outcomes created by the underlying statistical processes. Assessing the resulting outcomes against our goals is the basis of all probabilistic estimating and forecasting activities of any credible decision making processes.
It is the probabilistic and statistical information that informs our decisions. To ignore this means we are driving in the dark with our head lights off.
For as soon as the distribution of labour comes into being, each man has a particular, exclusive sphere of activity, which is forced upon him and from which he cannot escape. He is a hunter, a fisherman, a herdsman, or a critical critic, and must remain so if he does not want to lose his means of livelihood.
(from Marx’s German Ideology)
Marx goes on to say...
[I]n communist society, where nobody has one exclusive sphere of activity but each can become accomplished in any branch he wishes, society regulates the general production and thusmakes it possible for me to do one thing today and another tomorrow, to hunt in the morning, fish in the afternoon, rear cattle in the evening, criticise after dinner, just as I have a mind, without ever becoming hunter, fisherman, herdsman or critic. This fixation of social activity, this consolidation of what we ourselves produce into an objective power above us, growing out of our control, thwarting our expectations, bringing to naught our calculations, is one of the chief factors in historical development up till now.
With this in mind, our ability to influence the outcome of any endeavor, as specilist in our narrow field, that involves spending other people's money is limited. Without understanding this basic tenet of economics, we have little to say unless we can show what something will cost in exchange for its value. Those we're asking to fund our efforts in exchange for that value will have a difficult time listening. This may be considered undesirable by some, but it is a simple fact of life in our modern world of capitalism.
Statistics rule our lives. Traffic patterns for our commute to work, probabilistics weather anywhere you live. Your 401(k) earnigs reports that comes every month. For projects all the elements are statistically driven. The productivity of the engineers or developers. The partial testing coverage of sofwtare, hardware, integrated systems. The performance of any product. The efficacy of the product when it is in use. The forecasted cost to develop the products or provide the services. The total duration to produce the needed outcomes of the project. It's all statistics all the time.
To work on or manage projects with any hope of success, we need to know about statistics and the probabilistic outcomes that result. Let's start with a simple picture. All projects - at least ones beyond the simple single team working off a single set of work activities, looks like this. There are a collection of activities, connected with each other in a network. The durations are random numbers. The completed outcomes are randomly compliant with the needed quality, functionality, or capability. Even with high coverage testing, when we assemble parts into a whole new things happen. Things we didn't think would happen. The new system has a probability of working the first time. And that probability is not 100%. (We'll get to the quote on this idea).
Many in the agile community - especially the Kanban community - assume work can be divided into same sized chunks and the people doing the work can process these chunks in same sized time frames, but where I come from that would be considered naive at best. Steady arrivals, steady processes, steady exiting of finished products doesn't even happen on the Toyota assembly line. Let alone in the product development business.
So Now For The Quote
One of the great myths of science is that it is rigorous. More to the point is that the scientific method chops up any problem into small pieces that can be comprehended by the human mind, and an above-average mind at that. Some of the pieces are more rigorous than others, and the reassembly of the whole always requires letting something fall through the cracks.
- The Art of Modeling Dynamic Systems, Foster Morrison, Wiley, 1991.
So now let's think about the parts that have been decomposed and reassembled. Are they correlated in some way? Does change in one actually make a change in the other - causation? How can we tell? Start by assuming there is no causation between the parts, even though there is correlation.
So let's look at one more thing about the probability and statistics of projects. The picture below is critical to sorting out many of the misconceptions around how projects behave. Statistical processes drive the behaviour of projects. All the elements of a project are subject to these stochastic processes.
If we know something about a process - maybe by observing it - we can state things about the statistical behaviour of the process. Once we know about the underlying processes, we can make probablistic forecasts of future behaviours. For both the statistical and probabilistic measurement we also need to know the variance on those numbers. This allows us to make estimates of past, current, and future behaviours and make forecasts of future behaviors, both with uncertainty bounds.
The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data - John Tukey
This is the John Tukey of the Cooley-Tukey Fast Fourier Transform algorithm. I was introduced to the concept of a mini-computer when I wrote a FFT for a PDP-8M with 4K of 12 bit memory to implement a spectrum analyzer of an accelerator experience, circa 1976. This was later moved to a PDP-15, and then to the DEC-10, where that same algorithm was used for radar signal processing in search of SAM-6 signatures recorded on 9-track tape brought back for places unknown. That software, eventually running on a PDP-11 installed inside a NC-135 (renamed EC-135) was replaced by a Watkins Johnson receiver and we were out of the software business, at least for converting radar analog signals to digital and looking for signatures.
All of this is to say programming computers is not the same today as it was then, except in the same domain - Defense Systems, now with massive software content.
But more importantly all of this is to say, when you hear someone start out with a hypothesis that is not based on evidence, but rather opion, it's going to be a long hard road to the conclusion.
The urge to save humanity is almost always a false front for the urge to rule. - H. L. Mencken
Anderson don't talk out loud, you lower the IQ of the entire street
Sherlock Holmes to officer Anderson while working the case of Pink in Death a serial killer story
There are several quotes that resonant with me on a near daily basis in the project management domain. All of these come from direct projects or programs.
So when ever you hear a pithy platitude, see if you can put it into the following structure:
Never calculate without first knowing the answer - John Archibald Wheeler
When we work using other peoples money, we have an obligation to know what the all in cost is, or when we plan to be done, and if we have confidence that the technical outcome will be acceptable to those who hve paid for the outcomes of our project.
In order to answer these questions, we need to make estimates for cost, schedule, and technical performance before the project is complete, long before it is complete. These estimates are probabilistic forecasts of future outcomes based on past performance or out best assessment of what could be the outcomes.
We owe it to those providing the money for our. No business can operate for long without knowing what it's future obligations are for cash flow and payments to staff and vendors. To ignore this fundamental aspect of business is to ignore the creation of value for those providing the funding for that creation.
The ability to credibly estimate is a core competency of all engineeting and product development activities.
Every systematic development of any subject ought to begin with a definition, so that everyone may understand what the discussion is about.
Marcus Tullius Cicero (196BC ‒ 16BC), De Officiis, Book 1, Moral Goodness
When an idea is put forward, it is difficult to engage in a conversation without knowing the domain and context in which the conversation is taking place.
We don't try to predict the future in order to contol it, we try to predict the future to prepare a proper response to the possible emerging events.
Those who are not engaged in estimating the possible outcomes of future events and preparing to manage in the presence of this uncertainty will be managed by those events. Whether we are preparing for them or not, they are coming
Before accepting the notion that estimating, forecasting, and preparation are not needed, ask what business can we work in where the future cost, schedule, and technical performance of our work is of no interest to those providing the money for our efforts?
Forecasting involves making projections about future performance on the basis of historical and current data - from "Time series Forecasting using Holt-Winters Exponential Smoothing,"
Without current data and past performance data, no forecast of future data can be credible. Current data alone is of no use with knowing what this data should be representing at a performance level. This is the core paradigm of closed loop control. Simply taking samples and assuming these samples represent information to the decision makers is ignoring the underlying system that generated those samples. Without the reference point (goal) the feedback has no way of generating the measurement error and therefore no way of providing input to the system to produce the desired output.
All sampled data from the system, needs to be placed in the context of how those numbers came about, how they were generated, and what possible ranges those numbers could have taken on - not just the current set of numbers.
For example writing down your running times for the past 6 weeks over a trail is useful, but without knowing your capacity for how fast you could run, they are of little use in improving your times over the same or similar courses. The only way to know is to go out an try to run faster.
This is fine. But if you earn your living by running, as a few do in our neighborhood, the goal setting process is missing. Knowing how fast you need to run to win the Bolder Boulder in a Professional category is set before training starts in the winter. It's a training to goal. Amateurs don't usually do this in the Citizen category. But every A runner I know, has a target goal for the 10K before the season starts, a plan to reach that target goal by the May 30th race.
As rank amateurs we are lucking just to finish. But when we see Uta Pipping on the open space trail, she is always looking at her watch, she runs to a goal, we run for a different reason. Same for the pro cyclist on the road, always peddling to a pace needed to maintain their goal.
As Yogu says If you don't know where you're going, you'll wind up somewhere else
Ideology is a substitute for thought - Pete Hamill
Put you CFO hat on, all business is driven by cost. If not they won't be in business long
COO of a major grocery store chain to the suggestion that not estimating the cost of a development project is better for the business.
The notion that we can make decisions about the future without somehow knowing the possible behaviours we will encounter in this future is naive at best and is doomed to fail at worse. Here's some advice from others on this topic.
What Does All This Tell Us?
When we hear words about our inability to predict the future, make a forecast of something in the future, or make a decision in the presence of uncertainty, we must first assume that person has failed to understand the foundtaions of probability and statistics and how to apply these principles to making decisions. Next we might assume that person doesn't actually want to know how to do this.
If the former is true, start reading Books for Cost and Schedule Forecasting
When schemes are laid in advance, it is surprising how often the circumstances fit in with them — Sir William Osler
The notion of emergent anything without some understanding of what done looks likes lays the groundwork for spending without a goal. Defining the Needed Capabilities of the project upfront,sets the bounds for recognizing the attributes of done when it arrives.
Here's how to elicit, identify, assemble, and assess the needed capabilities before any project work starts.
Notice a few critical things
Don't know how much it will cost? Don't know how much you are willing to spend? Don't start!
Again and again and again — what are the facts? Shun wishful thinking, ignore divine revelation, forget what "the stars foretell," avoid opinion, care not what the neighbors think, never mind the unguessable "verdict of history" — what are the facts, and to how many decimal places?
You pilot always into an unknown future; facts are your single clue. Get the facts! - Robert Heinlein (1978)
When you hear about anything about anything involving changing what you are doing know, think of Heinlein.
The actual outcomes for the athletes that Just Do It and run like the wind, is nearly unlimited dedication to their sport. Unlimited coaching, training, and actual competing. Not to mention natural inate talent. I say this from having watched and supported our children from grade school through college sports. Having participated in sports as well. All of our family did well, not great, but well. Our son is a B Mountain Biker Racer at the collegite level. He has pros on his team (this is why collegiate mountain biking is not a NCAA sport). His coach was a world champion. His other coach was a New Zealand national champion. Our daughter was a swimmer in high school. She swam in college on the club team, where state champions went because her college did not have a NCAA swim team - Football dominated.
So when some one suggests aw just do it ask did you compete at the varsity level in High School or College? If not, you probably don't know what that phrase really means. If you did, you may have forgotten the endless hours in the pool, or on the track, or getting rained on for hours out on the course - just Trying to Do It.
The notion of #NoEstimates ignores a critical concept (one of many)...
Promises, schedules, and estimates are important instruments in a well-run business. You must make promises - don't lean on the often-used phrase: "I can't estimate it because it depends on many uncertain factors.
This quote is from William Swanson's Unwritten Rules of Management (Swanson is the former CEO of Raytheon). His book is a near copy of The Unwritten Laws of Engineering. W. J. King, 1944. Borrwoing those ideas without attribution got him in BIG trouble!
But in the SW Dev world, developers don't get to decide when spending other peoples money. The customer does. If they're spending your own money, no one really cares what they do with it.
If there is a suggestion from the #NE advocates on how to do estimates better, it should have a measurable business value. By measureable it doesn't mean we're exploring using the customers money or I'm always looking for dysfuntion - never saying what dysfuntions have been found.
There is always room for improvement in any work process. That is a tautology. But those improvement suggestions need to have a disciplined approach. They need to have an ROI. They need to have some Measure of Effectiveness - "customer satisfaction" is a Measure of Effectiveness.
Before proceeding with any discussion around estimating anything, I'd suggest a read of How To Think About Statiistics, 6th Edition, John L. Phillips, Freeman, 2013. Here the reader will learn that many of the statements made on #NoEstimates are not only wrong and uninformed, they are not even logical.
Things like you can't predict on software projects, is of course complete nonsense. It's done everyday. Things like we don't need to know the cost, just as much nonsense, since ROI has cost in the denominator, so you get a divide by zero error when doing the ROI calculation. Etc. Etc. Etc.
Time to call BS on the way #NoEstimates is being portrayed by some. Others have good ideas. Time box scheduling, backlog management, small - same sized - chunks of work with steady productivity, fed by the same small chunks from the backlog. This is logical. Doesn't address at all the need to know how much to budget for that work though by the people giving you money to produce the value, but that's another conversation.
A Fool and His Money Are Soon Parted
- Thomas Tusser in Five Hundreth Pointes of Good Husbandrie, 1573
Say a customer wants you to develop a piece of software, but you want to convince the customer he doesn't need to have an estimate from you about what it is going to cost. The #NoEstimates movement asks if there is a better way.
One way is the flow down a budget to the development team. A small budget, for a small amount of work. Perform that work to calibrate the efficacy of the development team in producing value for the customer. Keep doing this in drips of funding and getting drips of value back.
Sounds like a logical approach. Low risk. But how much total money is it going to take to get all the value for the customer? The all in value for the all in cost? #NE doesn't say how to do this, other than keep spending until the customer says stop. No estimates needed here, simple process, nothing revolutionary, the developers are just a sunk labor cost managed externally by the customer. Customer does all the work of managing the project.
But some where, some how, some one has to have an upper bound on that spend plan. No credible business can operate in the absence of a budget. You can't operate your house without a budget - or at least you shouldn't. If the business is small, the business owner may have that budget in her head. If the business is large, that budget is in a financial planning document. That document is reported in some report to the Board of Directors. It's the obligated funds needed to operate the company. It's called the operating budget.
So how is that budgeted developed. Small business person probably just hs a gut feel. I'm willing to spend $X to find out what you can do for me. Here's $Y get started and let's see what happens.
A large business might do that. It's internal Research and Development (IRAD). Money to try things out, but the budget for that try things out has to come from somewhere and that somewhere has to have an approval cycle, and that cycle needs to know How Much.
For a large project inside a small company and certainly inside a large company, the budget for software development likely has an approval cycle. Again if there is no approval cycle, then the value at risk is probably small enough that no one really cares what you do with the money. But let's pretend there are people who care. Like the CFO, the CIO, the COO, the CEO. They need to know the value at risk. Their exposure to not receiving the promised value in exchange for the money.
So now we're in a loop. #NE says that are better ways. One better way is the drip funding. But that just pushes the estimating process back upstream. Someone has to estimate how much is enough. For larger project, how much is enough needs to be know to some degree of confidence, before those approving the money, will approve the money. It's as simple as that. No estimate - from someone - no money. That's how the balance sheet works.
Now, #NE has provided - through one presentation - how drip might works. But someone has to do estimates. Who? Not the developers obviously.
So maybe #NE is well suited for 5 week projects, that have clear and concise outcomes, low exposure to cost risk, and a customer who well accept that you'll be working in the project until the 5 weeks are up, doing your very best to maximize the value for that 5 weeks. Yep, that sounds like a nice job to me. Good work if you can get it. And maybe that's the point. It's all about small chunked projects.
But just a question. Can I deploy a $150M claims processing system in this way. How about a $500M embedded flight avionics system? Or a $200M wireless umbrella replacement of a wired phone system for 5,000 users? Good question, still waiting to hear EXACTLY how to do this.
And by the way, that Yoda management style of you discover on your own, I'm just exploring ideas here sounds pretty lame in front of the CIO of a $7B health insurance firm.