Customers buy capabilities to accomplish a business goal or successfully complete a mission. Deliverables are the foundation of the ability to provide this capability. Here's how to manage a project focused on Deliverables.
Principles, Practices, and Processes to Increase Probability of Success
Customers buy capabilities to accomplish a business goal or successfully complete a mission. Deliverables are the foundation of the ability to provide this capability. Here's how to manage a project focused on Deliverables.
It's popular in the agile world and even more popular in the No Estimates paradigm to use the term empirical data as a substitute for estimating future outcomes. And my favorite meme that further confuses the conversation.
Probabilistic forecasting will outperform estimation every time
This of course is "It is not only not right, it is not even wrong."† Probabilistic forecasting IS estimating. Estimating is about the past, present, and future. Forecasting is estimating about the future. I'll save the embarrassment by not saying the name of the #NoEstimates person posting this.
First a definition. Empirical is originating in or based on observation or experience. But we all should know that that data needs to properly represent two sides of the problem, the past and the future.
Let's look at some flawed logic in this empirical data paradigm:
One more technical detail.
So Now Some Issues Of Using Just Empirical Data
The future is emergent in most development work. If it's a production line, and software development is not production, then past performance is a good indicator of future performance. So let's ask some questions before using this past empirical data:
Don't have the answers to these and working a non-trivial project? Our empirical data is not worth much because it doesn't actually represent the future. Might as well guess and stop using the term empirical as a substitute for you know know much of anything about the future.
With those answers we can build a credible model of the future, with interdependencies between the work, probability distribution functions for the statistical behaviors of the work elements and start asking the Killer question:
What's the probability of completing on or before the need date for the work we are producing?
This answer only tells us the probability, not the exact date. So here's the most important point.
All decisions about future outcomes in the presence of uncertainty need estimates that are placed in the model and assessed for their applicability.
This is called Closed Loop Statistical Process Control. And that's how non-trivial projects are managed. Low value at risk, no one cares if you estimate or not.
† Which by the way is the situation with most of #NoEstimates conjectures, starting with the willful ignorance of the MicroEconomics of decision making as an opportunity cost process. What will is cost us if we decided by multiple choices in the presence of an uncertain future? That questions can't be answered without making an estimate of that opportunity cost.
In the agile community it is popular to use the terms complex, complexity, complicated many times interchangeably and and many times wrongly. These terms are many times overloaded with an agenda used to push a process or even a method.
First some definitions
One more item we need is the types of Complexity
And Now To The Point
When we hear complex, complexity, complex systems, complex adaptive system, pause to ask what kind of complex are you talking about. What Type of complex system. In what system are you applying the term complex. Have you classified that system in a way that actually matches a real system.
It is common use the terms complex, complicated, and complexity are interchanged. And software development is classified or mis-classified as one or the both or all three. It is also common to toss around these terms with not actual understanding of their meaning or application.
We need to move beyond buzz words. Words like Systems Thinking. Building software is part of a system. There are interacting parts that when assembled, produce an outcome. Hopefully a desired outcome. In the case of software the interacting parts are more than just the parts. Software has emergent properties. A Type 4 system, built from Type 1, 2, and 3 systems. With changes in time and uncertainty, modeling these systems requires stochastic processes. These processes depend on estimating behaviors as a starting point.
The understanding that software development is an uncertain process (stochastic) is well known, starting in the 1980's  with COCOMO. Later models, like Cone of Uncertainty made it clear that these uncertainties, themselves, evolve with time. The current predictive models based on stochastic processes include Monte Carlo Simulation of networks of activities, Real Options, and Bayesian Networks. Each is directly applicable to modeling software development projects.
 Software Engineering Economics, Barry Boehm, Prentice-Hall, 1981.
When we hear about project planning and scheduling, we think about tasks being planned, organized in the proper order, durations assigned to the work, resources committed to perform the work.
This is all well and good, but without a risk adjusted view of the planned work, it's going to be disappointing at best. There are some key root causes of most project failure. Let's start here.
Each of these has been shown, through research on failed programs, to contribute to cost and schedule impacts. Unrealistic expectations of the project's deliverables, Technical issues, Naive Cost and Schedule estimating, and less than acceptable risk mitigation planning.
Project Mangement in Six Steps
Here's how to address the cost and schedule estimating
Develop a schedule. What ever your feelings are for Gantt's, sticky note, or any handwaving processes you've learned to use, you need a sequence of the work, the dependencies, the planned durations. Without something like that , you have no idea what work is needed to complete the project. Here's a straightforward Master Schedule for some flight avionics on a small vehicle. All software, has to complete as planned, otherwise the users can't do their job as planned. And since they're the ones paying for our work, they have an expectation of us showing up, near oru budget, with the needed capabilities. Not the Minimum, the NEEDED
Using a Monte Carlo tool (RiskyProject), here is run for the probabilities of cost, duration and completion dates. All project work is probabilistic, any notion that a deterministic plan can be successful is going to result in disappointment.
We usually call our planning sessions done when we can get the risk adjusted Integrated Master Schedule to show the completion date of on or before the need date to an 80% confidence level.
With a resource loaded schedule - or some external time phased cost model - we can now show the probability of completing on or before the need date, and at of below the planned budget. The chart below informs everyone what the chances are for success for the cost and schedule aspects of the project. Technically it has to work. The customer gets to say that. The Fit for Use and Fit for Purpose measures if we're in an IT Governance paradigm. The Measures of Effectiveness and Measures of Performance if we're in other paradigms. Those measures can be modeled as well, but I'm just focusing on cost and schedule here.
With this information we can produce the needed margin and management reserve to protect the delivery date and budget. Showing up late and over budget for a product that works is usually not good business.
Do You Need all This?
What's the Value At Risk?
Don't know the value at risk? Don't have a clear vision of what done looks like in units of measure meaningful to the decision makers? You've got bigger problems. This approach won't help?
If you work in a domain, as I do, the need to answer the question when will we providing that needed capability to produce the planned value for the planned amount of money, then estimating is going to be part of answering those questions.
If you work where those paying for the work have little or no interest in asking these questions or knowing these answers, or have confidence you'll not overrun the cost, schedule, and deliver the needed capabilities as planned, then maybe estimates are needed.
Be interesting to hear from those actually paying for those outcomes to see what they need to make decisions in the presence of uncertainty.
Here's some more guidance for getting started with estimating software efforts.
And some tools to help out
So you see a trend here? There is nearly unlimited resources on how to estimate software development projects, how to manage in the presence of uncertainty, how to elicit requirements, how the plan and schedule software projects.
So if you hear we're bad at estimates, that's likely the actual experience for the person making that statement, because the person saying that hasn't yet learned how to estimate. Or when we hear estimates are a waste, it's likely it's not their money, so to them estimates take away from some other activity they see as more important. Why do they care of the project overruns it's budget, is late, or doesn't produce the needed value? Or my favorite estimates are the smell of dysfunction, when there is no domain, root cause, or corrective actions suggested, because that's actually hard work, and it's much easier just to point out bad management than provide suggestions of good management.
Estimating is hard. Anything of value is hard. All the easy problems have been solved.
But if we are to ever get a handle on the root causes of software project failure modes, we do need to seek out the root causes. A that means much more than just asking the 5 Whys. That's one of many steps in RCA and far from the complete solution to removing the symptoms of our problems. So start there, but never leave it there.
Here's some approach we use
Unanticipated cost growth is a symptom of failing to properly estimate in the first place, update those estimates as the project progresses, and deal with the underlying risks that drive that cost growth. Same for lateness and less than acceptable delivered capabilities. Once the estimate has been established in some credible form, adjusted as the project progresses, you of course have to execute to the plan, or change the plan. It's a closed loop system
The answer to the root causes that create the symptoms we so quickly label as smells of dysfunction is NOT to stop doing something, but the learn what the actual cause is. Stopping before this knowledge is acquired leaves the symptom still in place. And that doesn't help anyone.
The notion that planning is of little value, seems to be lost on those not accountable for showing up on or before the need date, at or below the needed cost, and with the planned capabilities needed to fulfill the business case or successfully accomplish the mission.
Yogi says it best in our project management domain. And it bears repeating when someone says let's get started and we'll let the requirements emerge. Or my favorite, let's get started and we'll use our perform numbers to forecast future performance, we don't need no stink'in estimates.
Yogi says ...If you don't know where you are going, you'll end up someplace else.
This is actually a quote from Alice in Wonderland
"Would you tell me, please, which way I ought to go from here?"
"That depends a good deal on where you want to get to," said the Cat.
"I don't much care where--" said Alice.
"Then it doesn't matter which way you go," said the Cat.
"--so long as I get SOMEWHERE," Alice added as an explanation.
"Oh, you're sure to do that," said the Cat, "if you only walk long enough."
(Alice's Adventures in Wonderland, Chapter 6)
This is often misquoted as If you don't know where you're going, any road will get you there. Which is in fact technically not possible and not from Alice.
So What To Do?
We need a plan to deliver the value that is being exchanged for the cost of that value. We can't determine the result value - benefit - until first we know the cost to produce that value, then we have to know when that value will be arriving.
Both these conditions are basic Managerial Finance 101 concepts base on Return on Investment.
ROI = (Value - Cost) / Cost
The first thing some will say is but value can't be monetized. Ask the CFO of your firm to see what she thinking about monetizing the outcomes of your work on the balance sheet. Better yet, don't embarrass yourself, read Essentials of Managerial Finance, Brigham and Weston. Mine is 11th Edition, looks like its up to the 14th Edition.
As well, once it is established that both cost and value are random variables about measurements in the future, you'll need to estimate them to some degree of confidence if you're going to make decisions. These decisions are actual opportunity cost decisions about future outcomes. This is the basis of Microeconomics of software development.
So when you hear we can make decisions about outcomes in the future in the presence of uncertainty - the basis of project work - without estimating - don't believe a word of it. Instead read Weston and you too will have the foundational skills to know better.
Because the close loop management processes we need on project and product development require we make decisions in the presence of uncertainty. There is simply no way to do that without estimating all the variance when we Plan, Do, Check, Act. Each is a random variable, with random outcomes. each require some access of what will happen if I do this. A that notion of let's just try it reminds me of my favorite picture of open loop, no estimates, no measurement, no corrective action management.
In the business of building software intensive systems; estimating, performance forecasting and management, closed loop control in the presence of uncertainty for all variables is the foundation needed for increasing the probability of success.
This means math is involved in planning, estimating, measuring, analysis, and corrective actions to Keep the Program Green.
When we have past performance data, here's one approach...
Good estimates are the key to project success. Estimates provide information to the decision makers to assess adherence to performance specifications and plans, make decisions, revise designs and plans, and improve future estimates and processes.
We use estimates and measurements to evaluate the feasibility and affordability of products being built, choose between alternatives designs, assess risk, and support business decisions. Engineers compare estimates if technical baselines of observed performance to decide of the product meets its functional and performance requirements. These are used by management to control processes and detect compliance problems. Process manager use capability baselines to improve production processes.
Developers, engineers, and planners estimate resources needed to develop, maintain, enhance and deploy products, Project planners use estimates for staffing, facilities. Planners and managers use estimates for resources to determine project cost and schedule and prepare budgets and plans.
Managers compare estimates - cost and schedule baselines - and actual values to determine deviations from plan and understand the root causes of those deviations needed to take corrective actions. Estimates of product, project, and process characteristics provide baselines to assess progress during the project.
Bad estimates affect all participants in the project or product development process. Incomplete and inaccurate estimates mean inadequate time and money available for increasing the probability of project success.
The Nature of Estimation
The verb estimate means to produce a statement of the approximate value of some quantity that describes or characterizes an object. The noun estimate refers to the value produced by the verb. The object can be an artifact - software, hardware, documents - or an activity - planning, development, testing, or process.
We make estimates because we cannot directly measure the value of that quantity because:
Reasons to Estimate and Measure Size, Cost and Schedule
Reasons to Estimate Effort, Cost, and Schedule
Reasons to Estimate Capability and Performance
There are many sources of data for making estimates, some reliable some not. Human subject matter expert based estimates have been shown to be the least reliable, accurate and precise due to the biases involved in the human processes of developing the estimate. estimates based on past performance, while useful, must be adjusted for the statistical behaviors of the past and the uncertainty of the future.
If the estimate is misused in any way, this is not the fault of the estimate - both noun and verb - byt simply bad management. Fix that first, then apply proper estimating processes.
If your project or product development effort does none of these activities or has no need for information on which to make a decision, then estimating is likely a waste of time.
But before deciding estimate are the smell of dysfunction, with NO root cause identified for corrective action check with those paying your salary first, to see what they have to say about your desire to spend their money in presence of uncertainty with the absence of an estimate to see what they say.
† This post is extracted from Estimating Software Intensive Systems: Project, Products and Processes, Dr. Richard Stutzke, Addison Wesley. This book is a mandatory read for anyone working in a software domain on any project that is mission critical. This means if you need to show up on or before the need date, at or below your planned cost, with the needed capabilities - Key Performance Parameters , without which the project will get cancel - then you're going to need to estimate all the parameters of your project, If your project doesn't need to show up on time, stay on budget, or can provide less than the needed capabilities, no need to estimate. Just spend your customer's money, she'll tell you when to stop.
In Lean there is a supporting process of 5S's. 5S's is a workplace organization method that uses a list of five words seiri, seiton, seiso, seiketsu, and shitsuke. This list describes how to organize a work places for efficiency and effectiveness by identifying and storing items used, maintaining the areas and items, and sustaining the new order. The decision making process usually comes from a dialogue about standardization, which build understanding around the employees of how they should do their work.
At one client we are installing Microsoft Team Foundation Server, for development, Release Management and Test Management. The current processes relies on the heroics of many on the team every Thursday night to get the release out the door.
We started the improvement of the development, test, and release process with Root Cause Analysis. In this domain Cyber and Security are paramount, so when there is a cyber or a data security issue, RCA is the core process to address the issue.
The results of the RCA have show that the work place is chaotic at times, code poorly managed, testing struggles on deadline, and the configuration of the release base inconsistent. It was clear we were missing tools, but the human factors were also the source of the problem - the symptom of latent defects and a break fix paradigm.
There are many ways to ask and answer the 5 Whys and apply the 5 S's, but until that is done and the actual causes determined, and the work place cleaned up, the symptoms will continue to manifest in undesirable ways.
If we're going to start down the path of 5 Whys and NOT actually determine the Root Cause and develop a corrective action plan, then that is in itself a waste.
Economics is called the Dismal Science. Economics is the branch of knowledge concerned with the production, consumption, and transfer of wealth. Economics is generally about behaviors of humans and markets, given the scarcity of means, arises to achieve certain ends.
How does economics apply to software development? We're not a market, we don't create wealth, at least directly, we create products and services that may create wealth. Microeconomics is a branch of economics that studies the behavior of individuals and their decision making on the allocation of limited resources. It's the scarcity of resources that is the basis of Microeconomics. Software development certainly operates in the presence of scarce resources. MicroEconomics is closer to what we need to make decisions in the presence of uncertainty. The general economics processes ae of litle interest, so starting with Big Picture Econ books is not much use.
Software economics is a subset of Engineering Economics. A key aspect of all Microeconomics applied to engineering problems is the application of Statistical Decisions Theory - making decisions in the presence of uncertainty. Uncertainty comes in two types:
Aleatory uncertainty can be addressed by adding margin to our work. Time and Money. Epistemic uncertainty and the missing information has economic value to our decision making processes. That is there is economic value in decision based problems in the presence of uncertainty.
This missing information can be bought down with simple solutions. Prototypes for example. Short deliverables to test an idea or confirm an approach. Both are the basis of Agile and have been discussed in depth in Software Engineering Economics, Barry Boehm, Prentice Hall. 1981.
Engineering economics is the application of economic techniques to the evaluation of design and engineering alternatives. Engineering economics assesses the appropriateness of a given project, estimates of its value, and justification of the project (or product) from an engineering standpoint.
This involves the time value of money and cash-flow concepts, - compound and continuous interest. It continues with economic practices and techniques used to evaluate and optimize decisions on selection of strategies for project success.
When I hear I read that book and it's about counting lines of code, the reader has failed to comprehend the difference between principles and practices. The section of Statistical Decision theory are about the Expected Value of Perfect Information and how to make decisions with Imperfect information.
Statistical Decision Theory is about making choice, identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision. In Statistical Decision Theory, the underlying statistical processes and the resulting Probabilistic outcomes require us to Estimate in the presence of uncertainty.
Writing software for money, other people's money, requires us to estimate how much money, when we'll be done spending that money and what will result from that spend.
This is the foundation of the Microeconomics of Software Development
If there is no scarcity of resources - time, cost, technical performance - then estimating is not necessary. Just start the work, spend the money and you'll be done when you're done. If however
Making decisions in the presence of this uncertainty is part of our job as project managers, engineers, developers on behave of those paying for our work.
It's also the job of the business, whose money is being spent on the projects to produce tangible value in exchange for that money.
From the introduction of the book to the left...
Science and engineering, our modern ways of understanding and altering the world, are said to be about accuracy and precision. Yet we best master the complexity of our world by cultivating insight rather than precision. We need insight because our minds are but a small part of the world. An insight unifies fragments of knowledge into a compact picture that fits in our minds. But precision can overflow our mental registers, washing away the understanding brought by insight. This book shows you how to build insight and understanding first, so that you do not drown in complexity.
So what does this mean for our project world?
In both these conditions we need to get organized in order to address the underlying uncertainties. We need to put structure in place in some manner. Decomposing the work is a common way in the project domain. From a Work Breakdown Structure to simple sticky notes on the wall, breaking problems down into smaller parts is a known successful way to address a problem.
With this decomposition, now comes the hard part. Making decisions in the presence of this uncertainty.
Reasoning about things that are uncertain is done with probability and statistics. Probability is a degree of belief.
I believe we have a 80% probability of completing on or before the due date for the migration of SQL Server 2008 to SQL Server 2012.
Why do we have this belief? Is it based on our knowledge from past experience. Is this knowledge sufficient to establish that 80% confidence?
The answers to each of these informs our belief.
Chaos, Complexity, Complex, Structured?
A well known agile thought leader made a statement today
I support total chaos in every domain
This is unlikely going to result in sound business decisions in the presence of uncertainty. Although there may be domains where chaos might produce usable results, when some degree of confidence that the money being spent will produce the needed capabilities, on of before the need date, at of below the budget needed to be profitable, and with the collection of all the needed capability to accomplish the mission or meet the business case, we're going to need to know how to manage our work to achieve those outcomes.
So let's assume - with a high degree of confidence - that we need to manage in the presence of uncertainty, but we have little interest in encouraging chaos, here's one approach.
So In The End
Since all the world's a set of statistical processes, producing probabilistic outcomes, which in turn create risk to any expected results when not addressed properly - the notion that decisions can be made in the presence of this condition can only be explained by the willful ignorance of the basic facts of the physic of project work.
This Blog has been focused on improving program and project management process for many years. Over that time I've run into several bunk ideas around projects, development, methods, and process of managing other peoples money. When that happens, the result is a post or two about the nonsense idea and the corrections to those ideas, not just form my experience but from the governance frameworks that guide our work.
This Blog is dedicated to the proposition that all information is not created equal. Much of it is endowed by its creators with certain undeniable wrongs. Misinformation is dangerous!!
There's a lot of crap floating around the any business or technical field. Much of it gets passed around by well-meaning folks, but it is harmful regardless of the purity of the conveyer.
People who attempt to debunk myths, mistakes, and misinformation are often tireless in their efforts. They are also too often helpless against the avalanche of misinformation.
The Debunker Club is an experiment in professional responsibility. Anyone who's interested may join as long as they agree to the following:
When we hear a suggestion about a process that inverts the normal process based on a governance framework - say Microeconomics of Software Development, we need to ask who benefits? How would that suggestion be tangibly beneficial to the recipient that is now inverted?
Estimates for example are for the business, why would the business no longer what an estimate of cost, schedule, or technical performance of the provided capabilities?
In the world of spending money to produce value, the one that benefits, should be, must be, the one paying for that value and therefore have a compelling interest in the information needed to make decisions about how the money is spent.
When that relationship between paying and benefit is inverted, then the path to Qui Bono is inverted as well.
In the end follow the money must be the basis of assessing the applicability of any suggestion. If it is suggested that decision making can be done in the absence of estimating the impacts of those decisions, who benefits. If it's not those paying for the value, then Qui Bono is not longer applicable.
This month's issue of Communications of the ACM, has a Viewpoint article titled "Who Builds a House without Drawing Blueprints?" where two ideas are presented:
The example from the last bullet is there are many coding methods - test driven development, agile programming, and others ...
If the only sorting algorithm we know is a bubble sort no coding method will produce code that sorts in O(n log n) time.
Not only do we need to have somes sense of what capabilities the software needs to deliver in exchange for the cost of the software, but also do those capabilities meet the needs? What are the Measures of Effectiveness and Measures of Performance the software must fulfill? In what order must these be fulfilled? What supporting documentation is needed for the resulting product or service in order to maintain it over it's life cycle.
If we do not start with a specification, every line of code we write is a patch.†
This notion brings up several other gaps in our quest to build software that fulfills the needs of those paying. There are several conjectures floating around that willfully ignore the basic principles of providing solutions acceptable to the business. Since the business operates on the principles of Microeconomics of decision making, let's look at developing software from the point of view of those paying for our work. It is conjectured that ...
There are answers to each of these in the literature for the immutable principles of project management, but I came across a dialog that illustrates to naïvety around spending other people's money to develop software without knowing how much, what, and when.
Here's a conversation - following Galileo Galilei's Dialogue Concerning the Two Chief World Systems - between Salviati who argues for the principles of celestial mechanics and Simplicio who is a dedicated follower that those principles have not value for him as his sees them an example of dysfunction.
I'll co-op the actual social media conversation and use those words by Salviati and Simplicio as the actors. The two people on the social media are both fully qualified to be Salviati. Galileo used Simplicio as a double entendre to make his point, so neither is Simplicio here:
† Viewpoint: Who Builds a House without Drawing Blueprints?, Leslie Lamport, CACM, Vol.58 No.4, pp. 38-41.
There is a popular notion in the #NoEstimates paradigm that Empirical data is the basis of forecasting the future performance of a development project. In principle this is true, but the concept is not complete in the way it is used. Let's start with the data source used for this conjecture.
There are 12 sample in the example used by #NoEstimates. In this case stickies per week. From this time series an average is calculated for the future. This is the empirical data is used to estimate in the No Estimates paradigm. The Average is 18.1667 or just 18 stickies per week.
But we all have read or should have read Sam Savage's The Flaw of Averages. This is a very nice populist book. By populist I mean an easily accessible text with little or not mathematics in the book. Although Savage's work is highly mathematically based with his tool set.
There is a simple set of tools that can be applied for Time Series analysis, using past performance to forecast future performance of the system that created the previous time series. The tool is R and is free for all platforms.
Here's the R code for performing a statistically sound forecast to estimate the possible ranges values the past empirical stickies can take on in the future.
Put the time series in an Excel file and save it as TEXT named BOOK1
> SPTS=ts(Book1) - apply the Time Series function in R to convert this data to a time series
> SPFIT=arima(SPTS) - apply the simple ARIMA function to the time series
> SPFCST=forecast(SPFIT) - build a forecast from the ARIMA outcome
> plot(SPFCST) - plot the results
Here's that plot. This is the 80% and 90% confidence bands for the possible outcomes in the future from the past performance - empirical data from the past.
The 80% range is 27 to 10 and the 90% range is 30 to 5.
So the killer question.
Would you bet your future on a probability of success with a +65 to -72% range of cost, schedule, or technical performance of the outcomes?
I hope not. This is a flawed example I know. Too small a sample, no adjustment of the ARIMA factors, just a quick raw assessment of the data used in some quarters as a replacement for actually estimating future performance. But this assessment shows how to empirical data COULD support making decisions about future outcomes in the presence of uncertainty using past time series once the naive assumptions of sample size and wide variances are corrected..
If you hear you can make decisions without estimating that's pretty much a violation of all established principles of Microeconomics and statistical forecasting. When answer comes back we sued empirical data, that your time series empirical data, download R, install all the needed packages, put the data in a file, apply the functions above and see if you really want to commit to spending other peoples money with a confidence range of +65 to -72% of performing like you did in the past? I sure hope not!!
The notion that we can ignore - many times willfully - the microeconomics of decision making is common in some development domains. Any project driven paradigm has many elements, each interacting with each in random ways, in nonlinear ways, in ways we may not be able to even understand when the maturity of the organization is not yet developed to a level needed to manage in the presence of uncertainty.
So When We Say Project What Do We Mean?
The term project has an official meaning in many domains. Work that has a finite duration is a good start. But then what is finite? Work that makes a change to an external condition. But what does change mean, and what is external. In most definitions, operations and maintenance are not usually budgeted as projects. There are accounting rules the describe projects as well. Once we land on an operational definition of the project, here's a notional picture of the range of projects.
My favorite questionable conjecture is that we can make decisions about the spending of other peoples money without estimating the outcomes for that decisions. Making decisions about an uncertain future is the basis of Microeconomics.
One framework for making decisions in the presence of uncertainty is Organizational Governance. Without establishing a governance framework, ranging from one like that below, to No governance, just DO IT, it's difficult to have a meaningful conversation about the applicability of any project management process.
So when we hear a new and possibly counter intuitive suggestion, start by asking In What Governance Model Do You Think This Idea Might Be Applicable?
An estimate is the most knowledgeable statement you can make at a particular point in time regarding, cost/effort, schedule, staffing, risk, the ...ilities of the product or service.
Immature versus Mature Software Organizations 
Setting sensible goals for improving the software development processes requires understanding the difference between immature and mature organizations. In an immature organization, processes are generally improvised by practitioners and their management during the course of the project. Even if a process has been specified, it is not rigorously followed or enforced.
Immature organizations are reactionary with managers focused on solving immediate crises. Schedules and budgets are routinely exceeded because they are not based on realistic estimates. When hard deadlines are imposed, product functionality and quality are often compromised to meet the schedule.
In immature organizations, there is no objective basis for judging product quality or for solving product or process problems. The result is product quality is difficult to predict. Activities intended to enhance quality, such as reviews and testing, are often curtailed or eliminated when projects fall behind schedule.
In mature organizations possesses guide the organization-wide ability to manage development and maintenance processes. The process is accurately communicated to existing staff and new employees, and work activities are carried out according to the planned process. The processes mandated are usable and consistent with the way the work actually gets done. These defined processes are updated when necessary, and improvements are developed through controlled pilot-tests and/or cost benefit analyses. Roles and responsibilities within the defined process are clear throughout the project and across the organization.
Let's look at the landscape of maturity on estimating the work for those providing the funding for the work.
Projects are small, short, and while important to the customer, not likely critical to the success of the business in terms of cost and schedule.
The result of this level of maturity is poor forecasting of the cost and schedule of the planned work. And surprise for those paying for the work.
Projects may be small, short, and possibly important. Some for of estimating, either from past experience or from decomposition of the planned work is used to make linear projects of future cost, schedule, and technical performance.
This past performance is usually not adjusted for the variances of the past, just and average. As well the linear average usually doesn't consider changes in the demand for work, technical differences in the works, and other uncertainties in the future for that work.
This is the Flaw of Averages approach to estimating. As well the effort needed to decompose the work into same sized chunks is the basis of all good estimating processes. In the Space and Defense business the 44 day rule is used to bound the duration of work. This answers the question how long are you willing to wait before you find out you're late? For us, the answer is no more than one accounting period. In practice, project status - physical percent complete is done every Thursday afternoon.
There is an estimating process, using recorded past performance and the statistical adjustments of that past performance. Reference Classes are used to model future performance from the past. Parametric estimates can be used with those reference classes or other estimating processes. Function Points is common in enterprise IT projects where interfaces to legacy systems, database topology, user interfaces, transactions are the basis of the business processes.
The notion that we've never done this before so how can we estimate, begs the question do you have the right development team? This is a past performance issues. Why hire a team that has no understanding of the problem and then ask then to estimate the cost of the solution? You wouldn't hire a plumber to install a water system if she hadn't done this before in some way.
4. Quantitatively Managed
Measures, Metrics, Architecture assessments - Design Structure Matrix is one we use - are used to construct a model of the future. External Databases referenced to compare internal estimates with external experiences.
There is an estimating organization that supports development, starting with the proposal and continuing through project close out. As well there is a risk management organization helping inform the estimates about possible undesirable outcomes in the future.
 Improving Estimate Maturity for More Successful Projects, SEER/Tracer Presentation, March 2010.
 Software Engineering Information Repository, Search Results on Estimates
 The Capability Maturity Model for Software
We don't need to Hope the sun will come up tomorrow. That probability is 100%. The probability that the outpatient surgery you've scheduled next week will be successful is high. The surgeon has done this procedure 1,000's of time with a 98% success rate.
The probability that you'll be able to arrive on time with the needed features for the first release of the accounts payable and accounts receivable upgrade to the ERP system, is not like the sun coming up or the outpatient surgery. It's actually a probability that you'll have to think about. Actually have to calculate. Actually make an estimate of this probability before those paying for your work can make a decision about when to switch from the old system to the new system.
Hoping we can show up when needed is not a strategy. To show up on time, and on budget, with the needed capabilities. To do that, we need a Closed Loop Control System, in which we have a target performance, measures of actual performance, compared to estimates of the planned performance to create an error signal to take corrective actions.