There is a notion floating around that
Unless you have skin in the game your estimate for the work can't be credible
Let's look at the potential for such an approach in light of the credibility of an estimate. First a definition
An independent cost estimate (ICE) is a tool to assist in determining the reasonableness or unreasonableness of the bid or proposal being evaluated and is required for all procurements regardless of dollar amount.
The ICE - and it can be any type of estimate, cost, schedule, technical performance - is usually a life-cycle estimate conducted independently of the management of the project by some 3rd party. This can an external party or an internal party. The independence of the estimate is critical to removing unintended or intended biases in the outcomes of the estimate.
These biases start with the Narrative Bias. As humans, we become obsessed with creating a narrative to support our position that many times do not exist. This is most common in the support of our outlook on the future outcomes. The Narrative Fallacy is present in our efforts to estimate the future cost, schedule, and performance of software projects.
- The narrative fallacy occurs when a set of connected or disconnected facts are picked to fit a story.
- When there is skin in the game the person doing the estimate is motivated to have the outcome of the estimate to match some predefined result that will put the estimator in the best light.
- No one like to pay for bad news.
- No one getting paid to estimate likes to present bad news.
The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship upon them. Explanations bind facts together. They make them all the more easily remembered; they help them make more sense. Where this propensity can go wrong is when it increases our impression of understanding.
—Nassim Nicholas Taleb, The Black Swan
One of the popular notions is we can use past performance - historical data - to forecast future performance. Ignoring the fallacy that this forecast is not actually estimating, there are some other fallacies here that must be addressed before that past data can be used for the future.
- The Fallacy of Casual Analysis (False Association) occurs when the accumulated facts have minimal generalizations.
- This can be seen when you hear I know of projects that use some process successfully.
- Without the ability to generalize this observation, it is just an anecdote.
- There is no correlation that the estimate of the size of Project X is an equivalent of all the historical projects that are similar to Project X.
- The fallacy of silent evidence occurs when we see only what has been recorded and we remain willfully ignorant of the missing evidence.
- This is cherry picking the data.
- This is the basis of the fallacy of the Standish Chaos Report.
- It's also the basis of the fallacy of using 40-year-old NATO reports of project performance, and reports outside the domain of the problem - the abused quote of Hofstader - It always takes longer than you expect, even when you take into account Hofstadter's Law.
- The Ludic Fallacy assumes the data to be statistically analyzed is complete, unaffected by variation or not intentionally corrupted
- The past performance has a ±75% variance.
- This is the Flaw of Averages fallacy.
So when you hear Unless you have skin in the game your estimate for the work can't be credible that's bogus and subject to many fallacies of estimating.
Independence of the estimate is the starting point for correcting many of these estimating fallacies
In our domain, we start our estimating processes on the proposal with a Price to Win Strategy and fall in these fallacies on day one.
