With the #NoEstimates topic still ringing in my ears, here's an interesting presentation from one of the supporters of this concept. It has many good concepts, one serious math error, and connects well with how we manage and work billion dollar programs.
Ignoring for a moment some of the bad math in the beginning - can't forecast the future, not true and done all the time with high confidence - there are good observations in his domain that connect to how we forecast the future cost and schedule in ours.
- The difference between cardinal and ordinal values (numbers) when estimating. This is a core issue in agile. Story Points are ordinal. They are not calibrated prior to use. Items can be calibrated and are therefore are cardinal. Reference Class can be cardinal - and must be to be useful.
- The notion of small increments is critical to success. Little Bets is the general notion, we've tired to apply this more. Rolling Waves, 44-day rule are examples in our domain. Not used enough by the way. His notion is very similar - and it'd be interesting to have a long conversation - to Reference Class Forecasting.
- The notion that the backlog grows at the same time as it is being reduced brings up a critical notion that is entering our domain. Define the needed capabilities before defining any technical details. The concept of Capability Based Planning is entering a wider realm.
- The statement what we release to customers is features not story points. This is the basis of our Integrated Master Plan / Integrated Master Schedule deliverables based planning process. This deliverables focus is baked into the DOD acquisition process. The presentation make this clear. Customers bought a capability. They didn't actually buy the code and certainly didn't buy story points.
- The notion of rolling wave is stated when he speaks about no worrying about future work that is not planned in detail. Rolling Wave is also baked into our domain with the IMP/IMS and DOD acquisition process.
- Decomposing the story points needs to be broken up. Just like Work Packages and Tasks using the 44-day rule. That means no work can cross more than one accounting period. It answers the question how long are you willing to wait before you find out you're late?
- Here's a serious error, Gaussian distributions ONLY work when the underlying processes are identically independently distributed (iid). This almost NEVER happens in projects. Everything is coupled, almost nothing is independent.
- The notion of using Story Points versus Items Delivered is interesting. What's needed is the size of the items delivered in units independent from the story points. It may be that the items delivered is a better indicator of final cost, but we need to know why. It may be a function of the statistical distribution of the work that naturally occurs when defining deliverables.
