Best Evidentuary Materials
I think well of all skepticism to which I reply: "Let's try it." – Friedrich Nietzsche
When a suggestion is made that there is a solution to a long standing problem - such as productivity improvement or performance enhancement - my gut reaction is "really, you've shown that the improvement is connected to your process?"
A Physical Sciences Approach to Process Improvement
My core training (training is not the same as education BTW) is in physics. Experimental particle physics actually. This is where I learned to program - on a PDP-8M, doing Fast Fourier Transforms in 8K of 12bit wide memory, 125KHz 8-bit A/D converter, a Tektronix 4014 display and a pen plotter.
When someone came into the Principle Investigator's (PI) office with a picture of something new off the experiment - these pictures were taken with a Polaroid camera fitting to an oscilloscope. The first response would have been - go back to the lab and get me 3 more pictures just like that from your experiment. Do this with my RA watching you.
With those pictures in hand the next questions would be why do you think you should be seeing what you are seeing? This is where science is separated from conjecture, anecdotes and speculation. There needs to be a direct and traceable connection between cause and effect. Why should your proposed process actually result in improvement? This approach is very annoying to many in the process improvement or methodology business. They'll claim what they are doing is not science, engineering, or anything close. Rather "trust" me it works. Or "try it your self." Or the best disclaimer around "your mileage may vary."
In principle there is nothing wrong with this approach, in practice there is. Just don't bet your company or career on the process or method without first doing a bit of "experimenting" yourself. This is a good point to think of Nietzsche.
Like a very experienced risk analysis consultant said on our program
Can you introduce me to three managers on programs that have completed, that have used your tool or process?
The next step in the science lab world involves taking the experiment apart and putting it back together to see if it still behaves in the same way. Then having someone else take it apart and put it back together possibly on the other side of the country. With these steps in place, the theory of why and the practice of how need to be connected to make a projection of what else you should be seeing from the results.
This is the measurement problem with many process improvement initiatives. What should I be measuring? How can I connect these observed improvements with the underlying process
And the Point Is?
When there is a suggestion that a specific process or method provides an improvement - maybe through a case study - look deeper before nodding approval...
- Was there a baseline to which to compare the new results?
- Did this baseline contain really bad behavior in the first place? If so, then simply showing up and stopping the bad behavior would have resulted in some improvement.
- Is the improvement statistically significant. a 155% improvement on a process that has 200% swings in performance may not be statistically significant.
- Like the Fleischman and Ponds Cold Fusion experiment - can these improvements be repeated and sustained in the absence of the experimenter?
- Is the experiment or case study really neutral regarding the result and the conclusion. The concept of an Independent Non-aligned Review is a power incentive here in aerospace not to cook the books regarding benefits of any process. Be wary strangers bearing gifts.
Postscript
There is much talk about how the Toyota Production System (TPS) works and how it can be applied to a broad set of applications. This link provides some background. The central philosophy of the TPS is the complete elimination of all waste. When the TPS is mentioned in a conversation about process improvement. Ask the speaker to described how exactly they propose to eliminate all waste in the new process, in practice not in theory.
Comments