There are many new and exciting results in the world of science - physics is my area of knowledge. But many turn out to be wrong. There are examples from the world of physica science. And social scientist want to be called scientist, they are subject to the same scrutiny as a physicist.
Giovanni Schiaparelli's canals on mars, Professor Blondlot's n-rays (1903), Piltdown man, the English answer to the Neanderthal man and the Cro-Magnon man discovered on the continent. These are not unique in the world of physics. Polywater, Cold Fusion, pentaquarks. Cancer has been cured so often it is a surprise that any cancer cells are left.
Beyond Physics There Is Lots of Hype About New and Exciting Things
So why are so many exciting results wrong? to be exciting means, almost by definition, that the results have a low prior probability of being correct. The discovery that our dog hs been storing beef bones in the backyard is not very exciting. But if I found a gold vein that leads under the 16th green behind our house, now that'd be exciting - except I don't own the mineral rights!
It is the unexpected that is exciting and gets hyped and over hyped. The pressure to come up with something new is tremendous in our internet world. As well you want to get it widely distributed so you get the credit. This leads the pressure to not checking your results, confirming they are actually new and exciting and not just a rehash of an old idea you had not encountered. This also means you may not be as careful as one should,to check the results are in fact applicable outside you our anecdotal experience.
When we hear of "Something 2.0" or even better 3.0, and rush off to buy the book, read the web page, attend a semniar, then we've fallen into the Hype process. It's driven by the marketing engine of book publishers who have to get you hyped for the new and exciting idea that will - save the world, put your firm back on track, provide you with unimaginable skills to solve problems that have never been solved before.
So Before We Bite
So before we rush off and adopt the latest new and exciting anything - especially in the business worls, where there is no way to have a double blind test of the claimed outcomes, have any theory to explain why we should be seeing this results, and have any chance or replicating the outcome - at least ini short order - outside of the claimant of the outcome - we need ask and answer some questions.
- Has the outcome been verified by a third party observer. In other words has anyone else seen this work?
- Has this claim been tested somewhere else with similar outcomes in similar situations?
- Is there some basis for accepting the outcomes as anything other than just good practice that was not in place in the first place?
- And finally, can this beneficial outcome be reproduced in some other business or technical domain, like where I work and and claimant doesn't - or again is this just good practice?
So when you see 2.0 or 3.0 behind a common term, think twice. Have you seen this idea been somewhere else, maybe when the author was in short pants as they say in the UK. For example much of the agile process were in place and being used years before they were hyped in the software development world.
