“McIntyre’s first step in trying to replicate a paper was to collate the data. While data might be cited correctly and accurately in the papers, it was always possible that what had been used was different in some way to the official versions, whether due to an error in the archive or one made by the authors.”
Almost at every turn in this monumental exposé by A. W. Montford, I see parallels in the many cloud seeding reanalyses I did at the University of Washington with Peter Hobbs. The two sentences quoted above from Montford’s book, so fundamental a step in checking results, literally leapt off the page since that is exactly where the most basic replication starts, and where we always began in our cloud seeding (CS) reanalyses.
In our re-evaluation of perhaps the most important randomized wintertime cloud seeding experiments ever conducted, those at Climax, Colorado, 1961-1970, we started with the raw data that the experimenters said they had used. This was precipitation measurements at the cloud seeding target gage that were taken by an independent organization and archived by NOAA, thus making it publicly available. The experimenters high lighted this independence in their publications.
But when those values from NOAA were used in the re-evaluation of those Climax experiments, discrepancies were found, just as Montford reports that Steve McIntyre found so often in his proxy raw data examinations. In our case, the seeded days generally had more snow in the experimenters’ data at the NOAA target gage, and control days less than was actually the case according to the NOAA data. Furthermore, these discrepancies were only observed in the second “confirmatory” experiment (1966-1970) on the days that were supposed to respond the most to seeding. In our re-evaluation of the second experiment (aka, Climax II) the use of the NOAA precipitation values, along with other data corrections, degraded the results so badly that they did not confirm the first five season experiment after all. The experimenters had previously reported that Climax II had been a confirmation of the first (aka, Climax I) experiment.
As one might imagine, the initial reports of a the “confirmation” of the earlier “exploratory” CS experiment gave those two experiments together a great deal of caché as strong evidence that snow could be increased on a determinant basis through wintertime cloud seeding. And they were cited as having done so by the prestigious National Academy of Sciences Panel on Climate and Weather Modification in 1974. (As an interesting aside, the NAS Panel was also concerned at that time about the “…recent equatorward shift in ice boundaries.”)
Further work “de-constructing” those experiments at Climax, that is, the discovery of more discrepancies, can be found here.
Eventually the experimenters acknowledged the source of their errors in precipitation at the target gage in a journal exchange in 1995.
Epilogue
However, unlike the situation that McIntyre repeatedly encounters in Montford’s HSI account, where climate researchers refuse to honor requests for raw data, in our re-analysis of cloud seeding experiments in Colorado, the experimenters at Colorado State University were totally cooperative in supplying data that was occasionally requested by the present writer. They did this even though they KNEW that the requestor was a critic/skeptic, might challenge their earlier results. To their great credit, the ideals of science were given a higher priority than their egos by those at CSU and that finding problems and discrepancies were recognized as a way of advancing science, not hindering it.
