Last month I wrote a response to a paper written by Dr. Andrew Lawrence, Dr. Benjamin Sovacool, and Dr. Andrew Stirling. If you haven’t read that, here is a link to that analysis and response. Since then, I have been in contact with some of the authors, and last week I received word that the authors have decided to withdraw their paper, partially based on my analysis of their paper. The full situation has also been written about in Retraction Watch.

Dr. Lawrence, Dr. Sovacool, and Dr. Stirling have written an “Authorial Statement of Article Withdrawal” and have also written a response to my analysis. It should be noted that throughout this process, Dr. Sovacool has been very open and accessible, and I’d like to thank him and the other authors for taking the time to review their work, and admit when a mistake has been made. Here is their response.

Response to Nicholas Thompson

https://thompson.energy/2016/10/12/a-response-to-lawrence-sovacool-and-stirling/

We thank Nicholas Thompson (NT) for his serious engagement with our paper. Below, we respond to the specific issues raised. We also refer to the statement of retraction posted in the journal Climate Policy. The bottom line is that, NT has in his main criticism done an elegant job of deconstructive analysis. We commend and congratulate him on it.

Problem 1: The math is incorrect.

Unfortunately, this is a valid criticism, and on two counts. Very embarrassingly, mistakes were made both in data input and analysis. One column of data was mistranscribed. In addition, some percentage changes were not calculated correctly with respect to the stated baseline but with respect to an earlier date. When these mistakes are corrected, NT is correct that these particular data do indeed show that nuclear states did make the greatest absolute emissions reductions compared to all other groups. These errors should never have happened. A more detailed background is given in the statement of retraction. We are proactively correcting reports.

We also plan on undertaking a new analysis with the correctly transcribed data, to be completed and shared soon.

Problem 2: The method is incorrect and Problem 5 no attempt to account for population.

As acknowledged above, the method was unfortunately implemented in a flawed way. But, contrary to NT’s assertion, it is not self-evident that the method itself was intrinsically deficient. Based as it is on comparisons across qualitatively different institutional and jurisdictional settings, it is not clear that any single given weighting factor (e.g.: per capita emissions; emissions per GDP; or emissions attributable to specific policies) would of itself necessarily constitute an unequivocal improvement in the analysis. One specific dimension of variation could be made more explicit, but only at the expense of obfuscating others. The purpose of this analysis was simply to perform an initial test on a widely received assumption. The possibility of further elaborations does not invalidate the initial design.

Problem 3: There is no justification for the starting year of analysis.

The decision to use 2005 as the baseline year was made in recognition of the highly dynamic nature of change in this field. To use an earlier year would have diluted the effects of more recent trends (and been unfavourable to the stated findings). To use a more recent year would have emphasized the most recent developments (and been more favourable to the stated findings). So the chosen framing is not arbitrary or contrived. It is rather that a more complete analysis would systematically display sensitivities to these divergent framings.

Problem 4: Statistical significance.

At no stage was our analysis intended or claimed to rest on statistical arguments. As this present exchange illustrates, the underlying categories for any analysis of this kind are too ambiguous and volatile to warrant undue confidence in any particular aggregated statistical constructs. To assert this would be misleading. Instead, our contribution was intended to challenge a widespread assumption about the supposed climate benefits of nuclear power. Again, we must acknowledge that the test was in this case poorly implemented.

Conclusion

In summary, NT is correct that our article contains flaws that invalidate the analysis. This is by far the most important issue in this exchange. On other aspects of NTs critique, the position is more ambivalent. But it is certainly the case that analysis of this kind can be improved in ways prompted by NTs critique and discussed above, in order to yield more robust conclusions.

With respect to any implication of intentional bias towards renewable energy at the expense of nuclear power, we should like to stress that the errors described above were the result of honest mistakes. We hope that our unqualified acknowledgement of these errors here – and our proactive voluntary retraction of the article – insufficient to show that we are entirely open both to criticism and to different perspectives.

I do want to push back on a few things in this response:

1) As my original analysis showed, this methodology is somewhat arbitrary, and depending upon the time window that is chosen, will lead to different conclusions. Additionally, giving equal weight to extremely small and large countries can skew the data in different ways. If there is a new analysis down with the same methodology, I have concerns that it will fall prey to the same issues that came up in this paper.

But more importantly,

2) The authors don’t seem to deny that they have a bias against nuclear energy. I completely believe that any mistakes that were made in this paper were honest mistakes. But in this response, they state that their, “…contribution was intended to challenge a widespread assumption about the supposed climate benefits of nuclear power.”

In this sentence, the authors seem to be saying that it is their belief that nuclear does not have climate benefits, or at least that was what they were trying to prove.

Most Life Cycle Analyses (LCA) of nuclear energy show that nuclear energy has relatively low life cycle emissions of greenhouse gases (meaning that over the life of that facility, from mining the materials, building the facility, operating the facility, to decommissioning it and disposition of waste, the total carbon emitted per energy produced is low). One of the most comprehensive studies of this was done by the National Renewables Energy Laboratory in the US, where hundreds of these LCA studies were reviewed, and the results were that nuclear emits about the same amount of greenhouse gases over its life, per power produced, as wind power does (median values for nuclear were 12-13 g CO2eq/kWh, wind was 11-12 g CO2eq/kWh, solar PV was 44 g CO2eq/kWh – for comparison, coal was 979 gCO2eq/kWh).

Nuclear energy, when done well, does produce low carbon energy – this isn’t an assumption, and the climate benefits are real, not “supposed”.

There is some great social science research about how people’s beliefs can actually impact their quantitative reasoning, and this even effects people who are very skilled at math (it can actually effect them worse!). This result was found by Dan Kahan, Ellen Peters, Erica Dawson, and Paul Slovic in 2013 (for a simple explanation, go here). This is why I completely believe these were honest mistakes in the Lawrence, Sovacool, and Stirling paper – because the authors of the paper probably found a result that seemed to reinforce an idea they had (that nuclear does not have climate benefits), and did not question the results.

All scientists should start any project by looking for an answer to a question, and not looking for information that backs up a previous belief.