The Blog: a new mechanism of scientific review?

If Climategate, and the ensuing controversy, has given us nothing else, it have exposed shocking examples of how scientific processes of review can become so corrupted that bad science survives and thrives in the suppression or diversion of sound criticism. There is evidence of corruption at every turn of the process: from the apparent neglect of the proper checks of facts and data behind articles subject to peer review, to the obstruction of the normal critical processing of published papers – and this occurring in journals of the highest impact, and including the refusal to release the primary data upon which the published findings could be tested.

Climate science insiders (and outsiders) could see what was going on all along. And certainly anyone who cared to follow the controversy over the Hockey Stick graph after its publication in Nature in 1998 – and then prominently in the IPCC report of 2001 – they could detect dysfunction in these critical processes. But it would take the leaking of the Climategate emails to release more than a decade of pent-up protest breaking though the official silence, deflection and obfuscation.

Steve McIntyre

Climate Audit blogger Steve McIntyre as he emerged in a profile in The Guardian, 9 Feb 2010

The momentum of Climategate permitted at last a critical reading of the IPCC 4th assessment report. Until this time any questioning, any challenge to the reports from outsiders, had been shouted down with the mantra that we should ignore anything but the proper peer-reviewed sources, and we should allay any doubts by trusting the assessment of these sources in the ‘gold standard’ of such authority, the IPCC reports. And throughout the heat of the emails scandal this mantra was shouted all the louder. Closer scrutiny of the 2007 report put the lie to this claim, revealing that in fact some of its most alarmist and controversial claims about the impacts of the predicted warming ran roughshod over peer-review science to establish an obscured authority via reference to ‘grey literature.’

As I write, we are now passing the point where the more this ‘gold standard’ mantra is chanted, the more foolish its choir appears, and the more legitimacy is awarded its critics. Blocked for so long, the pipes are clearing, and the normal critical processes of science look likely to start flow again – criticism will be permitted, primary data will be released and the claims of ‘certain’ and ‘settled’ science will recede. In the meantime, while waiting for this to unfold, we can pause to reflect on how the breakthrough was achieved, and consider in particular the appearance of a new mechanism of review that emerged heroic during this brief episode of scientific corruption. I refer to the extraordinary corrective role played by the blog.

Establishment science

In a BBC discussion earlier this week, Roger Harrabin presents a picture of a scientific ‘establishment’ completely unprepared for the recent onslaught. He uses as an example the announcement last week of the commencement of the UEA inquiry into Climategate. An outline of the inquiry was released, and a reviewing panel was announced a long three months after the inquiry had first been heralded. The gathered press were told that the inquiry is being conducted by the UEA, funded by the UEA, and with the advice of the Royal Society. Harrabin tells us how he posed a question about perceptions of vested interest, whether they had thought of establishing the inquiry with some degree of independence. They had not (although they gave a gentleman’s assurances that they would be independent). Harrabin was incredulous:  Here we have an establishment behaving in a normal way as if the internet is not going to shout back at them.  Harrabin finds himself witnessing an establishment entirely unprepared and remaining ill-equipped for the fact that they are dealing in everything they do with a broad global public in a way that is inconceivable to them when they started their careers.

Harrabin recognises this controversy as presenting a huge challenge to the way science is conducted, not just climate science but across the board, and it is also a crisis for science journalism:

What’s been difficult for people reporting the mainstream debate in the past has been that what we would call our trusted sources of Science – the RoySoc and various other corollary bodies in different countries (and the IPCC  set up to be the touchstone of probity on this issue) – these have been the providers of news. And those who have been doubting this news are not academics…and it has been difficult to establish what are the credentials when all of these establishment voices end up on one side, how can we pit them against a blogger, who might [have researched a lot of climate science and so] he may have a good knowledge, but we don’t know how to test this.

Indeed, this crisis for science journalism – and the way it has been met in recent weeks variously by the likes of Roger Harrabin, Fred Pearce (the Guardian), Andy Revkin (NYT) and Robin Williams (Aust. ABC) – this is a fascinating case study in itself. But, although intertwined, our interest here is in the impacts not so much on the journalism of science but on the processes of science. And so I pose this question about the climate science controversy:

Just how powerful has blogging become as a review mechanism of science?

 

Auditing climate science online

Consider firstly the most notorious blogger in the whole scandal, Steve McIntyre, who has used the web since 2003 to publicise his ‘audit’ of the statistical analysis of the data that gave the Hockey Stick graph. The unanswered questions and criticisms he raised became so notorious that an investigation was conducted by the National Academy of Science, and then by US Congress. It seems that these extraordinary processes of review were necessary because the normal processes were either unavailable, or had little effect in a situation of no small significance: His criticism could not be more damaging to a conclusion that could not be more influential – namely, that recent global warming is historically unprecedented.

McIntyre’s attempts to continue his ‘audit’ of the statistical methodology was stonewalled by authors, universities and journals all refusing the normal process of releasing the data behind the published conclusions. With the academic establishment defying their enlightenment creed of disclosure so as to fall behind the Hockey Team, the last resort for McIntyre and others was to the law and the Freedom Of Information Act. Repeated unrequited requests continued right up until the FOI2009.zip file was leaked. Finally, in the midst of Climategate and further scandals about the sourcing of weather station temperature data, CRU declared that some of the data could never be released for they had in fact been lost or destroyed.

During these years the Hockey Team had been publicly dismissive of the criticism of McIntyre and others who were outside the climate science establishment. But one of the revelations of the Climategate emails was just how concerned privately they actually were. With MacIntyre mentioned over 100 times, sometimes in very disparaging terms, some would say they were ‘obsessed’ with him. The ‘bunker mentality’ that many commentators (including supporters of AGW Alarmism) find evident in the emails is a bunkering down against the onslaught of critics such as McIntyre blogging glaring criticism of their science and then pressing for more data to be subject to their critical eye. The only attempt at any real public engagement with their critics was the establishment of the Real Climate blog, which sceptical commenters soon discovered operated more at the level of PR (or propaganda) than as a real forum for scientific discussion. The Hockey Stick, so prominent in the 2001 report, did not appear in the 2007 report largely due to McIntyre’s persistent attack. McIntyre had come from nowhere and, mostly through posting analysis on his blog and thereby rallying a visible following in the comments, he profoundly influenced the course of the public debate over the science of global warming.

Well, this is all history now, and since Climategate the debate over climate science has shifted – making it hard to imagine a return to the conditions under which the blog emerge as a powerful mechanism in the review process, and so the next question is:

Will the blog remain important or will its moment soon pass?

 

Blogging and science journalism

What we do know is that its moment has not yet passed, for it is only in the last few weeks that science journalists in the mainstream media have found that (despite the problem of authority mentioned above by Harrabin) they can no longer dismiss the blogs as little else but sound and fury. For much of the criticism of the ‘grey’ and obscured authorities cited in the IPCC 4th assessment came through blogs, sometimes emerging through anonymously posted comments. And so now in The Guardian, The Times and the Harrabin’s own BBC no less, we find not only the reporting of these source of the news breaks, but also their legitimation as forums of review. The blogs run by the likes of McIntyre, Anthony Watts and A W Montford are now recognised as legitimate and credible sources of criticism. It is the difficulty, but necessity, of accepting their authority that Harrabin describes as ‘a huge challenge’ for science and ‘a really big moment’ for science journalism. This shift in journalistic response to the ‘blogosphere’ was most abruptly marked by a shift in The Guardian’s coverage of the controversy during the 2nd week February 2010, where the Bloggers began to appear as legitimate and somewhat vindicated actors in the whole drama (e.g., see this report featuring McIntyre). And this brings us back to the announcement last week of the UEA Climategate inquiry.

The head of the inquiry, Sir Muir Russell, issued a statement that none of the inquiry members have a ‘predetermined view on climate change and climate science.‘ Many saw this declaration as an ill-considered, if not absurd claim, considering that most of the panel, with the Royal Society, had already made their views on climate change and the science behind it pretty darn clear.

Meanwhile, at Bishop Hill, Montford duly reports the announcement of the panel members, and the claims of their impartiality. Phillip Campbell, the chief editor of Nature – the journal that first published the Hockey Stick – had been elected to the panel for his expertise on the peer review process. Clearly, he was exposed, and it was not long before a commenter attached a link to an interview where Campbell clearly stated his predetermined view that there was no evidence of wrong-doing in the emails. This was picked up by the mainstream media, and within 6 hours of the announcement Campbell had resigned. But it did not stop there, with other members of the panel easily identified with predetermined views on climate change. Faced with so much ridicule on the blogs, Sir Russell soon backed down from the claim of impartiality…and then this back down became the subject of ridicule…and so it continues

The serious side to this debacle is that the inquiry became necessary because of the apparent failure of normal processes of scientific review. And, as Harrabin had already intimated even before this farce had played out, here we have the scientific ‘establishment’ – in this case represented by the UEA, the Royal Society and Nature – surely on the highest alert, yet seemingly oblivious to the power of the blogs to quickly unmask yet another barely obscured contradiction. What is new is that now no one in the press, and no one in the establishment, can deny anymore that this unmasking of contradictions in the blogs does really matter.

So clearly the blog is continuing to play a vital role in bringing this establishment to accounts. The question is whether there is a role for the blog if the normal processes of review starts to work again. What I wonder is whether, when the dust finally settles, we find that these extraordinary condition has indeed introduced a new mode of review, and that the model for might be Anthony Watt’s blog Watts Up With That?

In just 3 years Watts Up With That? has established itself as a general forum for climate science discussion that is entirely open, voluntary and informal. Watts’ role is not so much as a climate science expert, but more as an editor, selecting items for discussion and debate, but in an environment free of the institutional framework of academic science. In recent times it has become more common for Watts to invited guest experts to post short article, and this places him in a role much more closely akin to a journal editor. Yet there are differences between his role and that of Phillip Campbell at Nature, and difference in the review processes they oversee.

For one, Campbell won his position through career advancement, while Watts won his position by popular interest. Another difference is that publication at Watts’ blog exists entirely outside the system linked to academic career advancement. Out in the wilderness Watts has unfetted freedom and power over his own domain – he has more personally control over what is promoted and publish. Another difference with the academic journal is that on the blog review (both peer and popular) occurs after publication, and it is much broader, open and immediate.

My question is whether this science blog is offering the model for a new process of scientific review, or is it just an artefact of a time when the normal process of review had become so obstructed that such extraordinary means became but a temporary necessity?

– BernieL

About these ads

One thought on “The Blog: a new mechanism of scientific review?

  1. Hi Bernie,

    I stumbled across your blog (which, btw, I’ve enjoyed reading) via my recent re-visit to ScientistForTruth’s blog.

    Wrt this particular post, I would note that you are mistaken in claiming that:

    “The Hockey Stick, so prominent in the 2001 report, did not appear in the 2007 report”

    It did, in fact, appear [albeit in a slightly disguised graph]. In fact [thanks to the Climategate files], I’ve actually conducted an analysis which illustrates the polluted IPCC process by which it was “resurrected”.

    You might want to take a look at:

    http://hro001.wordpress.com/2010/01/18/the-climate-change-game-monopoly-the-ipcc-version/

    There are two other areas of concern/question that I believe are worth exploring:

    1. The extent to which “science journalists” – who have failed to do their homework – are a big part of “the problem”. Their advocacy and personal bias frequently colour their “reporting” – and they readily accept at face value any press release that supports AGW.

    2. The use and abuse of the English language to obfuscate the uncertainties that surround the claims of “climate scientists”. For example, Mann claims that the NRC “supports” his conclusions; yet, if one examines the 2006 NRC report, one finds:

    “[...] Even less confidence can be placed in the original conclusions by Mann et al. (1999) that ‘the 1990s are likely the warmest decade, and 1998 the warmest year, in at least a millennium’ because the uncertainties inherent in temperature reconstructions for individual years and decades are larger than those for longer time periods and because not all of the available proxies record temperature information on such short timescales.”

    http://books.nap.edu/openbook.php?record_id=11676&page=4

    How this paragraph can be construed as “support” for Mann’s conclusions [as he and his fellow alarmists are wont to do] is somewhat beyond belief!

    And speaking of use of the English language … In your post, you describe Steve McIntyre – and his questions and criticisms – as “notorious”! That may well be the Hockey Team’s depiction of him; but (IMHO) “notorious” is a label that belongs to Mann and his hockey stick, not to their most well-known critic. Nemesis, bane of the “team’s” existence? Absolutely! But, “notorious” … nah, I don’t think so :-)

    As for your question: “Will the blog remain important or will its moment soon pass?”

    Given the shameful state of the MSM (with a few notable exceptions), I don’t see bloggers’ influence waning anytime soon!

Join the discussion

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s