Why the IPCC never writes its own reports

(And why that matters)

Here begins a number of posts drawing on themes raised in Searching for the Catastrophe Signal.

The Working Group 1 First Assessment Report was written not by the IPCC delegates but by scientific experts. When first presented for approval to the Panel, it was already a commercially published volume. The Brazil-lead revolt at that meeting soon resulted in a new intergovernmental negotiating committee completely separate from the IPCC and its parent bodies, and this committee then proceeded to replace the IPCC with its own subsidiary advisory body. Other difficulties related to the Houghton transformed of the assessment process only become apparent during the IPCC’s second and later assessments.

Have you ever wondered:

• Exactly who are the panelists on the Intergovernmental Panel on Climate Change?
• Is the IPCC the author of its own reports?
• Is it made up of scientists?

The answers to the last two questions are: ‘No’ it does not write its own reports, and; ‘No’ there are not many scientists on the panel.

What we call the IPCC is a revolving panel of government delegates, a few of whom have been scientists, others were science administrators, others had some science training. In short, over its 30 year history, few of the Panelists had much experience in the climate sciences. Not that that matters so much, because they do not undertake the IPCC assessments nor do they write the reports. Elected experts do that for them. All they do is ‘accept’ the expert reports and approve a summary.

This is why when the Panel shared the 2007 Nobel Peace Prize, it was so ridiculous that Michael Mann should claim to have shared the Nobel Prize with other scientists who undertook the assessment. Surely he knew that none of them were on the Panel! But his confusion also points to the peculiar design of the entire assessment process, which has contributed over the years to particular difficulties at both the political and scientific interfaces.

The transformation of the IPCC from its mandated design

In fact, the IPCC was originally supposed to write its own reports. The idea being that this would engage governments more fully in the scientific basis for response activities.

When the IPCC was first instituted at the 1987 World Meteorological Congress, theirs was not envisaged to be a particularly ambitious project, as one of those involved from the beginning later reflected:

What most of those who drew up the concept of the IPCC had in mind at the time was a group of, say, 40–50 climate experts nominated by 20–30 governments to produce a 30–40 page document which could be passed to all governments by the WMO and UNEP Secretariats to assist them (governments) in addressing the climate change issue in their national contexts. . .
[Zillman, Energy & Environment; 18(7): 869–892]

At the IPCC’s first meeting in 1988, the assessment framework was set up accordingly. They formed three working groups:

• WG1 The Scientific Basis
• WG2 Impacts
• WG3 Response Strategies.

Country delegations were assigned to each of them. The working groups then proceeded to schedule expert workshops to assist the delegates with their assessment. At least WG2 and WG3 did. The British chairman of WG1 had other ideas.

The UK had come to the first meeting with a winning bid to chair the scientific working group. Their proposal to put it under the guidance of the head of the UK Met Office, John Houghton, was backed by funding for a technical support unit of 3 or 4 staff. Houghton came to the position with ambitions to expand the assessment so that it involved the wider scientific community. He did this by electing ‘lead authors’ for individual chapters who were prominent experts in that field. To help give the report coherence, these chapter authors would come together for meetings where they would also help draft a summary. Once the report and summary were drafted, these would be sent around to the wider scientific community for expert review. The role of the panel was reduced to also reviewing the draft and, at the very end of the process, they would agree on a final version of the summary.

This assessment model was supported by the IPCC chairman, Bert Bolin, and gradually implemented across the other working groups as the assessment proceeded. Already by the second meeting there was no reason to assign particular delegations to particular working groups, and so full membership was open to all. In this way the intergovernmental panel structure was repeated in three subsidiary panels.

The consequences of the transformation

This transformation was all but completed before the (first) assessment was in, and it had some immediate effects. It allowed the panel to boast the involvement and authority of thousands of scientists who were experts in their field. But what it also did was shift power away from the panel and towards the IPCC ‘bureau’—essentially, towards the IPCC chair and the chairs of the working groups. To this day, the IPCC bureau is what is mostly taken for the IPCC.

Houghton and Bolin had replaced a representative system of country delegates with a meritocratic system of experts who were mostly residences and employees of rich western governments. The bureau itself was dominated in the same way (and this would continue at least until Bolin’s successor Bob Watson was removed under pressure from the USA when George W Bush arrived in the White House). This shift of power north and west caused tensions especially in WG3, which had a clear mandate develop the parameters of the climate treaty. Frustrated that they were not getting their way, a group of developing countries caused a revolt at the IPCC meeting where the assessment summary was supposed to be finalized. Their agitations nearly caused the meeting to collapse with no agreement. And no secret was made of their ultimate intentions.

They wanted the treaty process to be taken out of the hands of the IPCC and its parent bodies (UNEP and WMO), and for a separate negotiating committee to report directly to the UN General Assembly. They soon got their way. But for the IPCC, worse was to come. This new intergovernmental committee called for the establishment of its own subsidiary bodies for scientific and technical advice (SBSTA). This would eventually replace the IPCC in its exclusive advisory role. Thus, late in 1990 began its descent into a limbo where it had only a tentative, interim and terminal relationship with the treaty process. As it turned out, the IPCC did survive, but only after the extraordinary events during the second assessment where late changes to the IPCC report were used to justify a change of US climate policy (i.e., the Chapter 8 controversy).

Just how much the Houghton transformation of the assessment process contributed to these political problems is open to question. Much clearer is how the process he implement caused problems with the IPCC’s assessment of the science and then with the science itself.

Scientific assessment panels are nothing new. National Academies of science have a long history of setting up panels to assess controversial issues at government request, and protocols have long been established. Some of these protocols are to ensure that panelists do not have any perceived interest in the outcome of the assessment. This is why panelists are generally chosen who are scientifically competent in the field under review but yet not directly involved in the science and so not involved in the controversy. And then, if in the course of the assessment a panelist recognizes a conflict of interest, there are protocols to deal with that situation also. Other UN bodies like WHO have similar protocols.

No such protocols are evident in the IPCC assessment process. In fact, in Houghton’s model, it is leading experts in the field who undertake the assessment. There were a number of expressed reasons for this, the chief among them was based in concerns that many of the underlying fields of science were so underdeveloped and lacking in research that the lead authors needed to foster the research while also assessing it. Another reason was to ensure that the IPCC get access to the latest research. From the beginning of the IPCC, there was a strong ethos of prioritizing scientific currency over scientific scrutiny. This impatience for the latest science manifested in other ways.

Normally, an assessment could only use science that had survived exposure to criticism through peer review and publication. Even after publication there is usually great caution before confirmation by other studies. The slow and cautious processes towards the official uptake of science in the policy arena can be frustrating because it means new important findings may take years before they can have their impact on official assessments and so on policy. However, the advantage is that it allows time for scrutiny to reveal false leads. While newly published science has many false leads, unpublished findings have many more. However, in Houghton’s working group, a culture developed where it was acceptable to use unpublished work. Worse still, it became acceptable to use the unpublished work of the chapter authors. Already in the first assessment a spectacular precedence was set.

This happened at a lead author meeting convened after the report had been drafted, reviewed and redrafted. A leading modeller arrived with fresh results from new modelling that coupled the dynamics of the ocean with the dynamics of the atmosphere. It is hard to imagine how important this might have appeared at the time. But, with no empirical evidence to go on, with only primitive models that had embarrassing deficiencies, the science behind the proposed global warming mitigation treaty was looking rather weak. After much debate, additional sections were incorporated into the report with a citation to ‘private communications’.

Concerns over the use of unpublished (and non-peer reviewed) material would be repeatedly raised during the assessment process by both scientists and delegates. The response was mostly in attempting to speed up scrutiny during the actually assessment process. For example, drafts of unpublished work would be circulating, at least, on reviewer request. However, this was not permitted when an article was under publisher embargo while it awaited publication. But anyway, so it was argued, the debates during the assessment process (even the arguments about whether to even consider the unpublished findings) presents more than adequate peer review.

By the second assessment the problem was completely out of hand. During the review process one of the reviewers of the WG1 report, Pat Michaels, analyzed the citation authorities with the results given in Figure 2.

Figure 2: This is the World Climate Report survey results as reproduced in Searching for the Catastrophe Signal. The original article is available here

As it turned out, the three main controversies of this second assessment involved work of lead authors unpublished at the time of review (i.e., early 1995):

1. The Price of Life controversy revolved around the unpublished doctoral research of two young lead authors. (Tol and Fankhouser)
2. The refusal of the Met Office to release details of its model to a reviewer caused another controversy (John Mitchell refused Pat Michaels request for modelling data behind an unpublished paper)
3. The Chapter 8 Controversy revolved around the findings of two unpublished ‘detection’ papers by Santer and one skeptical paper by Tim Barnett (of Scripps).

It was not only Michaels who saw the warning signs during the second assessment. The coverage by Nature presents increasing concern.

During 1995, Nature had been following the IPCC story closely with news report that did not reflect well on the Panel. An editorial published on 23 November under the heading ‘Global Warming Rows’ warned that critics of the technical aspects of the IPCC’s assessment were developing a sense of exclusion from a ‘charmed circle’. Nature recommended that WG2 and WG3 be abandoned while the scientific working group were persuaded ‘to follow a more judicial course’. We now know that at the very time Nature was making this recommendation, the US government was pressing the IPCC WG1 to change its judgement on global warming ‘detection’. This change would be based on unpublished findings of the scientist leading on the ‘detection’ chapter who was also in the employ of the US government. If the removal of government delegates from the assessment process was supposed to mitigate against political interference, then by the second assessment this had already failed.

Conclusion

The Houghton-Bolin shift away from a smaller intergovernmental assessment towards an expanded expert-driven assessment had major consequences. Whatever one might say about the original design, the process they implemented during the first assessment shifted excessive power towards the expert authors, who had a clear vested interest in promoting their own views and their own work. Little effort was made to prevent this interest from influencing the assessment. Indeed, sometimes this influence was actively encouraged. As a consequence, there developed a ‘charmed circle’ culture (that would later mature into the bunker mentality evident in the climategate emails). This culture not only brings into question the impartiality of the assessment findings, but it can also be seen as an underlying cause of the controversies that came to plague the IPCC in later assessments.

–BernieL

________________________________________________________

A more detailed account of these early developments of the IPCC (with full referencing) can be found in Searching for the Catastrophe Signal.

________________________________________________________