Process-oriented validation
of coupled chemistry-climate models

CCMVal 2005
CCMVal 2005


Download pdf-file.

Summary of the CCMVal 2005 Workshop on “Process-Oriented Validation of Coupled Chemistry-Climate Models”

V. Eyring (DLR, Germany), A. Gettelman (NCAR, USA), N.R.P. Harris (EORCU, UK), S. Pawson (NASA, USA), T.G. Shepherd (Univ. of Toronto, Canada), N. Butchart (Met Office, UK), M.P. Chipperfield (Univ. of Leeds, UK), M. Dameris (DLR, Germany), D.W. Fahey (NOAA, USA), P.M. de F. Forster (Univ. of Leeds, UK), P.A. Newman (NASA, USA), R.J. Salawitch (JPL, USA), B.D. Santer (LLNL, USA), and D.W. Waugh (JHU, USA)

The CCM Validation Activity for SPARC (CCMVal) is a response to the need for consistent evaluation and validation of coupled chemistry-climate models (CCMs) with detailed descriptions of the stratosphere, which have been developed over the last 5-10 years.  These CCMs provide valuable indications of how stratospheric ozone will evolve in the future as halogen concentrations decline in an atmosphere with a changing climate (e.g., WMO, 2003). The high complexity of CCMs requires a systematic evaluation process in order to demonstrate that the models are representative of the atmosphere and to quantify the uncertainty of the model results. 

The first CCMVal workshop was held in November 2003 in Grainau, Germany, to develop a more comprehensive approach to CCM validation. The concept was based on model inter-comparisons of the dynamical-radiative state such as those within the GCM-Reality Intercomparison Project for SPARC (GRIPS) (Pawson et al. (Au, 2000) and on an assessment of chemistry-climate models of the stratospherestin et al., 2003). The strategy developed was to identify the core processes that determine the stratospheric state and to select a number of diagnostics for each process within four main categories: dynamics, stratospheric transport, radiation, and stratospheric chemistry and microphysics.  Processes associated with the Upper Troposphere/Lower Stratosphere (UTLS) were also included under these categories.  A full description of the approach can be found in Eyring et al. (2005).  An essential part of the strategy is that the diagnostics would evolve over time, e.g., as new data sets or approaches become available.

A second CCMVal workshop was held at the National Center for Atmospheric Research (NCAR), Boulder, USA, on 17 – 19 October 2005.  The goals of the workshop were to assess progress in the validation of CCMs following the guidelines developed in the first CCMVal workshop and to assess how CCM model results can support upcoming UNEP/WMO and IPCC Assessments. Approximately 90 members of the atmospheric and climate communities from Europe, the United States, Canada, Japan, and New Zealand attended the workshop to take stock of progress and to identify near-term and long-term goals within the validation framework.  The attendees included representatives from nearly all the major stratospheric CCM groups in the world. The agenda and a list of participants can be found at the workshop’s website at

Main points of discussion
The introductory session reviewed the background context for the CCMVal activity, including WMO/UNEP and IPCC assessments, discussed related activities in the tropospheric climate modeling community, and emphasized the importance of understanding uncertainties in corroborative measurements.
The central part of the workshop consisted of oral and poster sessions on the progress made in the four main areas of CCMVal.  The presentations and the accompanying discussions showed that (a) good progress was being made in the evaluation of CCMs since the first CCMVal workshop; (b) the evaluation needs to be more quantitative in the future; and (c) a more detailed description of the individual diagnostics is necessary in order to make the table more practical and to allow individual groups to perform the diagnostics themselves.
Some analyses compared the results of several models with observed quantities based on model data submitted to the CCMVal/SCOUT-O3 database at the British Atmospheric Data Centre (BADC), others were ‘ad hoc’ inter-comparisons, while still other studies described evaluations of single models, often based on new diagnostics that they had developed.  While all approaches have their merits, the advantages and need for a central data archive to allow consistent analyses between models was clearly identified during the meeting. 
It is important to maximize the resources available to CCM groups.  Most of the meeting was spent discussing how to ensure that CCMs can be evaluated better and more consistently in the future, given the finite resources available to the stratospheric CCM groups.  Each diagnostic was considered in turn and most were refined considerably.  This was done in a number of ways.  In some cases precise descriptions of each diagnostic will be produced, specifying the method of calculation, the measurement set to be used for comparison, providing central software tools for more complicated diagnostics, etc.  In others, particularly for the chemistry assessment, individuals volunteered to analyze data placed on the database.  
The diagnostics were prioritized again according to whether they were considered to be: 1) core, 2) important, or 3) useful.  A core diagnostic was considered to be proven, straightforward to calculate, and important for illuminating the model processes.  An important diagnostic was important, but somewhat difficult to calculate or not well defined and requiring additional research.  Finally, a useful diagnostic was well defined and of importance, but only complementary to the core diagnostics. 
The core, important, and useful categorization of the diagnostics will be updated. This CCMVal process will allow for future diagnostics to be added to our current tables.  Additional new diagnostics will be added that illuminate key model processes.  In addition, current important and useful diagnostics will be reevaluated in response to modeling and research results.
In particular, considerable discussion was devoted at the workshop to two areas of great importance where further research is needed to define suitable core diagnostics: UTLS transport, and polar chemical ozone loss. These diagnostics are currently listed as important but it is expected that they will evolve into core diagnostics in the future. The up-to-date version of the CCMVal process table will be maintained on the CCMVal website (
Future plans
Several aspects of future plans related to CCMVal activities were actively discussed.  The plans relate to maintaining progress and awareness with CCMVal tasks, interacting with the broader atmospheric sciences and climate communities, and documenting the progress of CCMVal.  The following were considered of high priority:

·      Presentations at international scientific meetings.  Presenting the results of model inter-comparison activities is considered valuable for documenting the skill of CCMs and their improvements, for creating awareness of CCMVal activities and thereby entraining new participants, and for addressing the scientific understanding issues that have arisen in the model inter-comparisons.  Suggested meetings are those of the European Geophysical Union and the American Geophysical Union.
·      Documenting the progress of CCMVal. Progress matrices will be set up to document the state of the evaluation of the listed diagnostics and the participation of individual CCM groups.  Again, the up-to-date version of the progress matrices can be found at the CCMVal website.
·      Creating an Ensemble and Central Archive of CCM runs. A central archive of CCM model runs for the 20th and 21st centuries which can be used to assess model performance and to support upcoming WMO/UNEP and IPCC assessments has been created as part of CCMVal and the European Integrated Project SCOUT-O3. At some time in the future this archive will be made available to the community as an ‘ensemble of opportunity’.
·      CCMVal 2007 Workshop.  A third CCMVal workshop is tentatively planned for 2007.  The workshops have been very effective at bringing together climate modelers to discuss and plan for evaluation and validation activities.  The workshop goals at this early stage are: (i) show analysis of recent model results using CCMVal diagnostics, (ii) update CCMVal model diagnostics, (iii) review scientific results from the 2006 UNEP/WMO Scientific Assessment of Ozone Depletion, (iv) form an outline and a team to write a model evaluation report for SPARC, and (v) make recommendations for forcing scenarios that could support the expected 2010 UNEP/WMO assessment.
·      SPARC Report in 2008/2009.  A SPARC Report on CCMVal results was proposed for the 2008/2009 time period.  The Report would be a comprehensive summary of the progress and results obtained from CCM inter-comparisons and the use of CCMs in the ozone and climate assessment activities.  The Report would document the CCMVal approach and discuss the table of processes and diagnostics that have been developed and used over a period of years since the inception of CCMVal.  The Report would be peer-reviewed by the atmospheric sciences community.
·      In addition, the possibility of using some of the approaches developed for assessing climate models at PCMDI will be considered in order to make the evaluation more quantitative and to have a better understanding of the overall stratospheric CCM ensemble.
In summary, good progress was made during the second CCMVal workshop.  Several people agreed to take the lead for specific diagnostics and analyses, and it is hoped that all CCM groups will have joined in the inter-comparison by the next CCMVal workshop in 2007 so that a more quantitative evaluation will be reached. Participation in and comments on CCMVal are requested from the international community. For full details on CCMVal activities and contacts see
The 2005 Workshop was held under the auspices of the CCM Validation Activity for WCRP's (World Climate Research Programme) SPARC (Stratospheric Processes and their Role in Climate), the National Center for Atmospheric Research (NCAR), and the National Oceanic and Atmospheric Administration (NOAA). We thank Andrew Gettelman, David Fahey, Christina Book and Suzette Milano-Schoser for the local organization and all the workshop participants for their valuable contributions.
Austin, J., et al.  Uncertainties and assessments of chemistry-climate models of the stratosphere, Atmos. Chem. Phys., 3, 1-27, 2003.
Eyring V., et al.  , 2005: A strategy for process-oriented validation of coupled chemistry-climate models. Bull. Am. Meteorol. Soc., 86, 1117–1133.
Pawson, S., et al.  The GCM-Reality Intercomparison Project for SPARC: Scientific Issues and Initial Results, Bull. Am. Meteorol. Soc., 81, 781-796, 2000.
WMO, Scientific Assessment of Ozone Depletion: 2002, Global Ozone Research and Monitoring Project - Report No. 47, 498 pp, Geneva, 2003.