Summary of the CCMVal 2005 Workshop
“Process-Oriented Validation of Coupled Chemistry-Climate Models”
(DLR, Germany), A. Gettelman (NCAR, USA), N.R.P. Harris (EORCU, UK), S.
(NASA, USA), T.G. Shepherd (Univ. of Toronto, Canada), N. Butchart (Met
UK), M.P. Chipperfield (Univ. of Leeds, UK), M. Dameris (DLR, Germany),
Fahey (NOAA, USA), P.M. de F. Forster (Univ. of Leeds, UK), P.A. Newman
USA), R.J. Salawitch (JPL, USA), B.D. Santer (LLNL, USA), and D.W.
Waugh (JHU, USA)
The CCM Validation Activity for SPARC
(CCMVal) is a response to the need for consistent evaluation and
coupled chemistry-climate models (CCMs) with detailed descriptions of
stratosphere, which have been developed over the last 5-10 years. These CCMs provide valuable
how stratospheric ozone will evolve in the future as halogen
decline in an atmosphere with a changing climate (e.g., WMO, 2003). The
complexity of CCMs requires a systematic evaluation process in order to
demonstrate that the models are representative of the atmosphere and to
quantify the uncertainty of the model results.
The first CCMVal workshop was held in
November 2003 in Grainau,
to develop a more
comprehensive approach to CCM validation. The concept was based on
model inter-comparisons of the dynamical-radiative
state such as those within the GCM-Reality Intercomparison Project for
SPARC (GRIPS) (Pawson
et al. (Au, 2000) and on an assessment
of chemistry-climate models of the stratospherestin et al.,
2003). The strategy developed
was to identify the core processes that determine the stratospheric
to select a number of diagnostics for each process within four main
dynamics, stratospheric transport, radiation, and stratospheric
microphysics. Processes associated with
the Upper Troposphere/Lower Stratosphere (UTLS) were also included
categories. A full description of the
approach can be found in Eyring et al.
(2005). An essential part of the
strategy is that the diagnostics would evolve over time, e.g., as new
or approaches become available.
CCMVal workshop was held at the National
Center for Atmospheric
(NCAR), Boulder, USA, on 17 – 19 October
goals of the workshop were to assess progress in the validation of CCMs
following the guidelines developed in the first CCMVal workshop and to
how CCM model results can support upcoming UNEP/WMO and IPCC
Approximately 90 members of the atmospheric and climate communities
Europe, the United States,
and New Zealand
attended the workshop to take stock of progress and to identify
long-term goals within the validation framework. The
attendees included representatives from
nearly all the major stratospheric CCM groups in the world. The agenda
list of participants can be found at the workshop’s website at http://www.pa.op.dlr.de/workshops/CCMVal2005/.
Main points of discussion
introductory session reviewed the background context for the CCMVal
IPCC assessments, discussed related activities in the tropospheric
climate modeling community, and emphasized the importance of
understanding uncertainties in corroborative measurements.
The central part of the workshop consisted
of oral and poster sessions on the progress made in the four main areas
CCMVal. The presentations and the
accompanying discussions showed that (a) good progress was being made
evaluation of CCMs since the first CCMVal workshop; (b) the evaluation
be more quantitative in the future; and (c) a more detailed description
is necessary in order to make the table more practical and to allow individual groups to perform the diagnostics themselves.
Some analyses compared the results of
several models with observed quantities based on model data submitted
CCMVal/SCOUT-O3 database at the British Atmospheric Data Centre (BADC),
were ‘ad hoc’ inter-comparisons, while still other studies described
evaluations of single models, often based on new diagnostics that they
developed. While all approaches have
their merits, the advantages and need for a central data archive to
consistent analyses between models was clearly identified during the
It is important to maximize the resources
available to CCM groups. Most of the
meeting was spent discussing how to ensure that CCMs can be evaluated
and more consistently in the future, given the finite resources
the stratospheric CCM groups. Each
diagnostic was considered in turn and most were refined considerably. This was done in a number of ways. In some cases precise descriptions of each
diagnostic will be produced, specifying the method of calculation, the
measurement set to be used for comparison, providing central software
more complicated diagnostics, etc. In
others, particularly for the chemistry assessment, individuals
analyze data placed on the database.
The diagnostics were prioritized again
according to whether they were considered to be: 1) core,
2) important, or 3)
useful. A core
diagnostic was considered to be proven, straightforward to
important for illuminating the model processes.
An important diagnostic was
important, but somewhat difficult to calculate or not well defined
requiring additional research. Finally,
a useful diagnostic was well defined
and of importance, but only complementary to the core diagnostics.
The core, important, and useful
categorization of the diagnostics will be updated. This CCMVal process
allow for future diagnostics to be added to our current tables. Additional new diagnostics will be added that
illuminate key model processes. In
addition, current important and useful diagnostics will be reevaluated
response to modeling and research results.
In particular, considerable discussion was
devoted at the workshop to two areas of great importance where further
is needed to define suitable core diagnostics: UTLS transport, and
chemical ozone loss. These diagnostics are currently listed as
important but it
is expected that they will evolve into core diagnostics in the future.
up-to-date version of the CCMVal process table will be maintained on
Several aspects of future plans related to CCMVal
actively discussed. The plans relate to
maintaining progress and awareness with CCMVal tasks, interacting with
broader atmospheric sciences and climate communities, and documenting
progress of CCMVal. The following were
considered of high priority:
at international scientific meetings. Presenting the results of
model inter-comparison activities
considered valuable for documenting the skill of CCMs and their
for creating awareness of CCMVal activities and thereby entraining new
participants, and for addressing the scientific understanding issues
arisen in the model inter-comparisons.
Suggested meetings are those of the European Geophysical Union
American Geophysical Union.
the progress of CCMVal. Progress
matrices will be set up to document the state of the evaluation of the
diagnostics and the participation of individual CCM groups. Again, the up-to-date version of the progress
matrices can be found at the CCMVal website.
an Ensemble and Central Archive of CCM runs. A central archive of CCM model runs for the 20th
centuries which can be used to assess model performance and to support
WMO/UNEP and IPCC assessments has been created as part of CCMVal and
European Integrated Project SCOUT-O3. At some time in the future this
will be made available to the community as an ‘ensemble of opportunity’.
2007 Workshop. A
third CCMVal workshop is tentatively
planned for 2007. The workshops have
been very effective at bringing together climate modelers to discuss
for evaluation and validation activities.
The workshop goals at this early stage are: (i) show analysis of
model results using CCMVal diagnostics, (ii) update CCMVal model
(iii) review scientific results from the 2006 UNEP/WMO Scientific
Ozone Depletion, (iv) form an outline and a team to write a model
report for SPARC, and (v) make recommendations for forcing scenarios
support the expected 2010 UNEP/WMO assessment.
Report in 2008/2009.
A SPARC Report on CCMVal results was proposed
for the 2008/2009 time period. The
Report would be a comprehensive summary of the progress and results
from CCM inter-comparisons and the use of CCMs in the ozone and climate
assessment activities. The Report would
document the CCMVal approach and discuss the table of processes and
that have been developed and used over a period of years since the
CCMVal. The Report would be
peer-reviewed by the atmospheric sciences community.
In addition, the
using some of the approaches developed for assessing climate models at
will be considered in order to make the evaluation more quantitative
have a better understanding of the overall stratospheric CCM ensemble.
good progress was made during the second CCMVal workshop.
Several people agreed to take the lead for specific diagnostics
and analyses, and it is hoped that all CCM groups will have joined in
the inter-comparison by the next CCMVal workshop in 2007 so that a more
quantitative evaluation will be reached. Participation in and comments
on CCMVal are requested from the international community. For full
details on CCMVal activities and contacts see http://www.pa.op.dlr.de/CCMVal/.
Workshop was held under the auspices of the CCM Validation Activity for
(World Climate Research Programme) SPARC (Stratospheric Processes and
Role in Climate), the National
Research (NCAR), and the National Oceanic and Atmospheric
(NOAA). We thank Andrew Gettelman, David Fahey, Christina Book and
Milano-Schoser for the local organization and all the workshop
their valuable contributions.
Austin, J., et al.
and assessments of chemistry-climate models of the stratosphere, Atmos. Chem. Phys., 3, 1-27, 2003.
Eyring V., et
, 2005: A strategy for process-oriented validation of coupled
chemistry-climate models. Bull. Am.
Meteorol. Soc., 86, 1117–1133.
Pawson, S., et
The GCM-Reality Intercomparison Project for SPARC: Scientific
Initial Results, Bull. Am. Meteorol. Soc.,
81, 781-796, 2000.
WMO, Scientific Assessment
of Ozone Depletion: 2002, Global Ozone Research and Monitoring Project
No. 47, 498 pp, Geneva,