Skip to main content

Intercomparison and Validation (IV-TT)

Introduction

The IV-TT designs, tests, and publishes advanced methods for verifying and validating the quality of operational ocean products with the aim to determine skills, strengths and weaknesses of ocean analysis and forecasting systems. The team coordinates and promotes scientific validation and intercomparison activities among operational centres, with the objective to establish best practices. As a legacy of GODAE projects, it pursues intercomparisons projects, where new methods are routinely proposed and tested.

The IV-TT has fostered the sharing of common approaches to the evaluation of operational oceanographic systems. These include the assessment of model errors and biases, the use of data assimilation metrics and forecast verification. New aspects have emerged over the past 10 years, such as the evaluation of prediction skill in a near-real time framework. Other aspects that have been investigated include: the unbalanced distribution of observations as “ground truth” (satellite vs in-situ, surface vs depth, high latitudes…), ensemble forecasting approaches, and the transition from standard error statistics to more sophisticated ocean process-oriented metrics. Evaluation methodologies that initially focused on physical parameters, now take into account sea-ice and biogeochemical products delivered by operational centres. Issues foreseen in the future include: the evaluation of complex coupled atmosphere-ice-ocean-wave systems; kilometer-scale systems that include sub-mesoscale processes, tides and coastal processes, and an extension toward social sciences (e.g. communication of product quality to non-expert users).

Ongoing since 2013, the intercomparison of IV-TT operational systems is carried out on “class 4 and class 1” metrics, where ocean analysis/forecasts are evaluated against observations or intercompared directly. These projects are now evolving to be carried out and supported by the Ocean Predict project for the UN Decade for Ocean Science.

The IV-TT is led by the co-chairs:

Objectives

  • Continue Class 4 global intercomparison on SST, Sea Level, T/S at depth, Sea-Ice concentration and surface velocity and extend this effort to incorporate regional centres.
  • Extend surface velocity Class 4 to Lagrangian/dispersion metrics.
  • Develop metrics for coupled ocean-atmosphere parameters and ensemble ocean forecasts.
  • Design process and user-oriented metrics.

Activities

  • Design and test metrics (Class4, Class1).
  • Implement metrics and test routinely as part of the IV-TT intercomparison.
  • Promote and publish intercomparison outcomes and propose best practices.
  • Transfer metrics standard and methodology toward the operational community (e.g. JCOMM).
  • Meet once a year to raise new issues on the team activity.

Members

 

No First name Last name Affiliation Country Role
1 Laurent Bertino NERSC Norway Member
2 Gary Brassington BoM Australia Member
3 Emanuela Clementi CMCC Italy Member
4 Jim Cummings NRL USA Member
5 Prasanth Divakaran BoM Australia Member
6 Marie Drevillon Mercator Ocean International France Member
7 Yosuke Fujii MRI-JMA Japan Member
8 Fabrice Hernandez IRD/LEGOS France Co-chair
9 Patrick Hogan NOAA USA Member
10 Jae-Il Kwon KIOST Korea Member
11 Yvonnick La Clainche DFO Canada Member
12 Jan Maksymczuk Met Office UK Member
13 Matt Martin Met Office UK Member
14 Simona Masina INGV Italy Member
15 Avichal Mehra NOAA-NCEP USA Member
16 Peter Oke CSIRO Australia Member
17 Fangli Qiao FIO China Member
18 Charly Regnier Mercator Ocean International France Member
19 Pavel Sakov CSIRO Australia Member
20 Greg Smith ECCC Canada Co-chair
21 Kerry Smout-Day Met Office UK Member
22 Todd Spindler NOAA-NCEP USA Member
23 Clemente Tanajura UFBA Brazil Member
24 Liying Wan NMEFC China Member

 

Patron champions:

Johnny Johannessen, NERSC, Norway

 

Latest update, February 2021

Workshop and meetings

An important  activity of the IV-TT is to organise regular workshops. In order to increase collaboration, these workshops are often held jointly with other TTs or groups. Planned and previous workshops include:

→ Workshop 2016 (Montreal)

→ Workshop 2011 (in collaboration with the OSEval-TT)

TT quad-charts

The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.

 

Intercomparison and Validation Task Team (IV-TT) – October 2019
Short description and objectives of the activities started or planned for this year:
Accomplishments of the TT this year:
Main activities:

Objective 1: Class 4 Intercomparison metrics project continuation

Objective 2: Development of Class1 intercomparison metrics

Objective 3: Contribution to the real time reanalysis intercomparison and ocean climate monitoring initiated by a CLIVAR-GSOP team

Objective 4: Standardisation towards ET-OOFS

Main achievements:

Objective 1: For the past two year, Class 4 for surface current have been under development, based on the provision of surface drifter trajectories and currents by the CMEMS, and processing by Mercator Océan International (MOI). A first set of test files were proposed and discussed during the OceanPredict’19 conference in Halifax during the short IV-TT meeting. Then at MOI, suggestions have been taken into account for a start of Class4 files provision on the US-GODAE server. A series of files starting mid-2018 is now uploaded and ready for processing for the Class 4 intercomparison. In parallel a technical document has been prepared.

Objective 2: No update

Objective 3: The NCEP and Bureau of Met. Sites are uploaded on a monthly basis, and bulletin produced regularly with intercomparison of 8 operational systems.

Objective 4: This past year, WMO has been re-organised, GOOS being officially a body of IOC. The JCOMM-ETOOFS panel is still under re-structuration. By the way, we still consider important that mature IV-TT activity should be endorsed and managed under a “ETOOFS-like” framework. During the OceanPredict’19 symposium, we presented to the Science Team or proposition. This proposition proposes to split IV-TT activities: the mature operation under WMO-IOC auspices, and the R&D under IV-TT OceanPredict. The direct benefit would be a commitment and resource allocation by operational centres for operational intercomparison, and a clearer dedication inside IV-TT for evaluation scientific developments.

Future plans to continue/ improve current activities:
Issues/ problems:
Main future plans:

Objective 1:

• Full comparison on surface velocities against drifter data. Consolidation of methods toward transfer to an operational panel (“ETOOFS-like”).

• The working tools for IV-TT are still not satisfactory: tools for regular exchanges, document repository are still necessary.

• IV-TT workshop being planned for 2020, back to back with the COSS-TT meeting in Montreal, Canada.

Objective 2: Actions to be determined

Objective 3: To be determined

Objective 4: Ask the Patron’s group to review our proposition for the future of IV-TT.

Issues/problems:

Objective 1: The reliability of the US-GODAE server is still an issue. The USA federal government policy, like the “shut-down” is totally paralysing the international activities. We consider important to offer some mirroring capability for real-time intercomparison activities.

Objective 4: Decision on re-definition of mature operational activities of intercomparison by IV-TT are strongly expected.

Share