Contact us

Intercomparison and Validation (IV-TT)

Introduction

The IV-TT designs, tests, and publishes advanced methods for verifying and validating the quality of operational ocean products with the aim to determine skills, strengths and weaknesses of ocean analysis and forecasting systems. The team coordinates and promotes scientific validation and intercomparison activities among operational centres, with the objective to establish best practices. As a legacy of GODAE projects, it pursues intercomparisons projects, where new methods are routinely proposed and tested.

The IV-TT has fostered the sharing of common approaches to the evaluation of operational oceanographic systems. These include the assessment of model errors and biases, the use of data assimilation metrics and forecast verification. New aspects have emerged over the past 10 years, such as the evaluation of prediction skill in a near-real time framework. Other aspects that have been investigated include: the unbalanced distribution of observations as “ground truth” (satellite vs in-situ, surface vs depth, high latitudes…), ensemble forecasting approaches, and the transition from standard error statistics to more sophisticated ocean process-oriented metrics. Evaluation methodologies that initially focused on physical parameters, now take into account sea-ice and biogeochemical products delivered by operational centres. Issues foreseen in the future include: the evaluation of complex coupled atmosphere-ice-ocean-wave systems; kilometer-scale systems that include sub-mesoscale processes, tides and coastal processes, and an extension toward social sciences (e.g. communication of product quality to non-expert users).

Ongoing since 2013, the intercomparison of IV-TT operational systems is carried out on “class 4 and class 1” metrics, where ocean analysis/forecasts are evaluated against observations or intercompared directly. These projects are now evolving to be carried out and supported by the Ocean Predict project for the UN Decade for Ocean Science.

The IV-TT is led by the co-chairs:

Objectives

  • Continue Class 4 global intercomparison on SST, Sea Level, T/S at depth, Sea-Ice concentration and surface velocity and extend this effort to incorporate regional centres.
  • Extend surface velocity Class 4 to Lagrangian/dispersion metrics.
  • Develop metrics for coupled ocean-atmosphere parameters and ensemble ocean forecasts.
  • Design process and user-oriented metrics.

Activities

  • Design and test metrics (Class4, Class1).
  • Implement metrics and test routinely as part of the IV-TT intercomparison.
  • Promote and publish intercomparison outcomes and propose best practices.
  • Transfer metrics standard and methodology toward the operational community (e.g. JCOMM).
  • Meet once a year to raise new issues on the team activity.

Members

No First name Last name Affiliation Country Role
1 Laurent Bertino NERSC Norway Member
2 Gary Brassington BoM Australia Member
3 Andrea Cipollone CMCC Italy New Member
4 Emanuela Clementi CMCC Italy Member
5 Jim Cummings NRL USA Member
6 Prasanth Divakaran BoM Australia Member
7 Marie Drevillon Mercator Ocean International France Member
8 Yosuke Fujii MRI-JMA Japan Member
9 Fabrice Hernandez IRD/LEGOS France Co-chair
10 Patrick Hogan NOAA USA Member
11 Robert King MetOffice UK Member
12 Jae-Il Kwon KIOST Korea Member
13 Yvonnick La Clainche DFO Canada Member
14 Jan Maksymczuk Met Office UK Member
15 Matt Martin Met Office UK Member
16 Avichal Mehra NOAA-NCEP USA Member
17 Peter Oke CSIRO Australia Member
18 Fangli Qiao FIO China Member
19 Charly Regnier Mercator Ocean International France Member
20 Pavel Sakov CSIRO Australia Member
22 Greg Smith ECCC Canada Co-chair
23 Todd Spindler NOAA-NCEP USA Member
24 Clemente Tanajura UFBA Brazil Member
25 Liying Wan NMEFC China Member

 

 

Patron champions:

Johnny Johannessen, NERSC, Norway

 

Latest update: March 2023

Workshop, seminars and meetings

An important  activity of the IV-TT is to organise regular workshops. In order to increase collaboration, these workshops are often held jointly with other TTs or groups.

During the Covid-19 pandemic all meetings will be held online as virtual events.

IV-TT Online Meeting Series
Date MEAP-TT meetings Topics Recordings and Presentations
16 May 2024 IV-TT – Online Meeting (4)
  • Discussion regarding the use of the external ECCC computing cluster as a class4 mirror (and more).
  • Policy regarding the use and sharing of class4 data
  • Preparation of publication on Clas4 intercomparison (draft to be circulated shortly)
 IV-TT meeting recording and presentation
11 December 2023 IV-TT – Online Meeting (3)
  • Feedback from the 2023 OceanPredict Science Team Meeting (Busan, Korea)
  • Class4 Publication
  • Redefine IV-TT around science actions
  • Proposed IV-TT Seminar Series
 IV-TT meeting recording and presentation
18 April 2023 IV-TT – Online Meeting (2)
  • Status of present Class4 activity (USGODAE server, OPOS-WG)
  • Class4 Publication
  • Redefine IV-TT around science actions
  • UN decade
 IV-TT meeting recording and presentation
22 November 2021 IV-TT online meeting (1)
  • contributions for UN Decade (ForeSea, CoastPredict, SynObs, …)
  • updates on drift prediction
  • other class-4 and IV-TT issues
 IV-TT meeting recording and presentation
Ocean Verification Online Seminar Series
MEAP-TT seminars Date Topics Recordings and presentations
13 June 2024 IV-TT – 1st Seminar Verification and Intercomparison of Global Ocean Currents IV-TT Seminar recording and presentation

Previous meetings and workshops can be viewed here:

→ Workshop 2016 (Montreal)

→ Workshop 2011 (in collaboration with the OSEval-TT)

TT quad-charts

The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.

 

Intercomparison and Validation Task Team (IV-TT) – October 2019
Short description and objectives of the activities started or planned for this year:
 Accomplishments of the TT this year:
Main activities:

• Objective 1: Class 4 Intercomparison metrics project continuation

• Objective 2: Development of Class1 intercomparison metrics

• Objective 3: Contribution to the real time reanalysis intercomparison and ocean climate monitoring initiated by a CLIVAR-GSOP team

• Objective 4: Standardisation towards ET-OOFS

 Main achievements:

Objective 1: For the past two year, Class 4 for surface current have been under development, based on the provision of surface drifter trajectories and currents by the CMEMS, and processing by Mercator Océan International (MOI). A first set of test files were proposed and discussed during the OceanPredict’19 conference in Halifax during the short IV-TT meeting. Then at MOI, suggestions have been taken into account for a start of Class4 files provision on the US-GODAE server. A series of files starting mid-2018 is now uploaded and ready for processing for the Class 4 intercomparison. In parallel a technical document has been prepared.

Objective 2: No update

Objective 3: The NCEP and Bureau of Met. Sites are uploaded on a monthly basis, and bulletin produced regularly with intercomparison of 8 operational systems.

Objective 4: This past year, WMO has been re-organised, GOOS being officially a body of IOC. The JCOMM-ETOOFS panel is still under re-structuration. By the way, we still consider important that mature IV-TT activity should be endorsed and managed under a “ETOOFS-like” framework. During the OceanPredict’19 symposium, we presented to the Science Team or proposition. This proposition proposes to split IV-TT activities: the mature operation under WMO-IOC auspices, and the R&D under IV-TT OceanPredict. The direct benefit would be a commitment and resource allocation by operational centres for operational intercomparison, and a clearer dedication inside IV-TT for evaluation scientific developments.
Future plans to continue/ improve current activities:
 Issues/ problems:
Main future plans:

Objective 1:

• Full comparison on surface velocities against drifter data. Consolidation of methods toward transfer to an operational panel (“ETOOFS-like”).

• The working tools for IV-TT are still not satisfactory: tools for regular exchanges, document repository are still necessary.

• IV-TT workshop being planned for 2020, back to back with the COSS-TT meeting in Montreal, Canada.

Objective 2: Actions to be determined

Objective 3: To be determined

Objective 4: Ask the Patron’s group to review our proposition for the future of IV-TT.

 Issues/problems:

Objective 1: The reliability of the US-GODAE server is still an issue. The USA federal government policy, like the “shut-down” is totally paralysing the international activities. We consider important to offer some mirroring capability for real-time intercomparison activities.

Objective 4: Decision on re-definition of mature operational activities of intercomparison by IV-TT are strongly expected.

Share