Intercomparison and Validation (IV-TT)
The IV-TT designs, tests, and publishes advanced methods for verifying and validating the quality of operational ocean products with the aim to determine skills, strengths and weaknesses of ocean analysis and forecasting systems. The team coordinates and promotes scientific validation and intercomparison activities among operational centres, with the objective to establish best practices. As a legacy of GODAE projects, it pursues intercomparisons projects, where new methods are routinely proposed and tested.
The IV-TT has fostered the sharing of common approaches to the evaluation of operational oceanographic systems. These include the assessment of model errors and biases, the use of data assimilation metrics and forecast verification. New aspects have emerged over the past 10 years, such as the evaluation of prediction skill in a near-real time framework. Other aspects that have been investigated include: the unbalanced distribution of observations as “ground truth” (satellite vs in-situ, surface vs depth, high latitudes…), ensemble forecasting approaches, and the transition from standard error statistics to more sophisticated ocean process-oriented metrics. Evaluation methodologies that initially focused on physical parameters, now take into account sea-ice and biogeochemical products delivered by operational centres. Issues foreseen in the future include: the evaluation of complex coupled atmosphere-ice-ocean-wave systems; kilometer-scale systems that include sub-mesoscale processes, tides and coastal processes, and an extension toward social sciences (e.g. communication of product quality to non-expert users).
Ongoing since 2013, the intercomparison of IV-TT operational systems is carried out on “class 4 and class 1” metrics, where ocean analysis/forecasts are evaluated against observations or intercompared directly. These projects are now evolving to be carried out and supported by the Ocean Predict project for the UN Decade for Ocean Science.
The IV-TT is led by the co-chairs:
- Continue Class 4 global intercomparison on SST, Sea Level, T/S at depth, Sea-Ice concentration and surface velocity and extend this effort to incorporate regional centres.
- Extend surface velocity Class 4 to Lagrangian/dispersion metrics.
- Develop metrics for coupled ocean-atmosphere parameters and ensemble ocean forecasts.
- Design process and user-oriented metrics.
- Design and test metrics (Class4, Class1).
- Implement metrics and test routinely as part of the IV-TT intercomparison.
- Promote and publish intercomparison outcomes and propose best practices.
- Transfer metrics standard and methodology toward the operational community (e.g. JCOMM).
- Meet once a year to raise new issues on the team activity.
|No||First name||Last name||Affiliation||Country||Role|
|6||Marie||Drevillon||Mercator Ocean International||France||Member|
|18||Charly||Regnier||Mercator Ocean International||France||Member|
Johnny Johannessen, NERSC, Norway
Latest update, February 2021
Workshop and meetings
An important activity of the IV-TT is to organise regular workshops. In order to increase collaboration, these workshops are often held jointly with other TTs or groups. Planned and previous workshops include:
→ Workshop 2011 (in collaboration with the OSEval-TT)
The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.
Intercomparison and Validation Task Team (IV-TT) – October 2019
|Short description and objectives of the activities started or planned for this year:
||Accomplishments of the TT this year:
• Objective 1: Class 4 Intercomparison metrics project continuation
• Objective 2: Development of Class1 intercomparison metrics
• Objective 3: Contribution to the real time reanalysis intercomparison and ocean climate monitoring initiated by a CLIVAR-GSOP team
• Objective 4: Standardisation towards ET-OOFS
Objective 1: For the past two year, Class 4 for surface current have been under development, based on the provision of surface drifter trajectories and currents by the CMEMS, and processing by Mercator Océan International (MOI). A first set of test files were proposed and discussed during the OceanPredict’19 conference in Halifax during the short IV-TT meeting. Then at MOI, suggestions have been taken into account for a start of Class4 files provision on the US-GODAE server. A series of files starting mid-2018 is now uploaded and ready for processing for the Class 4 intercomparison. In parallel a technical document has been prepared.
Objective 2: No update
Objective 3: The NCEP and Bureau of Met. Sites are uploaded on a monthly basis, and bulletin produced regularly with intercomparison of 8 operational systems.
Objective 4: This past year, WMO has been re-organised, GOOS being officially a body of IOC. The JCOMM-ETOOFS panel is still under re-structuration. By the way, we still consider important that mature IV-TT activity should be endorsed and managed under a “ETOOFS-like” framework. During the OceanPredict’19 symposium, we presented to the Science Team or proposition. This proposition proposes to split IV-TT activities: the mature operation under WMO-IOC auspices, and the R&D under IV-TT OceanPredict. The direct benefit would be a commitment and resource allocation by operational centres for operational intercomparison, and a clearer dedication inside IV-TT for evaluation scientific developments.
|Future plans to continue/ improve current activities:
|Main future plans:
• Full comparison on surface velocities against drifter data. Consolidation of methods toward transfer to an operational panel (“ETOOFS-like”).
• The working tools for IV-TT are still not satisfactory: tools for regular exchanges, document repository are still necessary.
• IV-TT workshop being planned for 2020, back to back with the COSS-TT meeting in Montreal, Canada.
Objective 2: Actions to be determined
Objective 3: To be determined
Objective 4: Ask the Patron’s group to review our proposition for the future of IV-TT.
Objective 1: The reliability of the US-GODAE server is still an issue. The USA federal government policy, like the “shut-down” is totally paralysing the international activities. We consider important to offer some mirroring capability for real-time intercomparison activities.
Objective 4: Decision on re-definition of mature operational activities of intercomparison by IV-TT are strongly expected.