Observing System Evaluation (OS-Eval TT)

The OS-Eval TT is led by the co-chairs:

• Yosuke Fujii, MRI-JMA, Japan
• Elisabeth Remy, Mercator Ocean International, France

Objectives

OceanPredict centers, working on ocean monitoring and forecasting, make heavy use of the ocean observing systems to serve a wide range of oceanic applications, from marine safety to seasonal forecasts. The accuracy of the analysis and forecasts highly rely on the availability and quality of the in situ and satellite observations that are routinely assimilated.

To benefit the most from the observation to constrain model forecast requires regular assessment of the role of observations in constraining the ocean circulation with an efficient data assimilation strategy based on a good understanding of the observation content and errors.

On the basis of improved understanding of data utility, the TT provides consistent and scientifically justified requirements and feedbacks to organizations/agencies in charge of sustaining the Global and Regional Ocean Observing Systems (GOOS and ROOS) used for ocean monitoring and forecasting at short-range, seasonal and decadal time-scales.

 OS-Eval TT aims at building up a positive feedback cycle between observational agencies and operational centres for better fit for purpose In Situ and Satellite Ocean observing systems.

The TT is a forum for operational centers and research laboratories involved in Observing System Evaluation (OS-Eval) to share experiences on methods, best practices and results of OS-Eval activities.

Activities

OceanPredict makes heavy use of the world’s ocean observing system. OceanPredict has a role to play to demonstrate the uptake and impact of observations from GOOS and ROOS, and to contribute to the design of new observing systems.

To this end, OS-Eval TT plans to continually assess the impact of the physical and biogeochemical observing systems on ocean forecast and reanalysis systems. Similarly, OS-Eval TT plans to facilitate the design and assessment of new observing systems, to identify “gaps” in the GOOS and ROOS, and to propose optimized observation arrays on coastal, regional, and global scales.

Through OS-Eval TT – in partnership with OOPC, CLIVAR, and the other OceanPredict TTs: COSS-, MEAP-, DA-, and IV-TT – OceanPredict seeks to provide consistent, scientifically sound, evidence-based service-data requirements to agencies and organizations that sustain global, regional, and coastal ocean observing systems.

This activity requires clear protocols for observation impact assessment, common tools for routine production of metrics that quantify observation impacts and objective recommendation methodologies.

To reach this goal, OS-Eval TT:

• Build the capacity of assimilating new-type of observation and improve the capacity of extracting observational information
• Evaluate and help optimize existing ocean observing systems and contribute to the optimal design of future observing systems, in order to enhance their impacts on ocean (and coupled) predictions (both coastal/shelf and large scale)
• Develop best practices to evaluate or design ocean observing systems using ocean (and coupled) prediction systems
• Helps the transfer from research to operation of new methodologies,
• Provides support to operational agencies to adopt new methods for OS-Eval and better use of observation
• Enhance the communication between OceanPredict and observational communities and international organization linked to ocean observing network management.
• Provides consistent and scientifically justified requirements and feedbacks to agencies in charge of GOOS and ROOS.

A short term objective is to publish regularly Ocean Observation Impact Annual Report (OIAR) to provide clear information on use of observation data and their impacts in ocean (and coupled) predictions.

In practice, different approaches are developed for observing system evaluation and design. Here are the most commonly used to assess the role of specific observations in constraining the ocean circulation at regional or global scale:

• Observing System Experiments (OSE) which involves the denial of a sub-set of observations, and the evaluation of the degradation in quality of the resulting analyses and forecasts. The degradation quantifies the impact of the with-held observations.
• Observing System Simulation Experiment (OSSE) approach allows evaluating the impact on ocean analysis and forecast of future observing system by simulating observations from a known numerical ocean state.
• Specific diagnostics such as analysis self-sensitivities, forecast sensitivities, singular vector analysis, DFS, …

OS-Eval TT meets every 2 years, preferably jointly with other TT or relevant research communities to enhance the collaboration on cross cutting subjects. Regular TT web meetings are also organized.

Members

Membership include core members, who are regular participants in OS-Eval TT activities and associate members, who are involved in OS-Eval TT activities in an advisory capacity.

Last update : 06 Dec 2023

List of core members:

List of associated members:

Workshops and meetings

A major part of the TT’s work is to bring the science community together in TT meetings and workshops.  The OS-Eval TT holds larger workshops or symposia every 2-3 years, sometimes jointly with other OP task teams or external groups.

→  Joint OS-Eval TT/CP-TT workshop in Tsukuba, Japan, 15-18 Nov 2022 including the SynObs kick-off

Workshop history:

→ Joint DA-TT/OSEval-TT meeting 2017 (La Spezia)

→ Joint GOV-GSOP Meeting 2014 (Toulouse)

→ Joint GOV-GSOP Workshop 2011 (Santa Cruz)

Regular TT online meetings and SynObs related meetings

Since 2020 the TT is holding regular virtual meeting every 6-8 weeks:

Projects

The Observing System Evaluation Task Team (OS-Eval TT) is submitting a proposal for an UN Decade of Ocean Science project called SynObs in support of ForeSea, CoastPredict and other programmes.

SynObs

Meetings with the OceanPredict task teams on the support for SynObs are taking place:

• OS-Eval TT co-chairs called a first meeting:
  • Overview presentation (Yosuke Fujii, MRI-JMA)

Documents

Meeting reports:

• Report from the Joint DA-TT / OS-Eval TT meeting, CMRE, La Spezia, Italy, Oct 2017
• Report from the 2nd OS-Eval TT workshop, Toulouse, France, December 2014
• Report from the 1st OS-Eval TT meeting, Santa Cruz, USA, June 2011
• Report from the 2nd GODAE Observing System Evaluation Workshop, June 2009, Toulouse, France
• Report from the 1st OOPC-GODAE Meeting on OSSEs/OSEs, November 2007, Paris, France

Observations for operational use

The tables (link below) provide an overview of what observations are used for validation, input data (non-assimilated), assimilated data in research and operational mode at OceanPredict operational monitoring and forecasting centres.

Link to observation info tables

The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.

Observing System Evaluation Task Team (OSEval-TT) – May 2021