Skip to main content

Observing System Evaluation (OS-Eval TT)

Introduction


OS-Eval TT supports observational communities by giving recommendation on observation system evolution from the OceanPredict perspective. The TT promotes studies assessing the observation data impacts in OceanPredict systems.

It also supports improvements for exploiting observational information more effectively and developments for assimilating new-type observation into OceanPredict systems. Thus, OS-Eval TT aims to create a positive feedback cycle between observational communities and OceanPredict centers to reinforce the value chain between the observations and the monitoring and forecasting systems based on data assimilation.

 

The OS-Eval TT is led by the co-chairs:

Objectives

OceanPredict centers, working on ocean monitoring and forecasting, make heavy use of the ocean observing systems to serve a wide range of oceanic applications, from marine safety to seasonal forecasts. The accuracy of the analysis and forecasts highly rely on the availability and quality of the in situ and satellite observations that are routinely assimilated.

To benefit the most from the observation to constrain model forecast requires regular assessment of the role of observations in constraining the ocean circulation with an efficient data assimilation strategy based on a good understanding of the observation content and errors.

On the basis of improved understanding of data utility, the TT provides consistent and scientifically justified requirements and feedbacks to organizations/agencies in charge of sustaining the Global and Regional Ocean Observing Systems (GOOS and ROOS) used for ocean monitoring and forecasting at short-range, seasonal and decadal time-scales.

 OS-Eval TT aims at building up a positive feedback cycle between observational agencies and operational centres for better fit for purpose In Situ and Satellite Ocean observing systems.

The TT is a forum for operational centers and research laboratories involved in Observing System Evaluation (OS-Eval) to share experiences on methods, best practices and results of OS-Eval activities.

Activities

OceanPredict makes heavy use of the world’s ocean observing system. OceanPredict has a role to play to demonstrate the uptake and impact of observations from GOOS and ROOS, and to contribute to the design of new observing systems.

To this end, OS-Eval TT plans to continually assess the impact of the physical and biogeochemical observing systems on ocean forecast and reanalysis systems. Similarly, OS-Eval TT plans to facilitate the design and assessment of new observing systems, to identify “gaps” in the GOOS and ROOS, and to propose optimized observation arrays on coastal, regional, and global scales.

Through OS-Eval TT – in partnership with OOPC, CLIVAR, and the other OceanPredict TTs: COSS-, MEAP-, DA-, and IV-TT – OceanPredict seeks to provide consistent, scientifically sound, evidence-based service-data requirements to agencies and organizations that sustain global, regional, and coastal ocean observing systems.

This activity requires clear protocols for observation impact assessment, common tools for routine production of metrics that quantify observation impacts and objective recommendation methodologies.

To reach this goal, OS-Eval TT:

  • Build the capacity of assimilating new-type of observation and improve the capacity of extracting observational information
  • Evaluate and help optimize existing ocean observing systems and contribute to the optimal design of future observing systems, in order to enhance their impacts on ocean (and coupled) predictions (both coastal/shelf and large scale)
  • Develop best practices to evaluate or design ocean observing systems using ocean (and coupled) prediction systems
  • Helps the transfer from research to operation of new methodologies,
  • Provides support to operational agencies to adopt new methods for OS-Eval and better use of observation
  • Enhance the communication between OceanPredict and observational communities and international organization linked to ocean observing network management.
  • Provides consistent and scientifically justified requirements and feedbacks to agencies in charge of GOOS and ROOS.

A short term objective is to publish regularly Ocean Observation Impact Annual Report (OIAR) to provide clear information on use of observation data and their impacts in ocean (and coupled) predictions.

In practice, different approaches are developed for observing system evaluation and design. Here are the most commonly used to assess the role of specific observations in constraining the ocean circulation at regional or global scale:

  • Observing System Experiments (OSE) which involves the denial of a sub-set of observations, and the evaluation of the degradation in quality of the resulting analyses and forecasts. The degradation quantifies the impact of the with-held observations.
  • Observing System Simulation Experiment (OSSE) approach allows evaluating the impact on ocean analysis and forecast of future observing system by simulating observations from a known numerical ocean state.
  • Specific diagnostics such as analysis self-sensitivities, forecast sensitivities, singular vector analysis, DFS, …

OS-Eval TT meets every 2 years, preferably jointly with other TT or relevant research communities to enhance the collaboration on cross cutting subjects. Regular TT web meetings are also organized.

Members

Membership include core members, who are regular participants in OS-Eval TT activities and associate members, who are involved in OS-Eval TT activities in an advisory capacity.

Last update : 06 Dec 2023

List of core members:

No Name Last name Affiliation Country Role
 1 Laurent Bertino NERSC Norway Member
 2 Paul Biswamoy INCOIS India Member
 3 Antonio Bonaduce NERSC Norway Member
 4  Jim Cummings IMSG at NCEP/EMC USA Member
 5 Yosuke Fujii JMA/MRI Japan Co-chair
 6 Florent Gasparin Mercator Ocean France Member
 7 Stephanie Guinehut CLS France Member
 8 Eric Hackert NASA USA Member
 9 Patrick Heimbach University of Texas at Austin USA Member
10 Young-Ho Kim KORDI South Korea Member
11 Gilles Larnicol CLS France Former co-chair
12 Matthieu Le Henaff NOAA USA Member
13 Daniel Lea Met Office UK Member
14 Shuhei Masuda JAMSTEC Japan Member
15 Peter Oke CSIRO Australia Former co-chair
16 Shastri Paturi NOAA/NCEP USA Member
17 Elisabeth Remy Mercator Ocean France Co-chair
18 Gregory Smith Environment Canada Canada Member (co-chair of the IV-TT)
19 Clemente Tanajura REMO Brazil Member
20 Zhaoyi Wang NEMFC China New member
21 Jennie Waters Met Office UK Member
22 Hao Zuo ECMWF UK Member

 

List of associated members:

No Name Affiliation Role
1 Bayler, Eric NOAA OPAS champion
2 DiGiacomo, Paul NOAA OPAS chair
3 Donlon, Craig ESA OPAS champion
4 Lee, Tong JPL/NASA Expert
5 Schiller, Andreas CSIRO Expert

 

 

Workshops and meetings

A major part of the TT’s work is to bring the science community together in TT meetings and workshops.  The OS-Eval TT holds larger workshops or symposia every 2-3 years, sometimes jointly with other OP task teams or external groups.

 

 

 Joint OS-Eval TT/CP-TT workshop in Tsukuba, Japan, 15-18 Nov 2022 including the SynObs kick-off

Workshop history:

Joint DA-TT/OSEval-TT meeting 2017 (La Spezia)

Joint GOV-GSOP Meeting 2014 (Toulouse)

Joint GOV-GSOP Workshop 2011 (Santa Cruz)

 

Regular TT online meetings and SynObs related meetings

Since 2020 the TT is holding regular virtual meeting every 6-8 weeks:

OS-Eval TT meetings Date Topics Location
SynObs (OS-Eval) web meeting 1 23 Jan 2023, 14:00 – 15:00 UTC TBC Virtual, Zoom (invitations by email)
SynObs (OS-Eval) web meeting 2 31 Jan 2023, 21:00 – 22:00 UTC TBC Virtual, Zoom (invitations by email)
15th OS-Eval TT online meeting 23 June 2022
  • Presentation on the adjoint sensitivity studies in the ECCO group
  • SynObs
  • Flagship collaborative on OSEs in SynObs and possible OSSE collaboration with partners
Virtual, MS Teams
14th OS-Eval TT online meeting  25 March 2022 The agenda:

  • SynObs project progress and plans
  • OS-Eval/CP-TT symposium, Nov 2022
  • AOB
Virtual, MS Teams
13th OS-Eval TT online meeting 24 Jan 2022 The agenda (tentative):

  • Report on the OSEs for the Argo fast salinity drift (by Y. Fujii) and discussion
  • Announcement on the submission of SynObs and report/discussion for the launch of SynObs
Virtual, MS Teams
12th OS-Eval TT online meeting 23 Nov 2021
  • Fishing vessel observations, deep Argo data reporting, SynObs application to the UN Decade
Virtual meeting, MS Teams
11th OS-Eval TT online meeting 28 Sep 2021
  • ESA, SynObs plans, general TT discussion
Virtual meeting, MS Teams
10th OS-Eval TT online meeting 20 July 2021
  • OPST-4 report, SybObs plans and Argo (fast salinity drifts)
Virtual meeting, MS Teams
9th OS-Eval-TT online meeting  28 April 2021
  • Presentations on Argo Salinity drift
Virtual meeting, MS Teams
8th OS-Eval-TT online meeting 16 March 2021
  • Presentations on National efforts wrt observing system experiments (OSE) and observing system simulation experiments (OSSE)
Virtual meeting, MS Teams
1st – 7th OS-Eval-TT online meetings

(details available from the GODAE OceanView website)

  • 8 Jan 2021 (7th)
  • 3 Dec 2020 (6th)
  • 13 Oct 2020 (5th)
  • 8 Sep 2020 (4th)
  • 28 Jul 2020 (3rd)
  • 22 Jun 2020 (2nd)
  • 11 May 2020 (1st)
  • Various
Virtual meeting, MS Teams

 

Projects

The Observing System Evaluation Task Team (OS-Eval TT) is submitting a proposal for an UN Decade of Ocean Science project called SynObs in support of ForeSea, CoastPredict and other programmes.

 

SynObs

Meetings with the OceanPredict task teams on the support for SynObs are taking place:

Documents

Meeting reports:

Observations for operational use

The tables (link below) provide an overview of what observations are used for validation, input data (non-assimilated), assimilated data in research and operational mode at OceanPredict operational monitoring and forecasting centres.

Link to observation info tables

 

The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.

 

Observing System Evaluation Task Team (OSEval-TT) – May 2021
Short description and objectives of the activities started or planned for this year:
Accomplishments of the TT this year:
Main activities:

  • Exchange on ongoing OSEval activities in the different centers, discuss methods, results and identify future challenges
  • Provide a forum for discussion: regular online meetings (every 2 months, 1 hour each)
  • Lead the discussion on Proposing the UN Decade Project “SynObs”
  • Participation to meetings to present results of OSEval and promote such activities
Main achievements:

  • Multi system Obs. System Evaluation:
    • SWOT, ESA SMOS, H2020 AtlantOS project (finished) were numerous European partners are involved.
    • TPOS2020 project is finished. (JMA, US, Mercator)
  • Generate the table of observation use in OP systems
  • Set-up regular online meetings to discuss ongoing activities across the TT.
  • Co-chairs attend relevant meetings (GSOP, OSOS, GOOS, TPOS, etc.) and have a discussion on the collaboration
Future plans to continue/ improve current activities:
Issues/ problems:
Main future plans:

  • Submit the Decade Project SynObs to the UN Decade of Ocean Science
  • Collaborate with GOOS and other observational communities based on SynObs. And contribute to the authorized ocean observation requirement report planned in the UN Decade Programme proposed by GOOS Observing Coordination Group.
  • Conduct multi-system OSEs for the fast salinity drift of Argo floats
  • Plan to organize a TT Meeting (jointed to CP-TT) in Nov. 30-Dec. 3rd, 2021, in Tsukuba, Japan
  • Updating the webpage. (Update the information on the references)
Main issues:

  • How to build up the cooperative framework between OceanPredict and observation communities for SynObs
  • How to better communicate on OSE/OSSE, their use and limitations (definition of best practices)
  • How to evaluate the status of current observing system in real-time (NRT-OSE is expensive and can be implemented in few institute)
  • How to compile the report on ocean observation impacts

Share