Skip to main content

Observing System Evaluation (OS-Eval TT)


OS-Eval TT supports observational communities by giving recommendation on observation system evolution from the OceanPredict perspective. The TT promotes studies assessing the observation data impacts in OceanPredict systems.

It also supports improvements for exploiting observational information more effectively and developments for assimilating new-type observation into OceanPredict systems. Thus, OS-Eval TT aims to create a positive feedback cycle between observational communities and OceanPredict centers to reinforce the value chain between the observations and the monitoring and forecasting systems based on data assimilation.


The OS-Eval TT is led by the co-chairs:


OceanPredict centers, working on ocean monitoring and forecasting, make heavy use of the ocean observing systems to serve a wide range of oceanic applications, from marine safety to seasonal forecasts. The accuracy of the analysis and forecasts highly rely on the availability and quality of the in situ and satellite observations that are routinely assimilated.

To benefit the most from the observation to constrain model forecast requires regular assessment of the role of observations in constraining the ocean circulation with an efficient data assimilation strategy based on a good understanding of the observation content and errors.

On the basis of improved understanding of data utility, the TT provides consistent and scientifically justified requirements and feedbacks to organizations/agencies in charge of sustaining the Global and Regional Ocean Observing Systems (GOOS and ROOS) used for ocean monitoring and forecasting at short-range, seasonal and decadal time-scales.

 OS-Eval TT aims at building up a positive feedback cycle between observational agencies and operational centres for better fit for purpose In Situ and Satellite Ocean observing systems.

The TT is a forum for operational centers and research laboratories involved in Observing System Evaluation (OS-Eval) to share experiences on methods, best practices and results of OS-Eval activities.


OceanPredict makes heavy use of the world’s ocean observing system. OceanPredict has a role to play to demonstrate the uptake and impact of observations from GOOS and ROOS, and to contribute to the design of new observing systems.

To this end, OS-Eval TT plans to continually assess the impact of the physical and biogeochemical observing systems on ocean forecast and reanalysis systems. Similarly, OS-Eval TT plans to facilitate the design and assessment of new observing systems, to identify “gaps” in the GOOS and ROOS, and to propose optimized observation arrays on coastal, regional, and global scales.

Through OS-Eval TT – in partnership with OOPC, CLIVAR, and the other OceanPredict TTs: COSS-, MEAP-, DA-, and IV-TT – OceanPredict seeks to provide consistent, scientifically sound, evidence-based service-data requirements to agencies and organizations that sustain global, regional, and coastal ocean observing systems.

This activity requires clear protocols for observation impact assessment, common tools for routine production of metrics that quantify observation impacts and objective recommendation methodologies.

To reach this goal, OS-Eval TT:

  • Build the capacity of assimilating new-type of observation and improve the capacity of extracting observational information
  • Evaluate and help optimize existing ocean observing systems and contribute to the optimal design of future observing systems, in order to enhance their impacts on ocean (and coupled) predictions (both coastal/shelf and large scale)
  • Develop best practices to evaluate or design ocean observing systems using ocean (and coupled) prediction systems
  • Helps the transfer from research to operation of new methodologies,
  • Provides support to operational agencies to adopt new methods for OS-Eval and better use of observation
  • Enhance the communication between OceanPredict and observational communities and international organization linked to ocean observing network management.
  • Provides consistent and scientifically justified requirements and feedbacks to agencies in charge of GOOS and ROOS.

A short term objective is to publish regularly Ocean Observation Impact Annual Report (OIAR) to provide clear information on use of observation data and their impacts in ocean (and coupled) predictions.

In practice, different approaches are developed for observing system evaluation and design. Here are the most commonly used to assess the role of specific observations in constraining the ocean circulation at regional or global scale:

  • Observing System Experiments (OSE) which involves the denial of a sub-set of observations, and the evaluation of the degradation in quality of the resulting analyses and forecasts. The degradation quantifies the impact of the with-held observations.
  • Observing System Simulation Experiment (OSSE) approach allows evaluating the impact on ocean analysis and forecast of future observing system by simulating observations from a known numerical ocean state.
  • Specific diagnostics such as analysis self-sensitivities, forecast sensitivities, singular vector analysis, DFS, …

OS-Eval TT meets every 2 years, preferably jointly with other TT or relevant research communities to enhance the collaboration on cross cutting subjects. Regular TT web meetings are also organized.


Membership include core members, who are regular participants in OS-Eval TT activities and associate members, who are involved in OS-Eval TT activities in an advisory capacity.

List of core members:

No Name Last name Affiliation Country Role
 1 Laurent Bertino NERSC Norway Member
 2 Paul Biswamoy INCOIS India Member
 3 Antonio Bonaduce NERSC Norway Member
 4  Jim Cummings IMSG at NCEP/EMC USA Member
 5 Yosuke Fujii JMA/MRI Japan Co-chair
 6 Florent Gasparin Mercator Ocean France Member
 7 Stephanie Guinehut CLS France Member
 8 Eric Hackert NASA USA Member
 9 Patrick Heimbach University of Texas at Austin USA Member
10 Young-Ho Kim KORDI South Korea Member
11 Robert King Met Office UK Member
12 Villy Kourafalou University of Miami / RSMAS USA Member (co-chair of the COSS-TT)
13 Gilles Larnicol CLS France Former co-chair
14 Matthieu Le Henaff NOAA USA Member
15 Daniel Lea Met Office UK Member
16 Shuhei Masuda JAMSTEC Japan Member
17 Peter Oke CSIRO Australia Former co-chair
18 Shastri Paturi NOAA/NCEP USA Member
19 Elisabeth Remy Mercator Ocean France Co-chair
20 Gregory Smith Environment Canada Canada Member (co-chair of the IV-TT)
21 Clemente Tanajura REMO Brazil Member
22 Tianyu Zhang NMEFC China Member
23 Hao Zuo ECMWF UK Member


List of associated members:

No Name Affiliation Role
1 Bayler, Eric NOAA OPAS champion
2 DiGiacomo, Paul NOAA OPAS chair
3 Donlon, Craig ESA OPAS champion
4 Lee, Tong JPL/NASA Expert
5 Schiller, Andreas CSIRO Expert

Status: Feb 2021


Meetings and workshops

A major part of the TT’s work is to bring the science community together in TT meetings and workshops.  The OS-Eval TT holds larger workshops and meetings every 2-3 years often involving other OceanPredict Task Teams and external research groups who can contribute to the meeting objectives. Since early last year (due to Covid) the TT has introduced regular virtual meeting every 6 weeks.


4th OS-Eval TT meeting, Tsukuba Center for Institutes, Tsukuba, Japan (near Tokyo), Nov 2021


Meeting history:

Joint DA-TT/OSEval-TT meeting 2017 (La Spezia)

Joint GOV-GSOP Meeting 2014 (Toulouse)

Joint GOV-GSOP Workshop 2011 (Santa Cruz)

Online meetings

From May 2020 onwards the OS-Eval TT* started holding regular online Task Team meetings to discuss TT activities and future plans.

                                                OS-Eval TT online meeting attendees – 28 July 2020


Link to Observation data table

Date Description Participants Presentations Meeting summary & recordings
April, 28 2021 9th online meeting of the OS-Eval TT TBC Agenda TBC
March, 16, 2021 8th online meeting of the OS-Eval TT Santha Akella, Eric Bayler, Paul Biswamoy, Fraser Davidson, Paul DiGiacomo, Craig Donlon, Yosuke Fujii, Audrey-Anne, Gauthier, Stephanie Guinehut, Eric C Hackert, Patrick Heimbach, YoungHo Kim, Villy Kourafalou, Matthieu Le Henaff, Elisabeth Remy, Paturi Shastri, Greg Smith, Clemente Tanajura, David Trossman, Kirsten Wilmer-Becker, Jiping Xie, Tianyu Zhang, Hao Zuo Agenda

Overview presentation (Yosuke Fujii, Elisabeth Remy)

Regional ocean modeling in support of hurricane forecast improvements (Matthieu Le Henaff, Uni. of Miami-CIMAS, NOAA-AOML)

Benefits of assimilating the SMOS SSS in the Arctic reanalysis (Jiping Xi, NERSC)

On the activities in KIOST’s ocean and climate prediction system development (YoungHo Kim, KIOST)

Bluelink ReANalysis, version 2020 (BRAN2020): impact of Near-Real-Time data (Peter Oke, CSIRO)

Plans for Arctic OSE and SWOT OSSE in Canadian Analysis Systems (Greg Smith, ECCC)

To access the meeting recording please visit the OS-Eval TT restricted area
Jan, 8, 2021 7th online meeting of the OS-Eval TT see minutes Agenda
1. Introduction of the current status on the contributions of OP communities to the UN Decade, including the OP proposal for the UN Decade programme
2. Discussion on the proposal of the OS-Eval projectOverview presentation OS-Eval TT UN Decade proposal (Yosuke Fujii)
 Meeting minutes
Dec, 3, 2020 6th online meeting of the OS-Eval TT
Oct,13, 2020 5th online meeting of the OS-Eval TT  See minutes Agenda

Recent OS-Eval Activity and future plans in JMA/MRI (Norihisa Usui, Yosuke Fujii)

Project proposals

Sep, 8, 2020 4th online meeting of the OS-Eval TT John Siddorn, YougHo Kim, Clemente Tanajura, Patrick Heimbach, Shuhei Masuda, Eric Hackert, Peter Oke, Yosuke Fujii, Elisabeth Remy and Kirsten Wilmer-becker TT-agenda and overview talk (Yosuke Fujii, Elisabeth Remy)

GOOS & OceanPredict   (John Siddorn, Met Office)

Link to Observation Data Table

 Minutes (pdf)
July, 28, 2020 3rd online meeting of the OS-Eval-TT Yosuke Fujii, Elisabeth Remy, Clemente Tanajura, Peter Oke, YoungHo Kim, Robert King, Paul Biswamoy, Shuhei Masuda, Eric Bayler, Arun Kumar and Fraser Davidson TT introduction presentation (Yosuke, Fujii, Elisabeth Remy)

Assimilation of Synthetic Salinity and OSEs with XBT and SSS Data in the South Atlantic with HYCOM+RODAS (Clemente Tanajura)

 Summary pdf
June 22, 2020 2nd online meeting of the OSEval-TT Yosuke Fujii, Elisabeth Remy, Peter Oke, Clemente Tanajura, Shastri Paturi, YoungHo Kim, Eric Hackert, Rob King, Shuhei Masuda, Arun Kumar, Hao Zou Overview presentation/intro (Yosuke Fujii)

Argo-to-OcePred (Peter Oke)

– Introduction to the Argo programme, invitation for joint Argo/OP meetings (next possibility March 2021, Moncao) (presentation from Peter Oke)

– consideration of working with Argo; about use of deep and BGC Argo data, advocay for use, demonatraiton of impact; help with trajectories under-ice; use of data at park depth; withholding experiments of old floats, array design (double density regions (questions in Peter’s talk)

– overview of OP science team meeting, strategy and implementation; questions bout what OSE/OSSEs are in progress in coastal areas, other

May 11, 2020 First online meeting of the OSEval-TT members to discuss Yosuke Fujii, Elisabeth Remy, Clemente Tanajura, Greg Smith, Hao Zou, Eric Hackert, Fraser Davidson, Jim Cummings, Shastri Paturi, Kirsten Wilmer-Becker TT meeting intro (Yosuke Fujii)

Jason-3 EoL OP recommendations (Elisabeth Remy)

– Agreement to run regular OSEval-TT meetings (frequency tbc)

– Publish table that shows use of observation data in DA systems

– For next meetings: provide talks and other documents ahead of the meeting

* Please note that the task team has updated is notation to OS-Eval TT to better reflect the Evaluation aspect of the TT remit.





The OS-Eval TT will be contributing to the UN Decade of Ocean Science, through collaboration with OceanPredict and CoastPredict for which the TT submitted and decade project.


Meeting reports:

The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.


Observing System Evaluation Task Team (OSEval-TT) – October 2019
Short description and objectives of the activities started or planned for this year:
Accomplishments of the TT this year:
Main activities:

1. Coordination of the TT

2. Co-chairs attended GOV-ST VIII and discuss on collaboration with other TTs

3. Generate a community white paper of OceanObs19’ on ocean observing system evaluation. Support other white papers associated with the TT activities.

4. Contribute to the session on assimilation of salinity data in the Ocean Salinity Science Conference

5. Co-chairs attend relevant meetings (25th Altimetry, Argo TPOS2020, WCRP, etc.) and appeal the TT activities

Main achievements:

1. OceanObs’19 community white paper on OSEval

2. Observing Impact Statement (OIS) for ESA-SMOS published

3. Multi system Obs. System Evaluation at international level:

• ESA SMOS studies(CLS, Met Office, Mercator)

• H2020 AtlantOS project (numerous  European partners)

• SWOT impact studies ongoing (US, Canada, France)

• TPOS2020 project ongoing (JMA, US Mercator)

Future plans to continue/ improve current activities:
Issues/ problems:
Main future plans:

1. Continuing the Collaboration with TPOS2020.

2. Plan to collaborate with international Argo ST for Argo network evaluation.

3. Update the TT’s homepage and improve the visibility of OSEval activities within the OP centers, and enhance communication among members

4. Regular publication of the Ocean Observation Impact Annual Report (instead of OIS).

5. Plan to organize a TT Meeting (jointed to CP-TT) in Nov. 10-13th, 2020, in Tsukuba, Japan

Main issues:

• How to strengthen the communication with observational agency

• How to communicate on OSE/OSSE, their use and limitations

• How to evaluate the status of current observing system in real-time (NRT-OSE is expensive and can be implemented in few institute)

• How to Extend the TT activity to international (Currently activity is mostly limited in EU countries)