Observing System Evaluation (OS-Eval TT)
OS-Eval TT supports observational communities by giving recommendation on observation system evolution from the OceanPredict perspective. The TT promotes studies assessing the observation data impacts in OceanPredict systems.
It also supports improvements for exploiting observational information more effectively and developments for assimilating new-type observation into OceanPredict systems. Thus, OS-Eval TT aims to create a positive feedback cycle between observational communities and OceanPredict centers to reinforce the value chain between the observations and the monitoring and forecasting systems based on data assimilation.
The OS-Eval TT is led by the co-chairs:
OceanPredict centers, working on ocean monitoring and forecasting, make heavy use of the ocean observing systems to serve a wide range of oceanic applications, from marine safety to seasonal forecasts. The accuracy of the analysis and forecasts highly rely on the availability and quality of the in situ and satellite observations that are routinely assimilated.
To benefit the most from the observation to constrain model forecast requires regular assessment of the role of observations in constraining the ocean circulation with an efficient data assimilation strategy based on a good understanding of the observation content and errors.
On the basis of improved understanding of data utility, the TT provides consistent and scientifically justified requirements and feedbacks to organizations/agencies in charge of sustaining the Global and Regional Ocean Observing Systems (GOOS and ROOS) used for ocean monitoring and forecasting at short-range, seasonal and decadal time-scales.
OS-Eval TT aims at building up a positive feedback cycle between observational agencies and operational centres for better fit for purpose In Situ and Satellite Ocean observing systems.
The TT is a forum for operational centers and research laboratories involved in Observing System Evaluation (OS-Eval) to share experiences on methods, best practices and results of OS-Eval activities.
OceanPredict makes heavy use of the world’s ocean observing system. OceanPredict has a role to play to demonstrate the uptake and impact of observations from GOOS and ROOS, and to contribute to the design of new observing systems.
To this end, OS-Eval TT plans to continually assess the impact of the physical and biogeochemical observing systems on ocean forecast and reanalysis systems. Similarly, OS-Eval TT plans to facilitate the design and assessment of new observing systems, to identify “gaps” in the GOOS and ROOS, and to propose optimized observation arrays on coastal, regional, and global scales.
Through OS-Eval TT – in partnership with OOPC, CLIVAR, and the other OceanPredict TTs: COSS-, MEAP-, DA-, and IV-TT – OceanPredict seeks to provide consistent, scientifically sound, evidence-based service-data requirements to agencies and organizations that sustain global, regional, and coastal ocean observing systems.
This activity requires clear protocols for observation impact assessment, common tools for routine production of metrics that quantify observation impacts and objective recommendation methodologies.
To reach this goal, OS-Eval TT:
- Build the capacity of assimilating new-type of observation and improve the capacity of extracting observational information
- Evaluate and help optimize existing ocean observing systems and contribute to the optimal design of future observing systems, in order to enhance their impacts on ocean (and coupled) predictions (both coastal/shelf and large scale)
- Develop best practices to evaluate or design ocean observing systems using ocean (and coupled) prediction systems
- Helps the transfer from research to operation of new methodologies,
- Provides support to operational agencies to adopt new methods for OS-Eval and better use of observation
- Enhance the communication between OceanPredict and observational communities and international organization linked to ocean observing network management.
- Provides consistent and scientifically justified requirements and feedbacks to agencies in charge of GOOS and ROOS.
A short term objective is to publish regularly Ocean Observation Impact Annual Report (OIAR) to provide clear information on use of observation data and their impacts in ocean (and coupled) predictions.
In practice, different approaches are developed for observing system evaluation and design. Here are the most commonly used to assess the role of specific observations in constraining the ocean circulation at regional or global scale:
- Observing System Experiments (OSE) which involves the denial of a sub-set of observations, and the evaluation of the degradation in quality of the resulting analyses and forecasts. The degradation quantifies the impact of the with-held observations.
- Observing System Simulation Experiment (OSSE) approach allows evaluating the impact on ocean analysis and forecast of future observing system by simulating observations from a known numerical ocean state.
- Specific diagnostics such as analysis self-sensitivities, forecast sensitivities, singular vector analysis, DFS, …
OS-Eval TT meets every 2 years, preferably jointly with other TT or relevant research communities to enhance the collaboration on cross cutting subjects. Regular TT web meetings are also organized.
Membership include core members, who are regular participants in OS-Eval TT activities and associate members, who are involved in OS-Eval TT activities in an advisory capacity.
List of core members:
|4||Jim||Cummings||IMSG at NCEP/EMC||USA||Member|
|9||Patrick||Heimbach||University of Texas at Austin||USA||Member|
|12||Villy||Kourafalou||University of Miami / RSMAS||USA||Member (co-chair of the COSS-TT)|
|20||Gregory||Smith||Environment Canada||Canada||Member (co-chair of the IV-TT)|
List of associated members:
|1||Bayler, Eric||NOAA||OPAS champion|
|2||DiGiacomo, Paul||NOAA||OPAS chair|
|3||Donlon, Craig||ESA||OPAS champion|
Status: Feb 2021
Meetings and workshops
A major part of the TT’s work is to bring the science community together in TT meetings and workshops. The OS-Eval TT holds larger workshops and meetings every 2-3 years often involving other OceanPredict Task Teams and external research groups who can contribute to the meeting objectives. Since early last year (due to Covid) the TT has introduced regular virtual meeting every 6 weeks.
→ 4th OS-Eval TT meeting, Tsukuba Center for Institutes, Tsukuba, Japan (near Tokyo), Nov 2021
From May 2020 onwards the OS-Eval TT* started holding regular online Task Team meetings to discuss TT activities and future plans.
OS-Eval TT online meeting attendees – 28 July 2020
Link to Observation data table
|Date||Description||Participants||Presentations||Meeting summary & recordings|
|April, 28 2021||9th online meeting of the OS-Eval TT||TBC||Agenda||TBC|
|March, 16, 2021||8th online meeting of the OS-Eval TT||Santha Akella, Eric Bayler, Paul Biswamoy, Fraser Davidson, Paul DiGiacomo, Craig Donlon, Yosuke Fujii, Audrey-Anne, Gauthier, Stephanie Guinehut, Eric C Hackert, Patrick Heimbach, YoungHo Kim, Villy Kourafalou, Matthieu Le Henaff, Elisabeth Remy, Paturi Shastri, Greg Smith, Clemente Tanajura, David Trossman, Kirsten Wilmer-Becker, Jiping Xie, Tianyu Zhang, Hao Zuo||Agenda
Overview presentation (Yosuke Fujii, Elisabeth Remy)
Regional ocean modeling in support of hurricane forecast improvements (Matthieu Le Henaff, Uni. of Miami-CIMAS, NOAA-AOML)
Benefits of assimilating the SMOS SSS in the Arctic reanalysis (Jiping Xi, NERSC)
On the activities in KIOST’s ocean and climate prediction system development (YoungHo Kim, KIOST)
Bluelink ReANalysis, version 2020 (BRAN2020): impact of Near-Real-Time data (Peter Oke, CSIRO)
Plans for Arctic OSE and SWOT OSSE in Canadian Analysis Systems (Greg Smith, ECCC)
|To access the meeting recording please visit the OS-Eval TT restricted area|
|Jan, 8, 2021||7th online meeting of the OS-Eval TT||see minutes||Agenda
1. Introduction of the current status on the contributions of OP communities to the UN Decade, including the OP proposal for the UN Decade programme
2. Discussion on the proposal of the OS-Eval projectOverview presentation OS-Eval TT UN Decade proposal (Yosuke Fujii)
|Dec, 3, 2020||6th online meeting of the OS-Eval TT|
|Oct,13, 2020||5th online meeting of the OS-Eval TT||See minutes||Agenda
Recent OS-Eval Activity and future plans in JMA/MRI (Norihisa Usui, Yosuke Fujii)
|Sep, 8, 2020||4th online meeting of the OS-Eval TT||John Siddorn, YougHo Kim, Clemente Tanajura, Patrick Heimbach, Shuhei Masuda, Eric Hackert, Peter Oke, Yosuke Fujii, Elisabeth Remy and Kirsten Wilmer-becker||TT-agenda and overview talk (Yosuke Fujii, Elisabeth Remy)
GOOS & OceanPredict (John Siddorn, Met Office)
Link to Observation Data Table
|July, 28, 2020||3rd online meeting of the OS-Eval-TT||Yosuke Fujii, Elisabeth Remy, Clemente Tanajura, Peter Oke, YoungHo Kim, Robert King, Paul Biswamoy, Shuhei Masuda, Eric Bayler, Arun Kumar and Fraser Davidson||TT introduction presentation (Yosuke, Fujii, Elisabeth Remy)||Summary pdf|
|June 22, 2020||2nd online meeting of the OSEval-TT||Yosuke Fujii, Elisabeth Remy, Peter Oke, Clemente Tanajura, Shastri Paturi, YoungHo Kim, Eric Hackert, Rob King, Shuhei Masuda, Arun Kumar, Hao Zou||Overview presentation/intro (Yosuke Fujii)
Argo-to-OcePred (Peter Oke)
|– Introduction to the Argo programme, invitation for joint Argo/OP meetings (next possibility March 2021, Moncao) (presentation from Peter Oke)
– consideration of working with Argo; about use of deep and BGC Argo data, advocay for use, demonatraiton of impact; help with trajectories under-ice; use of data at park depth; withholding experiments of old floats, array design (double density regions (questions in Peter’s talk)
– overview of OP science team meeting, strategy and implementation; questions bout what OSE/OSSEs are in progress in coastal areas, other
|May 11, 2020||First online meeting of the OSEval-TT members to discuss||Yosuke Fujii, Elisabeth Remy, Clemente Tanajura, Greg Smith, Hao Zou, Eric Hackert, Fraser Davidson, Jim Cummings, Shastri Paturi, Kirsten Wilmer-Becker||TT meeting intro (Yosuke Fujii)
Jason-3 EoL OP recommendations (Elisabeth Remy)
|– Agreement to run regular OSEval-TT meetings (frequency tbc)
– Publish table that shows use of observation data in DA systems
– For next meetings: provide talks and other documents ahead of the meeting
* Please note that the task team has updated is notation to OS-Eval TT to better reflect the Evaluation aspect of the TT remit.
The OS-Eval TT will be contributing to the UN Decade of Ocean Science, through collaboration with OceanPredict and CoastPredict for which the TT submitted and decade project.
- Report from the Joint DA-TT / OS-Eval TT meeting, CMRE, La Spezia, Italy, Oct 2017
- Report from the 2nd OS-Eval TT workshop, Toulouse, France, December 2014
- Report from the 1st OS-Eval TT meeting, Santa Cruz, USA, June 2011
- Report from the 2nd GODAE Observing System Evaluation Workshop, June 2009, Toulouse, France
- Report from the 1st OOPC-GODAE Meeting on OSSEs/OSEs, November 2007, Paris, France
The task team quad-chart provides a quick glance at the TT’s activities, achievements and future plans.
Observing System Evaluation Task Team (OSEval-TT) – October 2019
|Short description and objectives of the activities started or planned for this year:
||Accomplishments of the TT this year:
1. Coordination of the TT
2. Co-chairs attended GOV-ST VIII and discuss on collaboration with other TTs
3. Generate a community white paper of OceanObs19’ on ocean observing system evaluation. Support other white papers associated with the TT activities.
4. Contribute to the session on assimilation of salinity data in the Ocean Salinity Science Conference
5. Co-chairs attend relevant meetings (25th Altimetry, Argo TPOS2020, WCRP, etc.) and appeal the TT activities
1. OceanObs’19 community white paper on OSEval
2. Observing Impact Statement (OIS) for ESA-SMOS published
3. Multi system Obs. System Evaluation at international level:
• ESA SMOS studies(CLS, Met Office, Mercator)
• H2020 AtlantOS project (numerous European partners)
• SWOT impact studies ongoing (US, Canada, France)
• TPOS2020 project ongoing (JMA, US Mercator)
|Future plans to continue/ improve current activities:
|Main future plans:
1. Continuing the Collaboration with TPOS2020.
2. Plan to collaborate with international Argo ST for Argo network evaluation.
3. Update the TT’s homepage and improve the visibility of OSEval activities within the OP centers, and enhance communication among members
4. Regular publication of the Ocean Observation Impact Annual Report (instead of OIS).
5. Plan to organize a TT Meeting (jointed to CP-TT) in Nov. 10-13th, 2020, in Tsukuba, Japan
• How to strengthen the communication with observational agency
• How to communicate on OSE/OSSE, their use and limitations
• How to evaluate the status of current observing system in real-time (NRT-OSE is expensive and can be implemented in few institute)
• How to Extend the TT activity to international (Currently activity is mostly limited in EU countries)