Sea Ice Drift Forecast Experiment (SIDFEx)
The Sea Ice Drift Forecast Experiment (SIDFEx) is a community effort to collect and analyse Arctic sea ice drift forecasts at lead times from days to a year, based on arbitrary methods, for a number of sea-ice buoys and, ultimately, research icebreaker Polarstern, on a regular basis.
SIDFEx was motivated in part by the need to determine an optimal deployment position of the research icebreaker Polarstern when she started her year-long drift across the Arctic in autumn 2019 (Multidisciplinary drifting Observatory for the Study of Arctic Climate – MOSAiC), and to provide near-real-time forecasts for the ordering of high-resolution satellite images that should contain the MOSAiC camp on a daily basis. These goals have been accomplished successfully. The MOSAiC drift provided a template for assessing the capabilities to forecast sea-ice drift for a range of applications, ranging from logistics support for future field experiments to potential search and rescue operations.
Moreover, the examination of sea-ice drift forecasts provides an integrated assessment of many aspects of the coupled atmosphere-ice-ocean system and motivates in-depth investigations into how key variables are measured, modelled, and forecast. In particular, we expect coordinated drift forecasts to draw attention to the interaction between sea-ice physics and boundary layer physics in both atmosphere and ocean. We expect that a systematic assessment of real drift forecasting capabilities improves our physical understanding of sea ice and enables us to identify and resolve model shortcomings and identify limits of predictability.
SIDFEx is largely the result of discussions held at various meetings, in particular in the context of the Year of Polar Prediction (YOPP), MOSAiC, the Sea Ice Prediction Network (SIPN), the Forum of Arctic Modelling and Observations Systhesis (FAMOS), and the International Arctic Buoy Programme (IABP).
April 2022: SIDFEx reached another milestone - we now have more than 200k forecasts! The Figure shows a time series of the number of uploaded SIDFEx forecasts (upper panel) and an overview over the targets for which we have been collecting forecasts (lower panel). We would like to take this opportunity to thank all the contributors for the swift and fruitful contribution throughout the last five years and look forward to the next 200k forecasts!
March 2022: Stunning news from the Southern Ocean: Precisely 100 years after Sir Ernest Shackleton’s funeral, the wreck of the Endurance has been found on Saturday, March 5th 2022! Find video footage here: https://www.bbc.com/news/science-environment-60662541 and the official press release: https://endurance22.org/category/press-releases. The SIDFEx team supported the expedition by providing real-time sea-ice drift forecasts based on models by the UK Met Office (UKMO), the Environment and Climate Change Canada (ECCC), the Naval Research Lab (NRL) and the European Centre for Medium-Range Weather Forecasts (ECMWF). Special thanks go out to Ed Blockley (UKMO), Yukie Hata (ECCC), Joe Metzger and Michael Phelps (NRL), Steffen Tietsche (ECMWF), as well as all other SIDFEx contributors whose (in most cases non-global) forecasts did not make it into the Endurance22 product. More detailed information on the SIDFEx contribution will follow.
November 2021: SIDFEx has started preparations to support the search for the wreckage of the Endurance - Shackleton's famous ship that sank in the Weddell Sea in 1915. Our contributing forecast centres provide near-real-time forecasts for five fixed initial positions in and around the search area. These forecasts will be used in February/March 2022 to determine where to install the equipment needed to launch and navigate an automated underwater vehicle (AUV) that will search the ground. The camp, which can be relocated only up to three times, needs to drift over the search area, which will be a big challenge for our drift forecasts.
May 2021: Like in previous years, we invite drift forecasts associated with the 2021 SIPN2 Sea-ice Outlook in June, July, and August 2021. If you want to contribute to SIDFEx and haven't done so before, here is how it works.
On September 20th 2020, research icebreaker Polarstern has left the floe that hosted the second Central Observatory of the MOSAiC Drift. SIDFEx continues to provide near-real-time consensus drift forecasts for the floe to support satellite agencies in ordering high-resolution satellite images for the floe and its surroundings to further document the fate of the second MOSAiC floe. All of these dedicated consensus forecasts, with verifying observations, can be found here (forecasts for the second floe are with "floe2" in the file name, forecasts for the first floe without).
A special North Pole analysis for the MOSAiC Drift based on SIDFEx forecasts, in particular the consensus forecast, has been published on Monday, February 24th, at AWI's Sea Ice Portal here.
The latest status of SIDFEx has been presented at the YOPP Science Workshop, February 2020, in Bremerhaven. The slides are available here.
A section on SIDFEx has been included in the Sea Ice Prediction Network Sea Ice Outlook (SIPN-SIO) 2019 post-season report, available here (Section 7).
SIDFEx now provides a consensus forecast that combines short-term and longer-term forecasts automatically to obtain best-guess forecasts, including uncertainties, in near-real-time. Consensus forecasts for all active targets are available through the SIDFEx webtool. In addition, a graphical product and a product for Polarstern's on-board MapViewer based on the consensus forecast for the MOSAiC Drift is provided with updates every 6 hours. While the MapViewer product is not public, the corresponding graphical product (original and with verifying observations added retrospectively) is publicly available (see links in the box "MOSAiC Drift Forecast").
The SIDFEx webtool, based on the SIDFEx and spheRlab R-packages, has been launched at https://sidfex.polarprediction.net/. A big THANK YOU to Simon Reifenberg who implemented this tool in R-Shiny!
The latest status of SIDFEx has been presented at the 9th IICWG-DA Meeting, June 2019, in Bremen. The slides are available here.
The SIDFEx R-package (available here) has steadily evolved over the past months and now comes with a user guide that facilitates installing and working with the package. We hope that the package and the user guide prove to be helpful and enable others to join into the analysis of SIDFEx data. An "R-Shiny App", a webtool building on the SIDFEx R-package, is scheduled for a first release in July 2019, so keep tuned!
The June call for contributions to the SIPN2 Sea Ice Outlook has been published here. As in the previous two years, the call includes associated submissions to SIDFEx. Corresponding calls for July and August will follow in one and two months.
A new SIDFEx poster and two corresponding leaflets with additional, presented in April 2019 at the EGU General Assembly, can be downloaded here (poster), here (leaflet 1), and here (leaflet 2).
A section on SIDFEx has been included in the Sea Ice Prediction Network Sea Ice Outlook (SIPN-SIO) 2018 post-season report, available here (Section 7).
A news article on ECMWF's contribution to SIDFEx made it into the winter 2018/19 issue of the ECMWF Newsletter, available here.
As of November 2018, a list of all SIDFEx targets, their valid date ranges and latest positions in simple text format is available here. The list is automatically updated hourly and can be used by contributors to automate the target handling.
The Python-based scripts for diagnostic tracking of ice particles have received a minor update on 24 October 2018; see below.
A short report on SIDFEx has been included in the Sea Ice Prediction Network Sea Ice Outlook (SIPN-SIO) 2017 post-season report, available here. The SIDFEx section can be found at the very end of the report.
Background and guidelines for SIDFEx contributions
A document providing details on the design of SIDFEx and how to contribute drift forecasts can be obtained in the guidelines document here.
Contributors who would like to check in advance whether their files meet the formatting conventions (detailed in the guidelines document) may use this R function. How to use it is explained in the file header. The function (named sidfex.checkfileformat()) is also part of the SIDFEx R-package available here.
As detailed in the guidelines document, SIDFEx targets a number of buoys of the International Arctic Buoy Programme (IABP). The selected buoys are listed here, along with a near-real-time map showing their positions and drift history.
For easier automation of target handling for SIDFEx contributions, a list of all SIDFEx targets, their status, valid date ranges, and most recent positions, updated hourly, is available here.
Forecast Results and Analysis Tools
After submission, each forecast is automatically processed and made publicly available in real-time (<1h delay) at the Cloud Service of the German Climate Computing Centre. Individual forecast files, ordered by contributor GroupIDs, can be accessed directly here. In addition, a tar.gz archive of all forecasts and an index of these (in R binary as well as plain text (csv) format) are available here.
A SIDFEx R-package that can be used to download and analyse the SIDFEx data as well as the corresponding IABP observations is available on GitHub here. To get started with the package, please have a look at the SIDFEx package user guide. Building on this, an online tool based on R-Shiny to browse, search, plot, and analyse the results online is under development.
You can find the outcomes of some preliminary analyses of the SIDFEx data in presentations and posters listed above under Recent information.
Below are example scripts that can be taken as a starting point to generate SIDFEx forecasts and to automate the procedure as much as possible, for example such that new targets are automatically detected and forecasts and/or re-forecasts are produced for all targets active at the time.
Target checking and position retrieval
These are example Shell and Python scripts to read the SIDFEx target table to determine which buoys are (and have been) active targets from when to when. Ed Blockley, who kindly provided these example scripts used for the UKMO contributions (thank you!), noted the following: The shell script is the boss. It pulls off the SIDFEx list from AWI and decides which buoys are valid for the given date (default is yesterday - the analysis performed today). It then calls the python script for each buoy find the closest buoy report to the 0Z initial point. I locate the nearest point and complain if it's not within a certain tolerance (1 day used here) so that a buoy can be included even if the 0Z report were missing for some reason.
Tracing and submission:
Our colleagues at NERSC (a big Thank You to Maxime Beauchamp and Laurent Bertino) were so kind to provide their Python-based scripts which they use to conduct the 2D-tracing based on ARC-MFC (TOPAZ) sea-ice drift forecast fields. These scripts are invoked every day by a cron job to generate the near-real-time metno001 contributions. The scripts also include the automated submission using curl . Obviously, the scripts need some adaptions if they are to be used by other groups and to other drift fields, but they might serve as an excellent template.
Note that the scripts received a minor update on 24 October 2018 to allow a temporal tolerance around the time of the last position observation. This helps avoiding gaps with respect to forecast initial times.
Download scripts (version 31 August 2018, uploaded 24 October 2018)
Download older scripts (version 14 May 2018)
You may contact us via email.
Helge F. Goessling (1), Axel Schweiger (4), Valentin Ludwig (1), Laurent Bertino (2), Ed Blockley (3), Frédéric Dupont (6), Wendy Ermold (4), Rüdiger Gerdes (1), Robert Grumbine (5), Yukie Hata (6), Jennifer Hutchings (7), Frank Kauker (1), Thomas Krumpen (1), Jean-François Lemieux (6), François Massonnet (8), E. Joseph Metzger (9), Malte Müller (10), Cyril Palerme (10), Michael W. Phelps (11), Thomas Rackow (1), Till A. S. Rasmussen (12), Simon F. Reifenberg (1,13), Ignatius Rigor (4), Suman Singha (18), Greg Smith (6), Amy Solomon (14,15), Nick Szapiro (16), Steffen Tietsche (17), Jinlun Zhang (4)
1) Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research, Bremerhaven, Germany
2) Nansen Environmental and Remote Sensing Center, Bergen, Norway
3) Met Office Hadley Centre, FitzRoy Road, Exeter, UK
4) University of Washington, Applied Physics Laboratory, Polar Science Center, US
5) National Centers for Environmental Prediction, USA
6) Environment and Climate Change Canada, Canada
7) Oregon State University, College of Earth, Ocean, and Atmospheric Sciences, USA
8) Université catholique de Louvain, Belgium
9) Naval Research Laboratory, Ocean Sciences Division, Stennis Space Center, MS, USA
10) Norwegian Meteorological Institute, Norway
11) Perspecta, Inc., Stennis Space Center, MS, USA
12) Danish Meteorological Institute, Denmark
13) Johannes Gutenberg-Universität Mainz, Germany
14) Cooperative Institute for Research in the Environmental Sciences, University of Colorado, Boulder, CO, USA
15) Earth System Research Laboratory, National Oceanic and Atmospheric Administration, Boulder, CO, USA
16) University of Oklahoma School of Meteorology, USA
17) European Centre for Medium-Range Weather Forecasts, UK
18) German Aerospace Center (DLR), Germany
March 2022: Stunning news from the Southern Ocean: Precisely 100 years after Sir Ernest Shackleton’s funeral, the wreck of the Endurance has been found on Saturday, March 5th 2022! The SIDFEx team supported the expedition by providing real-time sea-ice drift forecasts; see details and links in the corresponding news item under the section "Recent information" on this page.
Quicklink to the SIDFEx Webtool to browse and evaluate all past forecasts and to obtain near-real-time forecasts, including for the MOSAiC Drift: Click HERE.
MOSAiC Drift Forecast
For the MOSAiC Arctic Drift campaign of the research icebreaker Polarstern we were providing a graphical product of the SIDFEx consensus forecast, updated in (near-)real-time every 6 hours:
> All forecasts with verifying observations (second floe with "floe2" in filename, first (main) floe without)
MOSAiC Climate Cube
The coordination and implementation of SIDFEx has been led by the research group Seamless Sea Ice Prediction (SSIP) and within the project SIDFExplore, funded by the German Ministry for Education and Research (BMBF, grants 01LN1701A and 03F0868A).