Outline
Weather observation is an important part of monitoring the state of our planet. Beyond the immediate values of weather forecasts and weather warnings, it informs about continuing changes of our environment. Weather observation systems rely on meteorological satellite images. Geostationary satellites provide complete spatial coverage of a specific region and a frequent update cycle. From raw satellite radiances, many atmospheric parameters can be derived, known as weather products.
In recent years, rapid advances in communication and sensor and technologies and data storage allowed collecting unprecedented amount of data about the Earth. Analysis and interpretation of these high-dimensional heterogeneous data remains a major challenge. Application of the state-of-the art machine learning methods can help identify the patterns and underlying processes, which is critical for better understanding of our environment and mitigating climate change.
Image source: eumetsat.int
Data example: cloud mask.
Data example: convective rainfall rate.
The Data
The competition data are obtained from Meteosat geostationary meteorological satellites operated by EUMETSAT. A range of weather products are derived from the satellite data by EUMETSAT Satellite Application Facilities (SAF) units dedicated to Nowcasting (NWC SAF). The second generation Meteosat images are processed by NWC SAF software developed by a consortium of National Meteorological Services, including AEMET.
The following weather products are selected as target variables for the competition: temperature (on accessible surface: top cloud or earth), convective rainfall rate, probability of occurrence of tropopause folding, and cloud mask. We also provide additional weather products, such as cloud type. They don’t need to be predicted, but can be incorporated to improve the models. The weather products are encoded as separate channels in the weather images. Each weather image contain 256 x 256 pixels, each pixel corresponds to the area of about 4 km x 4 km. The images are recorded at 15 minute intervals throughout a year. The competition data are split into training, validation, and test sets.
The core challenge data contain the training, validation, and test sets for five regions:
R1 – Nile region (covering Cairo),
R2 – Eastern Europe (covering Moscow),
R3 – South West Europe (covering Madrid and Barcelona),
R7 – Bosphorus (covering Istanbul),
R8 – East Maghreb (covering Marrakech).
The resulting training set contains about 25 billion data points.
The transfer learning challenge data contain only the test set for six additional regions:
R4 – Central Maghreb (Timimoun),
R5 – South Mediterranean (covering Tripoli and Tunis),
R6 – Central Europe (covering Berlin),
R9 – Canarian Islands,
R10 – Azores Islands,
R11 – North West Europe (London, Paris, Brussels, Amsterdam).
For each day, the images are grouped into a separate folder, within which there are several folders containing the weather products. The training and validation sets contain up to 96 files (24 hours x 4 images/hour) in each folder.
Acknowledgement of the data source: the competition data contains modified AEMET/NWC SAF products from February 2019 to February 2021.
Weather4cast 2021 Challenge
We are hosting 2 incremental competitions:
- Stage 1 (April 1 – June 30, finished): Selected papers will be invited to present their work in the 1st Workshop on Complex Data Challenges in Earth Observation @ CIKM.
- IEEE BigData Cup (July 1 – October 27): Selected papers will be invited to present their work in a special session devoted to the challenge @ IEEE BigData. While the data for the IEEE BigData Cup is not available yet, we recommend participants getting familiar with the data by downloading the subset of regions provided for the Stage 1 competition and submitting predictions to its leaderboard.
The competitions comprise two challenges:
1) Core challenge: predict the target weather products in the training regions:
- R1, R2, R3 – Stage 1 competition (finished).
- R1, R2, R3, R7, R8 – IEEE BigData Cup. Get the data.
2) Transfer learning: predict the target weather products in the additional regions where we do not provide any data for training nor evaluation:
- R4, R5, R6 – Stage 1 competition (finished).
- R4, R5, R6, R9, R10, R11 – IEEE BigData Cup. Get the data.
The goal is to predict the next 32 weather images (8 hours in 15 minute intervals) given 4 images (1 hour) . To solve each challenge, algorithms must predict the target weather products (temperature, convective rainfall rate, probability of occurrence of tropopause folding, and cloud mask) in the respective regions for the test dataset. The submission format is an array of size (32, 4, 256, 256) per each day of the 36 test days provided.
Download also the Static Channels such as altitude, longitude and elevation (common files for both challenges). Get the data.
Participants can upload their predictions on the test dataset to the leaderboard of each competition until the submission deadline. Within one week after the deadline, participants are requested to provide additional predictions on a new dataset (called held-out dataset), together with an abstract explaining their methodology.
Clone our github repository to download the starting kit and start playing with the baseline models effortlessly! Write in the forums if you have any question or suggestion!
Regions in blue: core challenge; regions in orange: transfer learning.
2021 IEEE BigData Cup
Prizes
The winners of the core competition and the transfer learning competition will be awarded the following prizes:
1st place – a voucher or cash prize worth 5,000€ to the participant/team + one free IEEE BigData 2021 conference registration;
2nd place – a voucher or cash prize worth 3,000€ to the participant/team + one free IEEE BigData 2021 conference registration;
3rd place – a voucher or cash prize worth 2,000€ to the participant/team + one free IEEE BigData 2021 conference registration.
To receive the prizes, participants must submit the working code, the learned parameters, and a short scientific paper describing their approach.
Timeline
All times and dates are Anywhere on Earth (UTC -12), unless specified otherwise.
Competition starts: July 1, 2021, 00:00 CET.
Data Release: July 1, 2021, 00:00 CET.
Leaderboard opens: July 9, 2021.
Competition ends (submission deadline): October 27, 2021, 23:59.
- held-out dataset available: October 28, 2021, 00:00 CET.
- held-out dataset prediction deadline: November 3, 2021 23:59
Abstract submission deadline: November 10, 2021, 23:59.
Camera-ready paper submission deadline: November 23, 2021, 23:59 EST
Announcement of the winners: November 25, 2021.
IEEE BigData Special Weather4cast Session: December 15, 2021
Stage 1 (finished)
Prizes
The winners of the core competition and the transfer learning competition will be awarded the following prizes:
1st place – a voucher or cash prize worth 5,000€ to the participant/team;
2nd place – a voucher or cash prize worth 3,000€ to the participant/team;
3rd place – a voucher or cash prize worth 2,000€ to the participant/team.
To receive the prizes, participants must submit the working code, the learned parameters, and a short scientific paper describing their approach.
Timeline
All times and dates are Anywhere on Earth (UTC -12), unless specified otherwise.
Competition starts: April 1, 2021, 00:00 CET.
Leaderboard opens: April 1, 2021.
Competition ends (submission deadline): June 30, 2021, 23:59.
- held-out dataset available: July 1, 2021, 00:00
- held-out dataset prediction deadline: July 8, 2021 23:59
Abstract submission deadline: July 29, 2021 (Anywhere on Earth) (extended).
Announcement of the winners: August 20, 2021 (extended).
CDCEO 2021 Special Weather4cast Competition Session: November 1, 2021