Git Product home page Git Product logo

dgmr's Introduction

Rainfall Nowcasting in France using the DGMR Model

This repository applies the pre-trained Deep Generative Model of Rainfall (DGMR) developed by Google DeepMind to Meteo-France rainfall data. The goal of this project is to provide real-time, high-quality short-term rainfall forecasts (nowcasting) for France.

Example forecast

Table of Contents

Features

  • Download radar data through the Meteo-France public data API
  • Short-term rainfall forecasting: max lead time 2h, timestep 5min, resolution 1km
  • Utilizes the DGMR pre-trained (UK data) model for realistic forecasts
  • Visualizations of rainfall predictions with GIFs

Installation

Clone the Repository

git clone https://github.com/meteofrance/dgmr
cd dgmr

Install Environment

With micromamba or conda:

micromamba create -f environment.yml
micromamba activate dgmr

Create the API Key on Météo-France's Portal

If you don't already have an account on the Météo-France Public Data portal, please create one.

  1. Once connected, subscribe to the Radar data API (not the "Radar Package" one).

  2. Then click on Configure the API.

  3. Select API Key as the token type, enter the duration (in seconds), and Generate Token.

  4. Copy the token; you will need it in the next step.

Download the Pretrained Model from DeepMind

Go to this Google Cloud Storage link and download the whole folder dm-nowcasting-example-data/tfhub_snapshots/1536x1280 (this folder contains the pre-trained model with a size of 1536x1280).

Note that the AI model is licensed under Creative Commons BY 4.0.

Set Up Environment Variables

Create a .env file in the root directory and add your configuration settings:

METEO_FRANCE_API_KEY="<your_meteo_france_api_key>"
METEO_FRANCE_DATA_PATH="<path_to_save_the_downloaded_data>"  # If empty, downloaded data will be saved in ./data
DGMR_MODEL_PATH="<path_to_the_pretrained_dgmr_folder>/1536x1280/"
DGMR_PLOT_PATH="<path_to_save_the_plots>"  # If empty, forecast GIFs will be saved in ./plot

Usage

Downloading Data with Meteo-France's API

Simply run the script ./dgmr/download_data.sh to download the latest radar images.

Setting up a Cron Job for Automated Data Download

For real-time forecasting, we need to download the radar data every 5 minutes.

To automate the data downloading process using a cron job, follow these steps:

  1. Open your terminal and type crontab -e to edit the cron table.

  2. Add a new line at the end of the file to define the cron job. The general syntax for a cron task is:

    */5 * * * * <ABSOLUTE_PATH_TO_REPO>/dgmr/download_data.sh

Making Forecasts

To make a real-time rainfall nowcast, use:

python main.py

This script expects that you have already downloaded the 4 latest radar images.

It saves the forecast as a GIF in the plot folder defined by the DGMR_PLOT_PATH env variable.

Data Sources

This project uses rainfall data from Meteo-France.

The H5 files contain 5-minute cumulated rainfall data over France, with a 500m resolution.

To match the pre-trained model inputs, we convert the data to instantaneous rainfall (mm/h) and downgrade the resolution to 1km.

Model Architecture

The DGMR model is a state-of-the-art generative model designed for weather nowcasting. It leverages a combination of convolutional neural networks (CNNs) and generative adversarial networks (GANs) to produce high-resolution, realistic rainfall forecasts.

For more details, please see Ravuri, S., Lenc, K., Willson, M. et al. Skilful precipitation nowcasting using deep generative models of radar. Nature 597, 672–677 (2021).

A notebook is also available as a supplement to the article: notebook.

The DGMR model is under the terms of the Creative Commons Attribution 4.0 International License.

Licenses

Contributors

  • From Météo-France's AI Lab: Théo Tournier, Léa Berthomier, Frank Guibert, Bruno Pradel
  • From the French National School of Meteorology: Flora Chapelle, Hugo Germain, Paul Lacotte & Ariadna Perello

dgmr's People

Contributors

lberth avatar tourniert avatar

Stargazers

 avatar  avatar  avatar

Watchers

Fabien MARTY avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.