Git Product home page Git Product logo

sentinel-download's Introduction

Sentinel-download

(I share tools I develop for my needs. These days, I am downloading data through The French Sentinel mirror site using another tool I developped Peps_download. From my computer, PEPS is much faster than Copernicushub.I have therefore not much time to maintain that one. Volunteers are welcome to take over its maintenance).

Description

Tool to download Sentinel data from ESA (using the wget or aria2 downloaders). This tool can download whole products or only one tile per product, with the -t option (http://olivierhagolle.github.io/Sentinel-download). It should work both on windows and linux.

The ESA Sentinel distribution website allows you to perform catalog searches through http requests. This blog post shows a few examples.

Sentinel-download searches the ESA apihub catalog and downloads the products which fit the criteria defined in the command line. You can select products by :

  • coordinates of a point, or of a rectangle
  • path number (sometimes, a point can be observed from several paths
  • date, of course

This tool was written thanks to ESA well documented scihub API : https://scihub.copernicus.eu/userguide/5APIsAndBatchScripting.

To use this tool, you need a scihub account obtained a long time in advance (see below). If you do not have such an account, or if you want to try a faster access, you might consider downloading the products from the French collaborative ground segment PEPS.

wget /aria2

To use it, you need either to have wget or aria2 installed. I guess it goes with any linux distribution. For windows users, first install aria2 (https://aria2.github.io). If your download stops for a network issue, you can restart S2-download, as wget knows how to resume without havind to download everything again. Wget doesnot download the products already fully downloaded (unlesss you have unzipped them).

wget is the downloader by default, If you want to use aria2, add --downloader aria2 in your command line

Proxy

In case your download must be done through a proxy, you have to setup the https_proxy variable in your.bashrc file or in your .profile.

Proxy with password :

export https_proxy = http://uname:[email protected]:8080

Proxy without password :

export https_proxy = http://proxy.truc.fr:8080

Examples

S2 examples

Here are a few examples for Sentinel-2

  • To download all the products over Toulouse

python %s --lat 43.6 --lon 1.44 -a apihub.txt (scene)"%sys.argv[0]

  • To download all products over Toulouse taken from Path 51

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -s S2

  • To download all L2A products over Toulouse

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -l L2A -s S2

  • To download all products over Toulouse taken from Path 51 with aria2 for windows

python Sentinel_download.py --downloader aria2 --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -s S2

  • To see all products over Toulouse taken from Path 51, but without downloading, thanks to -n option

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -n -s S2

  • To see all products in a rectangle and download only those with a small percentage of clouds :

python Sentinel_download.py --latmin 43 --latmax 46 --lonmin -1 --lonmax 2 -a apihub.txt -o 94 -m 23 -d 2015-12-06 -n -s S2

  • to download all products above Toulouse acquired after 2015-12-26 (December 26th) and before 2016-01-01

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -d 20151226 -f 20160101 -s S2

  • to download all products above Toulouse ingested in November 2016 (note thedate format is different)

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt --id 2016-11-01 --if 2016-12-01 -s S2

  • to download all products above Toulouse acquired after 2015-12-26 (December 26th)

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -d 20151226 -s S2

  • you may also change the output directory with the -w option

python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -d 20151226 -w /mnt/data/Sentinel-2/ -s S2

  • and finally, you may download only one tile from the product (the one which contains Toulouse for instance) python Sentinel_download.py --lat 43.6 --lon 1.44 -a apihub.txt -o 51 -t 31TCJ

S1 examples :

  • To download Sentinel-1 SLC products above a rectangle in France

python Sentinel_download.py --latmin 43 --latmax 46 --lonmin -1 --lonmax 2 -a apihub.txt -s S1A*SLC

  • for other examples see Sentinel-2 examples

authentification

The scihub site accepts the "guest" account and "guest" password, as provided in the apihub.txt file. But only two downloads at the same time are allowed worldwide on the same account. So do not forget to enter your own account and password in the password file provided in the -a option.

However, please also note that ESA has currently limited the acces to users registered some time ago.

In December : The API Hub Access is currently available only for users registered before the 20th of November 12:00 UTC, the user credentials as of the 20th November are valid to access this site.

In January : The API Hub Access is currently available only for users registered on SciHub before the 21st of December 16:46 UTC. The same user credentials are valid to access this site.

If you do not have such an account, you might consider downloading the products from the French collaborative ground segment PEPS.

Issues

Maximum of concurrent flows

2016-01-30 17:09:02 ERREUR 500: Internal Server Error. `

If I use the URL in firefox, I get the following message, which is not true, I have no other download on-going.

<error><code/><message xml:lang="en">An exception occured while creating a stream : Maximum number of 2 concurrent flows achieved by the user "hagolle"</message></error>

I have found a workaround with the --dhus option.

sentinel-download's People

Contributors

olivierhagolle avatar gerts avatar michel-lepage avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.