Import Data - ECMWF Download

<< Click to Display Table of Contents >>

Navigation:  Help Content > Import Data  > Download >

Import Data - ECMWF Download

Previous pageReturn to chapter overviewNext page

WARNING MESSAGE 

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

With the next SARscape sub-release the European Center Medium Weather Forecast (ECMWF) will reach the end-of-life. 

It's functionality will be superseded by the the Era 5 ECMWF Download, mwhich provides the most updated version of the dataset on a hourly basis and finer granularity.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Purpose

This tool allows querying and downloading of (mainly) meteorological data provided by the European Center Medium Weather Forecast (ECMWF). Such information is public available and procedurally accessible through a dedicated Web-API  (http://apps.ecmwf.int/datasets/) that interacts with their databases. At the moment, this tool uses the "ERA5" dataset, which contains global reanalysis data from January 1979 to present computed each hour and with a granularity of about 30km.

 

Although the information is freely accessible, an account must be first created as login credentials are required by this tool to perform queries. If you don't have an account yet, please apply at https://apps.ecmwf.int/registration/ and get one. You then have to do two additional steps before using this tool: 1) obtain your Web-API key and 2) accept the service terms and conditions.

 

To retrieve the Web-API key, which is a long alphanumeric string like "2e5ea248fa90fa53a0d2a11b772cde64", you have to login using your account (on https://apps.ecmwf.int/auth/login/) and get the key from this link: https://api.ecmwf.int/v1/key/. More information on this procedure can be found at https://software.ecmwf.int/wiki/display/WEBAPI/Access+ECMWF+Public+Datasets#AccessECMWFPublicDatasets-key.

Please keep in mind that you have to enter this Web-API key in the password field of the SARscape ECMWF Download panel!

 

Before being able to access the ECMWF datasets, you have to agree on the corresponding terms and conditions that can be found under the "Licence" link at https://software.ecmwf.int/wiki/display/WEBAPI/Available+ECMWF+Public+Datasets. The dataset used by this tool is named interim, so make sure you accept its terms and conditions. Since you're there, we suggest to also accept the terms and conditions of the era5 dataset, as future versions of this tool will be migrated to this dataset.

 

Technical Note

 

This tool performs a search for ECMWF data according to the query built through the input parameters: the tool is responsible for searching, downloading and processing the information for its usage within SARscape.

 

When started from the SARscape graphical user interface (under Import Data->ECMWF Download), the query is executed and, depending on several factors such as its complexity and ECMWF service traffic, can take a significant amount of time to complete.

 

ECMWF policies about data downloading are subject to vary over time and some specific restrictions apply, for example regarding the maximum temporal span or the number of fields that can populate one single query. Please refer to the ECMWF documentation for further details. In case of failed download, users should reduce the temporal span to be downloaded.

 

A query is built through the many input parameters that users can customize to their needs. These input parameters are then automatically translated into the proper ECMWF Web-API series of instructions that leads to the generation and acquisition of the required information. This information is also automatically converted into files that are natively supported by SARscape, including the automatic generation of series to see their evolution through animations. As a rule of thumb, a separate file is created for each observed day/hour.

 

The customization of the many input parameters cover a series of different cases that should simplify the acquisition of ECMWF data related to the user's needs. Users should first identify which kind of meteorological information to query by selecting one of the fields specified under the "Parameter" field. Those parameters cover information such as temperature, cloud cover, soil water, etc. For more detail about the very description of each parameter, please refer to the official documentation provided by ECMWF.

Once defined the kind of meteorological information, users have to specify a temporal interval and a geographical area. The first is done by entering the starting and ending date under the "From" and "To" fields. In addition, users have to specify at which hour(s) of the days within the temporal interval data should be downloaded. The ECMWF interim dataset provides results at 00:00, 06:00, 12:00 and 18:00. One or multiple hours can be selected.

The geographical area is specified through its starting/ending latitude and longitude coordinates, expressed in degrees.

A simple query is then completed by specifying the desired output projection (under the DEM/Cartographic tab) and output file root name (under the Output Files tab).

 

More advanced queries can be performed by letting the tool extracting the temporal interval and/or geographical area directly from input files. Users can specify an input shape file under the Optional Files tab. In that case, the query will automatically use a lat/lon bounding box surrounding the input shape file instead of the explicit coordinates specified under the Parameters tab.  If no image is included as reference, at least one sensing time slot has to be set.

Users can also put one or more files in the file list under the Input Files tab. In that case, the tool will automatically analyze such files and extract the temporal interval from the oldest and newest file, as well as constructing a lat/lon bounding box large enough to encompass the area covered by all the files. When this approach is used to specify the temporal interval and geographical area, an additional parameter named Download strategy (available under the Parameters tab) can be set to download all the data available in the interval (Continuous), only one acquisition per day within the interval and close the input file mean hour (Daily), or only at the days specified by the input files and close to the input mean hour (Nearest one).

 

 

Input Files

 

Input reference image list

Input file(s) name of the image to be used as geographic reference for the search query. These files are mandatory if no shapefile or coordinate box is given as a reference. These files can be geocoded files, slant range or ground range files. However, the data type of the whole list has to be the same.

 

Optional Files

 

Input reference shape

Shape file of a geographic region. All acquisitions inside or intersecting this polygon are returned. Shape file must be in GEO-GLOBAL cartographic system.

 

DEM/Cartographic System

 

Output Projection

The following parameters are compulsory to define the Cartographic System:

 

State

Definition of the country or general projection systems.

 

Hemisphere

Definition of the hemisphere. This is chosen according to the selected State and Projection.

 

Projection

Definition of the projection system of the selected State. In case that a general projection system is selected in State, the Projection is automatically set.

 

Zone

Definition of the zone. This is chosen according to the selected State and Projection.

 

Ellipsoid

Definition of the ellipsoid. This is chosen according to the selected State and Projection.

 

Datum Shift

Definition of the datum shift parameters. These are chosen according to the selected State and Projection.

 

Reference Height

In case that the Digital Elevation Model is not used, a constant ellipsoidal height must be provided. Default Reference Height is 0.

 

 

Parameters - Principal Parameters

 

Parameter

Definition of the product type. The following parameters can be selected:

Volumetric soil water layer 1, Volumetric soil water layer 2, Volumetric soil water layer 3, Volumetric soil water layer 4, Surface pressure, Total column water vapor, Total cloud cover, Soil temperature level 1, Soil temperature level 2, Soil temperature level 3, Soil temperature level 4, 2 meter temperature, 2 meter dew point temperature, Snow density, Snow depth, Skin reservoir content, Skin temperature, Log of surface roughness for heat).

 

From

Sensing period start date (day-MONTH-year), dd-MMM-yyyy, (01-JAN-1970).

 

To

Sensing period stop date (day-MONTH-year), dd-MMM-yyyy, (01-JAN-1970).

 

At: Delivery time of ECMWF dataset results: 00:00, 06:00, 12:00 and 18:00.

 

Geographic Region

Box coordinates. All acquisitions inside or intersecting this box are returned. These fields have to be left empty if a shapefile is used as spatial reference. Co-ordinate decimal values must be entered using the dot (e.g. 29.30) and not the comma (e.g. 29,30) character.

 

Initial longitude

The Westernmost cartographic co-ordinate (in decimal degrees).

 

Final longitude

The Easternmost cartographic co-ordinate (in decimal degrees).

 

Initial latitude

The Northernmost cartographic co-ordinate (in decimal degrees).

 

Final latitude

The Southernmost cartographic co-ordinate (in decimal degrees).

 

X Grid Size

The grid size of the output data  in Easting (X) must be defined only if the Reference file is not used; the default unit of measure is meters.

 

Y Grid Size

The grid size of the output data  in Northing (Y) must be defined only if the Reference file is not used; the default unit of measure is meters.

 

Login - USERNAME

Username of ECMWF Web-API login credentials. This field is mandatory.

 

Login - KEY

The long alphanumeric key (like "2e5ea248fa90fa53a0d2a11b772cde64") get from this link: https://api.ecmwf.int/v1/key/. This field is mandatory.

 

Proxy - URL

If a proxy server is being used for accessing the external network, write here the server address.This field is optional.

 

Proxy - USERNAME

If the proxy server requires a username put it here.This field is optional.

 

Proxy - PASSWORD

If the proxy server requires a password, put it here.This field is optional.

 

First name

Write here your personal given name.You can leave it as ANONYMOUS.

 

Last name

Write here your family name.You can leave it as ANONYMOUS.

 

E-mail

Write here your e-mail address.Not mandatory.

 

Parameters - Global

It brings to the general section of the Preferences specific parameters. Any modified value will be used and stored for further processing sessions.

 

ECMWF ACCOUNT

It brings to the User account section of the Preferences common parameters.  Any modified value will be used and stored for further processing sessions.

 

Login - USERNAME

Username of ECMWF Web-API login credentials. This field is mandatory.

 

Login - PASSWORD

The long alphanumeric key (like "2e5ea248fa90fa53a0d2a11b772cde64") get from this link: https://api.ecmwf.int/v1/key/. This field is mandatory.

 

Proxy - URL

If a proxy server is being used for accessing the external network, write here the server address.This field is optional.

 

Proxy - USERNAME

If the proxy server requires a username put it here.This field is optional.

 

Proxy - PASSWORD

If the proxy server requires a password, put it here.This field is optional.

 

First name

Write here your personal given name.You can leave it as ANONYMOUS.

 

Last name

Write here your family name.You can leave it as ANONYMOUS.

 

E-mail

Write here your e-mail address.Not mandatory.

 

ECMWF SERVICE

 

Base URL

It allows setting the ECMWF Web-API entry point (https://api.ecmwf.int/v1/).

 

Service URL

String to be appended to base URL for performing search queries (datasets/interim/requests).

 

Connection timeout

Connections have to success before this time out, in seconds, is triggered.

 

Download timeout

A threshold value, in seconds,  after which "Download Speed" bytes must be downloaded, otherwise the connection is considered lost and an error is triggered.

 

Download Speed

A threshold value, in byte, of data that must be downloaded within the specified download time out, otherwise the connection is considered lost and an error is triggered.

 

Download Directory

A specific folder is set, where the required ECMWF files are stored.

 

Parameters - Other Parameters

 

Parameter

Definition of the product type. The following parameters can be selected:

Volumetric soil water layer 1, Volumetric soil water layer 2, Volumetric soil water layer 3, Volumetric soil water layer 4, Surface pressure, Total column water vapor, Total cloud cover, Soil temperature level 1, Soil temperature level 2, Soil temperature level 3, Soil temperature level 4, 2 meter temperature, 2 meter dew point temperature, Snow density, Snow depth, Skin reservoir content, Skin temperature, Log of surface roughness for heat).

 

From

Sensing period start date (day-MONTH-year), dd-MMM-yyyy, (01-JAN-1970).

 

To

Sensing period stop date (day-MONTH-year), dd-MMM-yyyy, (01-JAN-1970).

 

At: Delivery time of ECMWF dataset results: 00:00, 06:00, 12:00 and 18:00.

 

Geographic Region

Box coordinates. All acquisitions inside or intersecting this box are returned. These fields have to be left empty if a shapefile is used as spatial reference. Co-ordinate decimal values must be entered using the dot (e.g. 29.30) and not the comma (e.g. 29,30) character.

 

Initial longitude

The Westernmost cartographic co-ordinate (in decimal degrees).

 

Final longitude

The Easternmost cartographic co-ordinate (in decimal degrees).

 

Initial latitude

The Northernmost cartographic co-ordinate (in decimal degrees).

 

Final latitude

The Southernmost cartographic co-ordinate (in decimal degrees).

 

X Grid Size

The grid size of the output data  in Easting (X) must be defined only if the Reference file is not used; the default unit of measure is meters.

 

Y Grid Size

The grid size of the output data  in Northing (Y) must be defined only if the Reference file is not used; the default unit of measure is meters.

 

 

Output Files

 

Output Root Name

Name of the output root. This file is mandatory.

 

 

General Functions

 

Exec

The processing step is executed.

 

Store Batch        

The processing step is stored in the batch list. The Batch Browser button allows to load the batch processing list.

 

Close        

The window will be closed.

 

Help

Specific help document section.

 

 

Specific Function(s)

 

None.

 

See Also

 

Task, SARscapeBatch object, SARscapeBatch script example

 

References

 

None.

 

 

 

 

© sarmap SA 2022, SARscape 5.6.2.1