Different methods of data download

There are different ways to download data from Meteo-Hub portal: 

Web Interface

A registered user can obtain data from one or more datasets of the platform using the Data Extraction feature on Meteo-Hub Web Interface.

How to use Data Extraction Feature

In the first step, the user must select one or more datasets (multiple selection is allowed only with datasets of the same category: observation, forecast, radar). 

In the second step, the user can filter respect the provided parameters that are specific for the dataset category: 

If the user is allowed to use post-processing tools, they can be applied in the third step. The set of post-processing tools are specific for the dataset category: 

In the fourth and last step, the user must enter a name for the current request and submit the request to the system. Before submitting the request, it is also possible to schedule it using the “Schedule” button.  

After submitting a request, the user is led to the “My Requests” page where, once the extraction has been completed, the output data can be downloaded. 

A video tutorial illustrating the four steps of construction and submission of the request is available on MISTRAL project YouTube channel: 

A tutorial (Italian language) with further information on data extraction is available:

How to download packages of open data

For all users, even those not logged in, it is possible to download packages of open data. The packages are produced daily and are available on the Meteo-Hub main page.

Currently this feature is available for the following datasets: 

• COSMO-2I: COSMO at 2.2km – Italy area 
• COSMO-5M: COSMO at 5km – Mediterranean Region 

• Multimodel ensemble forecast by Arpa Piemonte 
• Surface Rainfall Intensity from Radar-DPC 
• Italy Flash Flood 

The open data from the observational datasets can be downloaded from the map of Observations webpage.

A client application that allows users to perform the main operations on the Meteo-Hub platform from the command line is available in this GITLab repository.

The application allows users to download data extracted by an immediate or scheduled request prepared interactively on the Meteo-Hub web interface. It also provides a wait-and-download functionality for users having access to an AMQP data-ready queue. To have further information on how to have access to an AMQP data-ready queue please contact the Mistral technical team at mistral-support@cineca.it 

At the moment, the command-line client does not allow to perform or schedule requests for data extraction that must be prepared interactively through the web interface. 

A tutorial (Italian language) on how to use the Meteo-Hub client is available:

API

Users can directly use Meteo-Hub APIs to create and schedule requests of data extraction and to download the output data. The API are documented using the swagger definition at this URL. 

How to download the open data using the API 

In order to get the list of all the packages of open data for a specific dataset this API has to be used: 

GET https://meteohub.mistralportal.it/api/datasets/{dataset_name}/opendata  

Where the query parameter dataset_name is the name of the dataset of which the packages list is referred. The currently parameters available for this functionality are: 

• COSMO-2I : refers to COSMO at 2.2km – Italy area dataset 
• COSMO-5M: refers to COSMO at 5km – Mediterranean Region dataset 
• multim-forecast: refers to Multimodel ensemble forecast by Arpa Piemonte dataset 
• radar_dpc: refers to Surface Rainfall Intensity from Radar-DPC dataset 
• COSMO-2Ipp_ecPoint: refers to Italy Flash Flood dataset 

To ask for download a specific package of open data this API has to be useed: 

GET https://meteohub.mistralportal.it/api/opendata/{filename} 

Where the query parameter filename is the name of the package to download 

No authentication is required to use these previously described APIs 

How to get an access token 

In order to use the authenticated API the user has to provide an access token. The token is obtained as a response to the login API. 

POST https://meteohub.mistralportal.it/auth/login  

Example of the request body:

{ 
  "username": "user@example.com", 
  "password": "string", 
} 

How to create a request of data extraction using APIs 

In order to create a request of data extraction the following API has to be used:  

POST https://meteohub.mistralportal.it/api/data  

Example of the request body: 

{ 
"request_name": name of the request, 
 "reftime": { 
   "from": "2022-01-10T11:04:54.615Z", 
   "to": "2022-01-10T11:04:54.615Z" 
  }, 
  "dataset_names": [ 
    list of datasets 
   ], 
   "filters": { 
      "area": [{}], 
      "level": [{}], 
      "origin": [ {}], 
      "proddef": [{} ], 
      "product": [{}], 
      "quantity": [{}], 
      "run": [{}], 
      "task": [{}], 
      "timerange": [ {}], 
       "network": [{}] 
     }, 
     "output_format": "json"/null, 
     "only_reliable": true/false, 
     "postprocessors": [ object ] //list of requested postprocessors with their specific configuration 
} 

In the request body the metadata related to the desired data has to be specified. The filters are specific for the dataset category (see table in the previous chapter “Download data from web interface”). Output_format and only_reliable are parameters specific for observations data: the first specifies if the user wants the ouput data in json format instead of the default format (BUFR). The second one specifies if the user wants to download only validated data 

To use the previously described API an authentication is required. The access token get from the login API has to be included as Authorization in the request Header .

How to create a scheduled request of data extraction using APIs 

In order to create a scheduled request the following API has to be used: 

POST https://meteohub.mistralportal.it/api/schedules  

The request body, where all the metadata related to the request are specified, is similar to the one described in the previous chapter “How to create a request of data extraction”. Additional parameters has to be added in order to specify the scheduling interval. The available scheduling methods are: 

Periodical request: data are extracted at specific time intervals 

"period-settings": { 
  "every": number,  // specifies the time interval of the extraction,    
  "period": string  // specifies the time period. Choose one of "hours”,"days",”months”,”years”  
} 

At-time request: data are extracted on specific days at a specific time 

"crontab-settings": { 
  "minute": number //  range between 0-59,     
  "hour": number  //  range between 0-23,     
  "day_of_week": number	//  range between 0-7 where 0 and 7 refers to sunday, 1 to monday, and so on,    
  "day_of_month": number  //  range between 0-31,     
  "month_of_year": number //  range between 1-12 where 1 refers to january and 12 refers to december  
} 

On-data-ready request: data are extracted as soon as they are available on the platform. This option is available only for  COSMO-2I and COSMO-5M datasets. 

"on-data-ready": true, 

To use the previously described API an authentication is required. The access token get from the login API has to be included as Authorization in the request Header.

How to download data using APIs 

In order to get the list of all the submitted requests and their status this API has to be used: 

GET https://meteohub.mistralportal.it/api/requests  

If the data extraction has been completed the fileoutput parameter in the response indicates the name of the output file. 

To download the output data the following API has to be used: 

GET https://meteohub.mistralportal.it/api/data/{filename}  

Where the query parameter filename specifies the file to download. 

To use the previously described API an authentication is required. The access token get from the login API has to be included as Authorization in the request Header.