Newsletter 12 March 2020

News item | 12-03-2020 | 12:00

KNMI Data Centre (KDC) and KNMI Data Platform (KDP) transition phase

In our previous newsletter, we announced the transition from the current KNMI Data Centre (KDC) to the new KNMI Data Platform (KDP). FTP and HTTP downloads will be replaced by API downloads. The conversion from KDC to KDP will only affect the way of downloading data, not the data products we offer.

The plan is to switch from KDC to KDP from the end of March 2020. We will use the month April as a transition phase in which both FTP/HTTP and API will be parallel available. This will give you time to convert data scripts, testing them and asking questions you may have.

A brief review of the maintenance of the KNMI Data Centre (KDC) on 2 March 2020

In order to improve our services, we carried out maintenance on Monday 2 March 2020. As a result, the KNMI Data Centre was unavailable for a large part of the day. At the end of the working day, the KDC was back online without any problems.

Questions received in response to the digital transformation

As a result of our reporting on the digital transformation at KNMI we received questions. Below are some of the questions we would like to share with the readers of the newsletter:

Questions Answers As a frequent user of your data, offered via the KNMI Data Centre, we are very interested in the new KNMI Data Platform, which is scheduled to become available at the end of March. We would like to prepare for this. Can you tell me if there is already a description of the KDP API, or even an online test version? We aim to have the API’s and documentation in test versions available before the end of March. In the month of April KDC and KCP will both be available so you can run and test in parallel. Our goal is to offer KDC and KDP in parallel for at least 4 weeks. We currently use OpenDAP to retrieve data. Does this also have consequences for us? Yes, OpenDap is initially no longer available as a service on KDP. Before the API is operational, according to your website there will first be an 'initially more primitive data platform'. How does bringing in the synoptic dataset work there? The data is downloaded via a download API. This API consists of two calls: listfile, which retrieves the list of available files based on dataset name and getfile, which can retrieve a specific file. Is there also an idea to make the model data of Hirlam and Harmonie available on an AWS S3 in the future? The new KDP runs on AWS. We have not yet provided direct access to the S3 buckets, but this is a feature that we will certainly consider. As a result of the newsletter, I wanted to indicate that we remain interested in the weather alarms that we are now bringing in via XML and FTP. The availability of the XML data product 'weather alarm' (via ftpservice.knmi.nl) does not change, the announced migration is completely separate from this. When will the transition happen, and how much warning do you give? The transition phase will start in April. There is a phase in which both data streams run parallel, allowing you to perform test runs. We will inform you via the newsletters and via our website data.knmi.nl. The aim is to offer KDC and KDP in parallel for 4 weeks.

Explanation KNMI Open Data download facility

The KNMI Open Data download facility, KDC/KDP, is used to make Open Data available for reuse. That is: actively making data available as free downloadable files. Our aim is to make KNMI data available and accessible to as many parties as possible. However, the service that comes with this free downloads is restricted in cost point of view. The download facility and its capacity is limited to an effort obligation on availability and timeliness. Service conditions do not apply to Open Data use. KNMI has an alternative: a (payment) option for data downloads with a Service Level Agreement. If you are interested in this facility, please contact dataservices@knmi.nl.

Invitation to participate in the development of 'radar data API’s’

At this moment we have received a number of responses to participate in the development of the 'radar data API’s'. We will start developing these radardata API’s in April. There are different working methods to participate in the development depending on the availability and time of the interested people. For example: participation in a design sprint, API design workshop, testing of API versions, Radar API hackaton. Together with the interested people we will choose a form.

Not registered yet but interested? You can still register by sending an email to dataservices@knmi.nl with Radar API as subject, then we will contact you to discuss this.