GRIB files and extracting data, Linux, Raspberry

Updated 2024-02-08

GRIB is a data format, used for meteorology, to store historical and forecast weather data. 

The use case for me was analyzing forecast data, from GRIB files, so an early warning could be rised before a high water level or coming storm appears. This could then be used at our weather site.

There are several applications, also ones that visualizes the data, but this post will handle how to extract data from files using command line.

Intro

After a lot of Googling and testing I finally implemented the ecCodes package from ECMWF. ECMWF is the European Centre for Medium-Range Weather Forecasts.

Installation

If you accept a little older version of the command line (CLI) toools for GRIB files there are binaries available which can be installed with 

sudo apt install libeccodes-tools

and then check with 

codes_info

If you must have the latest version you have to install manually from source.

CLI examples

Download file

To get some sample GRIB files I used the OpenSkiron site and their forecast files. Just download on of the files clicking on it. Another way, using the "Copenhagen_......" file, check the URL and then use "wget" or "curl" to download it from command line 

wget -O cph.grb2 https://openskiron.org/gribs_icon-d2/Copenhagen_ICON-D2_EWAM_YYYYMMDD-nn.grb2

curl -o cph.grb2 https://openskiron.org/gribs_icon-d2/Copenhagen_ICON-D2_EWAM_YYYYMMDD-nn.grb2

Check ShortNames

To check out the "Parameter ID" "ShortNames" use the "grib_ls" command

grib_ls cph.grb2

another variant to minimize the output is 

grib_ls  -p shortName cph.grb2 | sort | uniq -c | sort -r

where "-p shortName" selects just the "shortname" and you will get a sum of how many times each unique "ShortName" appears in the file.

To check what parameter ID:s are available, in the file, use the parameter database and the "Search:" function.

Extract data

With the above use of "grib_ls" command you found that, for example, the "Parameter ID" => "VMAX_10M", Maximum wind velocity and 2 meter temp, "2t", is available in the file. If you, like me 😎, live at latitude=56.12  longitude=12.60 and are interested to check the forecast, use the command

grib_ls cph.grb2 -l 56.12,12.60,1 -w shortName=VMAX_10M/2t -p shortName,date,validityDate,validityTime

In my example it gave

cph.grb2

shortName     count         date          validityDate  validityTime   value 

2t            442           20220928      20220928      1200          284.783       

2t            443           20220928      20220928      1300          284.81        

2t            444           20220928      20220928      1400          284.881       

2t            445           20220928      20220928      1500          284.972       

...

...

VMAX_10M      683           20220928      20220930      1000          6.0198        

VMAX_10M      684           20220928      20220930      1100          6.04429       

VMAX_10M      685           20220928      20220930      1200          6.1439       

98 of 685 messages in cph.grb2

98 of 685 total messages in 1 files

Input Point: latitude=56.12  longitude=12.60

Grid Point chosen #2 index=8662 latitude=56.12 longitude=12.60 distance=0.00 (Km)

Other grid Points

- 1 - index=8663 latitude=56.12 longitude=12.62 distance=1.24 (Km)

- 2 - index=8662 latitude=56.12 longitude=12.60 distance=0.00 (Km)

- 3 - index=8512 latitude=56.10 longitude=12.62 distance=2.55 (Km)

- 4 - index=8511 latitude=56.10 longitude=12.60 distance=2.22 (Km)

which shows the forecast for wind, m/s, and temperature, Kelvin, some hours ahead.

Wrap up

This was a short intro to the CLI tools but there is a lot more to learn !

Remark

In the above extract example it would have been natural to also extract the wind, but in most grib files the info is not reported as just wind and wind direction. The wind is instead reported as 2 vektors, component "u" and "v", which demands a little mathematics to decode. (Another explaination). 

In the above example file the wind vectors are reported as "10u" "10v" which is 10 meter "u" and "v" component. Here you find a post that handles the math to give the wind speed/direction

A way to avoid dealing with the wind vectors is using a GRIB source that have other parameters for wind. If you live in Europe you could use DMI as a source, the Wave Model (WAM), where they use the parameters "wind" and "dwi"(10 meter wind speed and direction).