Extracting metadata
Simple dump of metadata
The basic way to view context in textual format is to write a file with '.txt' extension:
Using --outputFile , the structure of the current data can be written as plain text in a file (*.txt) or to standard input (-):
rack volume.h5 -o volume.txt
Definition DataSelector.cpp:44
File volume.txt will consist of lines of ODIM entries as follows:
dataset1
dataset1/data1
dataset1/data1/data
dataset1/data1/data:image=[500,360]
dataset1/data1/what
...
...
...
how:simulated="False"
how:software="IRIS"
how:sw_version="8.13"
how:system="VAISWRM200"
how:wavelength=5.33
Formatting metadata output using templates
There is often need to produce metadata text for:
- Text files completing data formats not supporting metadata
- Log entries
- HTML files applied in monitoring product generation
- Graphical layouts (eg. SVG)
- Geographic definition files (eg. KML) for adding radar products to GIS applications
One can easily export metadata of current HDF5 structure (latest volume read or product generated). The output is a text file in which variables ${variable} have been expanded to their current values.
The text template can be defined briefly on command line by --format or by reading a file with --formatFile .
For example, from a Cartesian product one may extract the projection and a bounding box with the following command:
rack volume.h5 -c --format
'BBOX=${where:LL_lon},${where:LL_lat},${where:UR_lon},${where:UR_lat}\n' -o formatOut.txt
The result:
BBOX=21.4734,62.4469,32.0369,66.9151
The applicable variables are those the ODIM variables (what:product, where:lat, ...). In addition, what:source fields (NOD, WMO , RAD , PLC , CMT ) are searched for and expanded to convenience variables (${NOD}, etc...). The list of current variables can be examined with --status command.
For example, assume template file radar-conf.tpl with the following contents:
# Metadata for radar '${NOD}'
SOURCE="${what:source}"
NOD="${NOD}"
WMO="${WMO}"
PLC="${PLC}"
LAT="${where:lat}"
LON="${where:lon}"
HEIGHT="${where:height}"
PROJDEF="${where:projdef}"
BBOX="${where:LL_lon},${where:LL_lat},${where:UR_lon},${where:UR_lat}"
Then,
rack volume.h5 -c --formatFile radar-conf.tpl --formatOut out.cnf
produces code which can be used by applications supporting Proj4 projection syntax. The result:
# Metadata for radar 'fiuta'
SOURCE="WMO:02870,RAD:FI47,PLC:Utajärvi,NOD:fiuta"
NOD="fiuta"
WMO="02870"
PLC="Utajärvi"
LAT=""
LON=""
HEIGHT=""
PROJDEF="+proj=aeqd +lon_0=26.3189 +lat_0=64.7749 +ellps=WGS84"
BBOX="21.4734,62.4469,32.0369,66.9151"
Further, using --script one can quickly create similar conf files with a single command:
rack --formatFile radar-conf.tpl --script
'--cReset -c --formatOut site-${NOD}.cnf' <volumes.h5>
New. Latest versions of Rack support variable formatting with following options
bash style substring extraction: bash shell variables)- C
printf() styled value formatting, like %4.2f or %8d (see https://cplusplus.com/reference/cstdio/printf/ )
date style date and time formatting, like Y/m/d or H:M – for variables with names ending with "date" or "time".
This options are illustrated with examples:
#
rack volume.h5 --format
'time: ${what:startdate|%Y-%m-%d} ${what:starttime|%H:%M} elangle: ${where:elangle|%04.1f} \n' --select dataset:/ -o elangles.out
time: 2014-08-27 09:00 elangle: 00.3
time: 2014-08-27 09:00 elangle: 00.7
time: 2014-08-27 09:00 elangle: 01.5
time: 2014-08-27 09:01 elangle: 03.0
time: 2014-08-27 09:01 elangle: 05.0
time: 2014-08-27 09:01 elangle: 09.0
time: 2014-08-27 09:02 elangle: 02.0
time: 2014-08-27 09:02 elangle: 07.0
time: 2014-08-27 09:02 elangle: 11.0
time: 2014-08-27 09:03 elangle: 15.0
time: 2014-08-27 09:03 elangle: 25.0
time: 2014-08-27 09:03 elangle: 45.0
time: 2014-08-27 09:04 elangle: 00.4
Sampling data
The input volume as well as the generated polar or Cartesian products can be sampled, producing a text file. The sampling is performed automatically if output file has the extension '.dat'. For example:
rack volume.h5 -o samples.dat
The result:
## input properties
# bbox=[1507.9672560563260504,3892.9862409188299353,1507.9629643527550797,3892.9862409202742128]
# height=360
# lat=64.77493012323975563
# lon=26.318880077451467514
# width=500
## sampling parameters
# commentChar="#"
# handleVoid="null"
# i=[-1,1]
# iStep=10
# j=[-1,1]
# jStep=0
# skipVoid=false
# format='${i},${j},${LON},${LAT},${X},${Y},${RANGE},${AZM},${HGHT}'
## resulting geometry
# rows=50
# rols=36
# samples=1800
5,5,1508.25,3712.74,239.678,2739.54,2750,5,14.844 #void
25,5,1509.31,3717.86,1111.24,12701.5,12750,5,76.3267
65,5,1511.44,3728.09,2854.35,32625.4,32750,5,234.606
75,5,1511.97,3730.65,3290.13,37606.3,37750,5,281.533
85,5,1512.51,3733.21,3725.91,42587.3,42750,5,331.402
95,5,1513.05,3735.77,4161.69,47568.3,47750,5,384.214
...
...
...
455,355,1482.1,3827.87,-19849.7,226883,227750,355,4244.53
465,355,1481.48,3830.43,-20285.5,231864,232750,355,4406.15
475,355,1480.86,3832.99,-20721.3,236845,237750,355,4570.71
485,355,1480.24,3835.55,-21157.1,241826,242750,355,4738.21
495,355,1479.62,3838.1,-21592.8,246807,247750,355,4908.65
The default sampling step is 10 in both horizontal and vertical direction, and all the quantities are sampled. One may apply --select command to get desired subset of quantities. Further, --format modifies output layout; variables inserted like ${NAME} are converted to actual values in output. (See format in the file header above).
- i and j : image coordinates, from top-left corner (bin and azimuth index in polar sampling);
- j2: inverted vertical image coordinate ie. from bottom-left corner (in Cartesian sampling only)
- LON and LAT : geographical coordinates, in degreees
- X and Y : map coordinates, in metres
- AZM and RANGE: polar coordinates (in polar sampling only)
- the quantities, eg.
DBZH , VRAD , and so on, including quality quantities like QIND. The default format consists of all the applicable coordinates and quantities separated by space. Values can be negated by putting a minus sign inside the curly braces of a format string.
Sampling steps, start, and end can be changed with --sample command. One may also skip lines containing undetect or nodata samples. If a comment char is defined some metadata, including applied format, appears as comments in the start of the output.
--sample <iStep>,<jStep>,<i>,<j>,<commentChar>,<skipVoid>,<handleVoid> (section: general)
Extract samples. See --format.
See also: format
iStep=10 [horz coord step]
jStep=0 [vert coord step]
i=-1:1 [horz index or range]
j=-1:1 [vert index or range]
commentChar=# [comment prefix (char or bytevalue)]
skipVoid=false [skip lines with invalid/missing values]
handleVoid=null [skip or mark invalid values [skip|null|<number>]]
Example of a formatted sampling:
rack volume.h5 --select quantity=DBZH --sample 10,30,i=50:200,j=0:50 --format
'${i},${j} (${LON},${LAT}) ${DBZH}' -o samples-formatted.dat
The result:
## input properties
# bbox=[1507.9672560563260504,3892.9862409188299353,1507.9629643527550797,3892.9862409202742128]
# height=360
# lat=64.77493012323975563
# lon=26.318880077451467514
# width=500
## sampling parameters
# commentChar="#"
# handleVoid="null"
# i=[50,200]
# iStep=10
# j=[0,50]
# jStep=30
# skipVoid=false
# format='${i},${j} (${LON},${LAT}) ${DBZH}'
## resulting geometry
# rows=50
# rols=12
# samples=600
50,0 (1507.96,3724.3) -13.99
90,0 (1507.96,3734.58) -10.79
60,30 (1526.34,3724.77) -11.71
70,30 (1529.41,3726.99) -5.82
80,30 (1532.48,3729.21) -10.65
...
...
...
160,30 (1557.43,3746.9) null #void
170,30 (1560.59,3749.11) null #void
180,30 (1563.76,3751.31) 6.74
190,30 (1566.93,3753.52) 19.96
200,30 (1570.12,3755.72) null #void
Likewise, one can sample Cartesian data. Example:
rack volume.h5 --select quantity=DBZH --cSize 500,500 -c --sample 50,50 -o samples-cart.dat
The result:
## input properties
# bbox=[21.473369970467263812,62.446873711034854182,32.036889576527528334,66.915064520295331363]
# epsg=0
# height=500
# proj="+proj=aeqd +lon_0=26.3189 +lat_0=64.7749 +ellps=WGS84"
# width=500
## sampling parameters
# commentChar="#"
# handleVoid="null"
# i=[-1,1]
# iStep=50
# j=[-1,1]
# jStep=50
# skipVoid=false
# format='${i},${j},${LON},${LAT},${X},${Y},${j2}'
## resulting geometry
# rows=10
# rols=10
# samples=100
25,25,21.2287,66.7069,-224500,224500,474 #void
225,25,25.7622,66.7873,-24500,224500,474
275,25,26.8983,66.7872,25500,224500,474
225,75,25.7721,66.339,-24500,174500,424
175,125,24.6855,65.8829,-74500,124500,374
225,125,25.7816,65.8906,-24500,124500,374
...
...
...
275,475,26.8178,62.7511,25500,-225500,24 #void
325,475,27.7958,62.7441,75500,-225500,24 #void
375,475,28.773,62.7303,125500,-225500,24 #void
425,475,29.7492,62.7096,175500,-225500,24 #void
475,475,30.7237,62.682,225500,-225500,24 #void
Note that --cCreate maps a single quantity to Cartesian coordinates; the quantity can be changed with --select .
Does a radar overlap with a given geographical scope?
1.9.8