Cartesian conversions and composites

Compositing

Rack can also create radar image composites, sometimes called mosaics. In weather services, composites are often preferred to single-radar products. On the other hand, a Cartesian product displaying single-radar data is identified with an single-radar "composite" in Rack . This section lists related commands and examples. A quick reference is found in Compositing and Cartesian products. Compositing motion vectors is art of its own, hence explained in a separate section, Composites of motion fields .

A radar image composite over the Baltic area.

Compositing methods

Compositing can be apply several compositing methods (or rules):

  • AVERAGE - average value
  • MAXIMUM - maximum value
  • MAXW - maximum-quality (using a quality field provided with the data)
  • WAVG,p,r,offset - weighted average with data and quality enhancement coefficients p and r ; also offset is needed for ensuring positive data values. See also Handling special codes .

The commands related to compositing and Cartesian products start with 'c'. In computation Rack supports two schemes:

  • Direct scheme: data from all the radars are added to a composite sequentially, applying a single command line
  • Tiled (distributed) scheme: data from each radar are pre-processed and stored in parallel, asynchronously; the tiles are later composited by a separate, fast process

In both direct and tiled schemes, data of each radar can undergo anomaly detection and removal. In addition, a polar product (like pseudo CAPPI or echo height) can be computed.

Projections

Projection definition command –cProj uses Proj.4 syntax, supporting also EPSG codes given like '+init=epsg:4326' . When writing GeoTIFF files, using EPSG codes is recommended because not all Proj.4 project definition strings are supported by the GeoTIFF library.

4326
+init=epsg:4326
# Note: only partially conformant with WKT2:2019
+proj=longlat +datum=WGS84 +no_defs +type=crs
# Note: only partially conformant with WKT2:2019
2393
+init=epsg:2393
+proj=tmerc +lat_0=0 +lon_0=27 +k=1 +x_0=3500000 +y_0=0 +ellps=intl +towgs84=-96.062,-82.428,-121.753,4.801,0.345,-1.376,1.496 +units=m +no_defs +type=crs
3067
+init=epsg:3067
+proj=utm +zone=35 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs
5125
+init=epsg:5125
+proj=tmerc +lat_0=58 +lon_0=25.5 +k=1 +x_0=100000 +y_0=1000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs
3844
+init=epsg:3844
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=sterea +lat_0=46 +lon_0=25 +k=0.99975 +x_0=500000 +y_0=500000 +ellps=krass +units=m +no_defs +type=crs
3035
+init=epsg:3035
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +units=m +no_defs +type=crs
3995
+init=epsg:3995
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=stere +lat_0=90 +lat_ts=71 +lon_0=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +type=crs

Handling special codes

Radar data complying the ODIM standard applies special marker codes undetect and nodata . These codes have to be handled carefully in product generation and compositing. In Rack , pixel values with nodata are simply skipped; they are not accumulated in the composite array, for example.

Handling of undetect can be tuned by the user with –quantityConf and –undetectWeight . The physical value or "virtual zero" replacing undetect codes is set using –quantityConf ,
for example, –quantityConf DBZH,zero=-32.0 (corresponding the limit of signal detection sensitivity). See also Default encoding of quantities. The corresponding relative weight is set with –undetectWeight <value> . The value 0.0...1.0 describes the significance of the value relative to a "true" detected measurement value. The value is zero by default, which in compositing means that the location is marked as processed (a counter is incremented), but the value will not be accumulated.

Compositing schemes

Direct compositing

Creating composites in a direct mode involves the following steps:

  1. Initialising a composite to apply projection, geographical scope, size, and compositing method
  2. For each radar, reading and processing volume data and projecting them on the array
  3. Extracting the final product from the array

The process is illustrated in the graph below.

dot_inline_dotgraph_1.png

A 800 x 1000 maximum composite over Finland in a lat-lon projection is obtained with

rack --odim 2.2 --cMethod MAXIMUM \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 800,1000 --cBBox 17,57.7,33,69.5 --cInit \
\
--script ' -Q 'DBZH' --cAdd' \
data/pvol_fianj.h5 \
data/pvol_fiika.h5 \
data/pvol_fikor.h5 \
data/pvol_fikuo.h5 \
data/pvol_filuo.h5 \
data/pvol_fiuta.h5 \
data/pvol_fivan.h5 \
data/pvol_fivim.h5 \
--cExtract dwsc \
-o composite-fi.h5

A similar composite of CAPPI in 1000 m altitude, with quality-weighted compositing method is obtained with

rack --odim 2.2 --cMethod WAVG,2,2,-32 \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 --cInit \
\
--script ' --pCappi 1000 --cAdd' \
data/pvol_fianj.h5 \
data/pvol_fiika.h5 \
data/pvol_fikor.h5 \
data/pvol_fikuo.h5 \
data/pvol_filuo.h5 \
data/pvol_fiuta.h5 \
data/pvol_fivan.h5 \
data/pvol_fivim.h5 \
--cExtract dwsc \
-o composite-fi-cappi.h5

Rack installation package contains a utility script make-composite.sh which compiles a command line string - like the one above - and executes it. For example:

PRODUCT=pCappi,1000 METHOD=WAVG,2,2,-32 SIZE=500,750 BBOX=17,57.7,33,69.5 ./make-composite.sh data/pvol_fi*.h5

The extraction command –cExtract extracts one or more data layers out of the compositing (accumulation array):

  • DATA - the actual composited data, like reflectivity (DBZH )
  • WEIGHT - quality (confidence) of the data value
  • COUNT - number of radars contributed
  • DEVIATION - difference or discrepancy of data (if more than one radar contributed)

The desired combination is given as a column (: ) separated list of above keywords. Note that for now, also the older syntax is supported: a string composed of single letters ( d ), weight, (w ), deviation (std.dev., s ) and count of radars (c ).

The new, second parameter is optional cropping: the desired bounding box <LLx>:<LLy>:<URx>:<URy> or DATA (automatic bounding box) or OVERLAP (overlapping area).

Layers extracted from the accumulation array: the composite, quality field, deviation and count of radars.

Rack makes composites of any selected quantity or polar product. Examples are shown in the image below.

Composites of CAPPI, max echo, echo top, rhoHV, and HydroClass.

Hint. Command lines may get lengthy in compositing. You can shorten file arguments by using general options –inputPrefix and –outputPrefix .

Tiled compositing

In a Tiled (distributed) mode, the composite is initialized as in direct mode but the tile generated is with –cCreateTile and then saved with –outputFile command. In the accumulation stage, the composite is loaded and then copied to the internal compositing array with –cAdd. The stages are illustrated in the diagrams below.

dot_inline_dotgraph_2.png

For example, creation of a tile:

rack --odim 2.2 --cMethod WAVG,2,2,-32 \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 --cInit \
data-20140525/201405251200_radar.polar.fivim.h5 --pCappi 1000 --cCreateTile -o tiles/201405251200_radar.polar.fivim-tile.h5

Then, tiles are collectively added to composite with a single command. Notice that the composite definitions should be exactly the same as in the tile creation:

rack --odim 2.2 --cMethod WAVG,2,2,-32 \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 --cInit \
--script '--cAdd' \
tiles/201405251200_radar.polar.fianj-tile.h5 \
tiles/201405251200_radar.polar.fiika-tile.h5 \
tiles/201405251200_radar.polar.fikor-tile.h5 \
tiles/201405251200_radar.polar.fikuo-tile.h5 \
tiles/201405251200_radar.polar.filuo-tile.h5 \
tiles/201405251200_radar.polar.fiuta-tile.h5 \
tiles/201405251200_radar.polar.fivan-tile.h5 \
tiles/201405251200_radar.polar.fivim-tile.h5 \
--cExtract dwsc \
-o composite-fi-cappi.h5

(The above commands have been generated using make-composite.sh with SCHEME=TILE and SCHEME=TILED .)

Tiles containing several radars

By default, each tile contains data from a single radar. Alternatively, for a large composite, one can construct a tile containing data from several radars by calling –cBBoxTile command immediately after –cBBox and –cSize . The latter commands are equal to those of the large composite. A multi-radar tile is otherwise generated like any composite, that is, –cAdd and –cExtract commands are needed (unlike in constructing a single-radar tile).

For example, consider receiving a precomputed the OPERA composite, to which the Russian and Belarussian radars are added. First, the latter data is composited to a tile cropped with –cBBoxTile under OPERA geometry:

rack --odim 2.2 --cMethod MAXIMUM \
--cProj '+proj=laea +lat_0=55.0 +lon_0=10.0 +x_0=1950000.0 +y_0=2300000.0 +units=m +ellps=WGS84' \
--cSize 1900,2200 --cBBox -10.4346,31.7462,57.812,67.621 --cBBoxTile 24.4357,51.754,47.9115,65.4706 --cInit \
\
--script ' -Q 'DBZH' --cAdd' \
data-east/201703071200_radar.polar.bymin.h5 \
data-east/201703071200_radar.polar.ruavn.h5 \
data-east/201703071200_radar.polar.ruudb.h5 \
data-east/201703071200_radar.polar.ruudk.h5 \
data-east/201703071200_radar.polar.ruudl.h5 \
data-east/201703071200_radar.polar.ruudp.h5 \
data-east/201703071200_radar.polar.ruudv.h5 \
data-east/201703071200_radar.polar.ruudx.h5 \
data-east/201703071200_radar.polar.ruuwj.h5 \
--cExtract dwsc \
-o tile-east.h5

Finally, the two composites are combined

rack 201703071200_radar.opera.h5 --cAdd tile-east.h5 --cAdd --cExtract dw -o composite-combined.h5
Definition: DataSelector.cpp:44

Notice that the geometry is implicitly determined by the first composite data read (or projected).

Continuous compositing

It is also possible to continuously update a composite. It means that a previous composite is loaded as a "warm start" and –cAddWeighted command. The size and geometrical scope are set to those of the composite file. Fading with a low weight value is motivated by substituting missing radars with older measurements with however lower confidence.

rack composite.h5 --cAddWeighted 0.5 volume.h5 --cAdd --cExtract dwsc -o composite-new.h5
dot_inline_dotgraph_4.png

Utilities

Bounding box of input data

Minimal bounding box of input data, cropped by the bounding box of the composite, is stored in how:dataBBOX variable.

Adding lat-lon grid to Cartesian images

A grid of latitudes and longitudes can be added onto Cartesiand products with –cGrid. The following command creates a grid with 5 degree spacing in latitudes and longitudes, using width of 0.9 pixels and 50% gray intensity:

rack composite.h5 --palette default --cGrid 5,5,0.9,0.5 -o composite-grid.png

The input data – a Cartesian product – can be read as a file as in this example or generated from polar data with –cCreate .