Loading...
Searching...
No Matches
Cartesian conversions and composites

Compositing

Rack can also create radar image composites, sometimes called mosaics. In weather services, composites are often preferred to single-radar products. On the other hand, a Cartesian product displaying single-radar data is identified with an single-radar "composite" in Rack . This section lists related commands and examples. A quick reference is found in Compositing and Cartesian products. Compositing motion vectors is art of its own, hence explained in a separate section, Composites of motion fields .

A radar image composite over the Baltic area.

Compositing methods

Compositing can be apply several compositing methods (or rules):

  • AVERAGE - average value
  • MAXIMUM - maximum value
  • MAXW - maximum-quality (using a quality field provided with the data)
  • WAVG,p,r,offset - weighted average with data and quality enhancement coefficients p and r ; also offset is needed for ensuring positive data values. See also Handling special codes .

The commands related to compositing and Cartesian products start with 'c'. In computation Rack supports two schemes:

  • Direct scheme: data from all the radars are added to a composite sequentially, applying a single command line
  • Tiled (distributed) scheme: data from each radar are pre-processed and stored in parallel, asynchronously; the tiles are later composited by a separate, fast process

In both direct and tiled schemes, data of each radar can undergo anomaly detection and removal. In addition, a polar product (like pseudo CAPPI or echo height) can be computed.

Initialisation

Essential properties of a composite are:

  • Size of the array (image)
  • Geographical projection
  • Geographical extent

Size

The size of the composite, in pixels, is set with –cSize.

Projection

Projection definition command –cProj uses Proj.4 syntax and additionally supports direct EPSG codes, --cProj\ 3067 .

If the projection is not given, Rack uses the Azimuthal Equidistant (AEQD) projection centered at radar site of the first input.

When writing GeoTIFF files, using EPSG codes is recommended because not all Proj.4 project definition strings are supported by the GeoTIFF library. The following list produced as a part of release tests.

4326
+init=epsg:4326
# Note: only partially conformant with WKT2:2019
+proj=longlat +datum=WGS84 +no_defs +type=crs
# Note: only partially conformant with WKT2:2019
2393
+init=epsg:2393
+proj=tmerc +lat_0=0 +lon_0=27 +k=1 +x_0=3500000 +y_0=0 +ellps=intl +units=m +no_defs +type=crs
3067
+init=epsg:3067
+proj=utm +zone=35 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs
5125
+init=epsg:5125
+proj=tmerc +lat_0=58 +lon_0=25.5 +k=1 +x_0=100000 +y_0=1000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs +type=crs
3844
+init=epsg:3844
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=sterea +lat_0=46 +lon_0=25 +k=0.99975 +x_0=500000 +y_0=500000 +ellps=krass +units=m +no_defs +type=crs
3035
+init=epsg:3035
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +units=m +no_defs +type=crs
3995
+init=epsg:3995
# LibGTIF fails with (plain) Proj.4 definition string:
# +proj=stere +lat_0=90 +lat_ts=71 +lon_0=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +type=crs

Bounding box

The geographical area – the bounding box – of the composite is set with --cBBox taking lower left and upper right points as arguments. The points can be always given in degrees. If the applied projection is metric, the coordinates can be given as meters. If the bounding box is not explictly given with --cBBox command, the corner points of the first radar input are used.

Hint: look at additional attributes of prefix where:BBOX written in output files. These attributes display the coordinates of metric projections in native (i.e. metric) units and also report extent and overlap of input data. The bounding boxes can be used in optimising composite products. (Write of additional non-standard attributes is controlled with the --odim command. )

Initialisation

Command --cInit creates an empty compositing array of desired size in the memory and stores the essential attributes listed above. The command is optional – initialisation is implicitly invoked when the first input is introduced with --cAdd or --cAddWeighted . However, explicit command is recommended, if the user wants to get an empty image (instead of an error) in case no inputs are available.

Handling special codes

Radar data complying the ODIM standard applies special marker codes undetect and nodata . These codes have to be handled carefully in product generation and compositing. In Rack , pixel values with nodata are simply skipped; they are not accumulated in the composite array, for example.

Handling of undetect can be tuned by the user with --quantityConf and --undetectWeight . The physical value or "virtual zero" replacing undetect codes is set using --quantityConf ,
for example, --quantityConf DBZH,zero=-32.0 (corresponding the limit of signal detection sensitivity). See also Default encoding of quantities. The corresponding relative weight is set with --undetectWeight <value> . The value 0.0...1.0 describes the significance of the value relative to a "true" detected measurement value. The value is zero by default, which in compositing means that the location is marked as processed (a counter is incremented) but the value is not accumulated.

Compositing schemes

Direct compositing

Creating composites in a direct mode involves the following steps:

  1. Initialising a composite to apply projection, geographical scope, size, and compositing method
  2. For each radar, reading and processing volume data and projecting them on the array
  3. Extracting the final product from the array

The process is illustrated in the graph below.

dot_inline_dotgraph_1.png

A 800 x 1000 maximum composite over Finland in a lat-lon projection is obtained with

rack \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 800,1000 --cBBox 17,57.7,33,69.5 --cInit --cMethod MAXIMUM \
\
--script ' -Q DBZH --cAdd' \
data/pvol_fianj.h5 \
data/pvol_fiika.h5 \
data/pvol_fikor.h5 \
data/pvol_fikuo.h5 \
data/pvol_filuo.h5 \
data/pvol_fiuta.h5 \
data/pvol_fivan.h5 \
data/pvol_fivim.h5 \
--cExtract dwsc \
--gTitle 'Rack - ClutterMap ${what:date|%Y/%m}' -o composite-fi.h5

A similar composite of CAPPI in 1000 m altitude, with quality-weighted compositing method is obtained with

rack \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 --cInit --cMethod WAVG,2,2,-32 \
\
--script ' --pCappi 1000 --cAdd' \
data/pvol_fianj.h5 \
data/pvol_fiika.h5 \
data/pvol_fikor.h5 \
data/pvol_fikuo.h5 \
data/pvol_filuo.h5 \
data/pvol_fiuta.h5 \
data/pvol_fivan.h5 \
data/pvol_fivim.h5 \
--cExtract dwsc \
--gTitle 'Rack - ClutterMap ${what:date|%Y/%m}' -o composite-fi-cappi.h5

Rack installation package contains a utility script make-composite.sh which compiles a command line string - like the one above - and executes it. For example:

PRODUCT=pCappi,1000 METHOD=WAVG,2,2,-32 SIZE=500,750 BBOX=17,57.7,33,69.5 ./make-composite.sh data/pvol_fi*.h5

The extraction command --cExtract extracts one or more data layers out of the compositing (accumulation array):

  • DATA - the actual composited data, like reflectivity (DBZH )
  • WEIGHT - quality (confidence) of the data value
  • COUNT - number of radars contributed
  • DEVIATION - difference or discrepancy of data (if more than one radar contributed)

The desired combination is given as a comma-separated list of above keywords. Notice that the command –cCreate for creating single radar "composite" is equivalent to --cAdd cExtract DATA,WEIGHT . For example:

Upon initialisation of teh compositing array, Rack stores the encoding parameters – storage type, scaling and marker values (undetect , nodata) – given explicitly or adopted from the first input. Then, in extraction stage, that encoding is applied by default. The internal compositing array itself is floating point data, so encoding of the output can be also set upon extraction. Reusing the above single-radar example, as follows:

The encoding is applied to the first field (like DATA ) listed; the remaining field (typically quality fields) use their default encodings. But if desired, encoding of each field can be set, for example as follows:

For now, the old syntax is also supported: a string composed of single letters (d ), weight, (w ), deviation (std.dev., s ) and count of radars (c ).

Cropping is supported by a new optional parameter: the desired bounding box can be given explicitly <LLx>,<LLy>,<URx>,<URy> or DATA (automatic bounding box) or OVERLAP (overlapping area).

Layers extracted from the accumulation array: the composite, quality field, deviation and count of radars.

Rack makes composites of any selected quantity or polar product. Examples are shown in the image below.

Composites of CAPPI, max echo, echo top, rhoHV, and HydroClass.

Hint. Command lines may get lengthy in compositing. You can shorten file arguments by using general options --inputPrefix and --outputPrefix .

Tiled compositing

In a Tiled (distributed) mode, the composite is initialized as in direct mode but the tile generated is with --cCreateTile and then saved with --outputFile command. In the accumulation stage, the composite is loaded and then copied to the internal compositing array with --cAdd. The stages are illustrated in the diagrams below.

dot_inline_dotgraph_2.png

For example, creation of a tile:

rack \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 \
data-20140525/201405251200_radar.polar.fivim.h5 --pCappi 1000 --cCreateTile -o tiles/201405251200_radar.polar.fivim-tile.h5

Then, tiles are collectively added to composite with a single command. Notice that the composite definitions should be exactly the same as in the tile creation:

rack \
--cProj '+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs' \
--cSize 500,750 --cBBox 17,57.7,33,69.5 --cInit --cMethod WAVG,2,2,-32 \
--script '--cAdd' \
tiles/201405251200_radar.polar.fianj-tile.h5 \
tiles/201405251200_radar.polar.fiika-tile.h5 \
tiles/201405251200_radar.polar.fikor-tile.h5 \
tiles/201405251200_radar.polar.fikuo-tile.h5 \
tiles/201405251200_radar.polar.filuo-tile.h5 \
tiles/201405251200_radar.polar.fiuta-tile.h5 \
tiles/201405251200_radar.polar.fivan-tile.h5 \
tiles/201405251200_radar.polar.fivim-tile.h5 \
--cExtract dwsc \
--gTitle 'Rack - ClutterMap ${what:date|%Y/%m}' -o composite-fi-cappi.h5

(The above commands have been generated using make-composite.sh with SCHEME=TILE and SCHEME=TILED .)

Tiles containing several radars

By default, each tile contains data from a single radar. Alternatively, for a large composite, one can construct a tile containing data from several radars by calling --cBBoxTile command immediately after --cBBox and --cSize . The latter commands are equal to those of the large composite. A multi-radar tile is otherwise generated like any composite, that is, --cAdd and --cExtract commands are needed (unlike in constructing a single-radar tile).

For example, consider receiving a precomputed the OPERA composite, to which the Russian and Belarussian radars are added. First, the latter data is composited to a tile cropped with --cBBoxTile under OPERA geometry:

rack \
--cProj '+proj=laea +lat_0=55.0 +lon_0=10.0 +x_0=1950000.0 +y_0=2300000.0 +units=m +ellps=WGS84' \
--cSize 1900,2200 --cBBox -10.4346,31.7462,57.812,67.621 --cBBoxTile 24.4357,51.754,47.9115,65.4706 --cInit --cMethod MAXIMUM \
\
--script ' -Q DBZH --cAdd' \
data-east/201703071200_radar.polar.bymin.h5 \
data-east/201703071200_radar.polar.ruavn.h5 \
data-east/201703071200_radar.polar.ruudb.h5 \
data-east/201703071200_radar.polar.ruudk.h5 \
data-east/201703071200_radar.polar.ruudl.h5 \
data-east/201703071200_radar.polar.ruudp.h5 \
data-east/201703071200_radar.polar.ruudv.h5 \
data-east/201703071200_radar.polar.ruudx.h5 \
data-east/201703071200_radar.polar.ruuwj.h5 \
--cExtract dwsc \
--gTitle 'Rack - ClutterMap ${what:date|%Y/%m}' -o tile-east.h5

Finally, the two composites are combined

rack 201703071200_radar.opera.h5 --cAdd tile-east.h5 --cAdd --cExtract dw -o composite-combined.h5
Definition DataSelector.cpp:44

Notice that the geometry is implicitly determined by the first composite data read (or projected).

Continuous compositing

It is also possible to continuously update a composite. It means that a previous composite is loaded as a "warm start" and --cAddWeighted command. The size and geometrical scope are set to those of the composite file. Fading with a low weight value is motivated by substituting missing radars with older measurements with however lower confidence.

rack composite.h5 --cAddWeighted 0.5 volume.h5 --cAdd --cExtract dwsc -o composite-new.h5
dot_inline_dotgraph_4.png

Utilities

Bounding box of input data

Minimal bounding box of input data, cropped by the bounding box of the composite, is stored in how:dataBBOX variable.

Adding lat-lon grid to Cartesian images

A grid of latitudes and longitudes can be added onto Cartesiand products with --cGrid. The following command creates a grid with 5 degree spacing in latitudes and longitudes, using width of 0.9 pixels and 50% gray intensity:

rack composite.h5 --palette default --cGrid 5,5,0.9,0.5 -o composite-grid.png

The input data – a Cartesian product – can be read as a file as in this example or generated from polar data with --cCreate .