Accumulation products

Accumulation products

Rack can be used to accumulate polar and Cartesian data - raw data or products. Cumulative rainfall or clutter maps are examples of accumulation products than can be generated.

Internally, the accumulation process uses the same modules as in compositing: an accumulation array is first updated, and finally an accumulation product is extracted.

Accumulation products

Basically, data in polar coordinates (ie. sweeps or polar products) can be accumulated to an array by issuing input files proceeded with –pAdd command, one by one. More conveniently the command is set with –script and the inputs are given as a list. Finally, the accumulated data is extracted with –pExtract . For example:

# 'Undetect' values replaced with physical low value
rack --select /dataset1,quantity=DBZH --script '--pAdd' data-acc/2017*.h5 --encoding C --pExtract 'dw' -o accumulated0.h5
# 'Undetect' values discarded
rack --select /dataset1,quantity=DBZH --undetectWeight 0 --script '--pAdd' data-acc/2017*.h5 --encoding C --pExtract 'dw' -o accumulated1.h5
Definition: DataSelector.cpp:44

In these examples, input selector –select /dataset1,quantity=DBZH is used for faster file read.

Input data for accumulation (DBZH ).

Like in compositing, bins marked undetect can be either replaced with physical values (set with –quantityConf ) or discarded (with –undetectWeight 0 ).

Result of accumulation: with (left) or without (right) undetect replacement (-32dBZ).

The desired data storage type and encoding can be set with –encoding command. Issuing it explicitly (before –pAdd command) prevents warnings of a limited value range.

Accumulation can be incremental, that is, one may continue accumulation on a previously computed result. The new data will be assigned weight 1/N which normalizes the accumulation.
For example, the above set of 25 volumes can be accumulated as follows, as monthly sets:

# Cold start
rack --select /dataset1,quantity=DBZH --undetectWeight 0 --script '--pAdd' data-acc/20170[2-5]*.h5 --pExtract 'dw' -o acc-part.h5
# Warm start: read previous accumulation as a background
rack --select /dataset1,quantity=DBZH --undetectWeight 0 --script '--pAdd' acc-part.h5 data-acc/201706*.h5 --pExtract 'dw' -o accumulated.h5

By default, quantities DBZH and RATE are used for accumulation. This can be changed with –select . For example, accumulation of total reflectivity (TH ) is obtained with:

rack --encoding S --select 'quantity=^TH$' --script '--pAdd' <files> --pExtract 'dw' -o accumulated.h5

Application: creating clutter maps

As can be seen from above images, accumulating data without quality control produces anomalies in results, especially wind turbines and other clutter that has not been removed even by Doppler filtering. Utility script make-cluttermap.sh accumulates data and creates a clutter map applicable with anomaly detection command –aClutter (Detecting ground clutter) . The accumulation part is similar to the processes described above. After obtaining the accumulated DBZH field it can converted to CLUTTER for example as follows:

rack accumulated0.h5 -Q DBZH --encoding C,0.004,-0.004,quantity=CLUTTER --iFuzzyStep -20:21 --keep quantity=CLUTTER -o cluttermap.h5

One can say that the above command practically defines a clutter map. In other words, Rack contains no built-in support in creating CLUTTER quantity but the user may use general purpose commands as above to produce a clutter map. In this example, the accumulated DBZH data is simply rescaled by a fuzzy mapping and the quantity label is changed to CLUTTER. Other fuzzy functions can be considered as well (see –iFuzzy prefixed commands in List of image operators).

The above command appends the CLUTTER data group in the file. The original DBZH group can be removed with –keep quantity=CLUTTER . Further, in batches generating clutter maps for several radar sites the filenames can be automatically derived by using –expandVariables followed by filenames containing implicit variables like 'cluttermaps/clutterMap-${NOD}.h5'. (See Formatting metadata output using templates for details).

Accumulating Cartesian data

Accumulating data that has already been projected to a Cartesian coordinate system can be accumulated using –cAdd command . This process is technically similar to compositing; now a single radar is repeatedly "composited" on itself.

(Under construction.)