Motion

Motion

In radar meteorologically, motion can be divided to two phenomenon: atmospheric motion, which is motion of air and falling hydrometeors, and motion of precipitation area, which can be further divided into motion of convective and stratiform areas, for example. Rack is capable of detecting both of them.

Basic Doppler products

Weather radars capable of measuring Doppler speed produce hydrometeor velocities projected on the beam – hence one-dimensional. The obtained speeds are further aliased by the maximum unambiguous velocity, $ V_{Nyq} $. In the following, two derived products by are presented.

Doppler wind smoothing

The Doppler field can be smoothed with –pDopplerAvg command:

rack volume.h5 --pDopplerAvg '1500,3,0.5,false,false' -o pDopplerAvg.h5
% width=1500 [metres]
% height=3 [deg]
% threshold=0.5 [percentage]
% compensate=false [cart/polar[0|1]]
% relativeScale=false [false|true]

There are several parameters, determining the window width and height, minimum percentage for the samples in the window, switch for compensating the beam broadening, as well as switch for scaling the result with maximum unambiguous speed $V_{Nyq}$ (stored as how:NI in ODIM metadata).

Raw Doppler data (left) and averaged data (right).

There is also an exponential (yet asymmetric) –pDopplerAvgExp . It is recommended to use at least 16-bit resolution to store averaged fields.

rack volume.h5 --pDopplerAvgExp '0.75:0.75:0.75:0.75,0,0' -o pDopplerAvgExp.h5
% decay=0.75:0.75:0.75:0.75
% horzExtension=0 [pix]
% vertExtension=0 [pix]
Raw Doppler data (left) and averaged data (right).

Doppler wind variation

The standard deviation in Doppler field can be displayed with –pDopplerDev command.
The parameters are the same as in the Doppler averaging above.

rack volume.h5 --pDopplerDev '1500,3,0.5,false,true' -o pDopplerDev.h5
% width=1500 [metres]
% height=3 [deg]
% threshold=0.5 [percentage]
% compensate=false [cart/polar[0|1]]
% relativeScale=true [true|false]
Raw Doppler data (left) and standard deviation (right).

See also the corresponding anomaly detector: Detecting variations in Doppler wind .

Deriving two-dimensional wind field from Doppler data

rack volume.h5 --pDopplerInversion '500,3,' -o pDopplerInversion.h5
% width=500 [metres]
% height=3 [degrees]
% altitudeWeight= [Functor:a:b:c...]

Basic usage creates polar products, ie. a resulting array is in polar coordinates:

rack volume-widespread.h5 --pDopplerInversion 2500,90 -o amv-polar.h5
Definition: AndreOp.h:45

The vector field can be stored in text format to as follows:

rack volume-widespread.h5 --pDopplerInversion 2500,90 --sample 30,30,skipVoid=1 --format '${LON} ${LAT} ${AMVU} ${AMVV} ${QIND}' -o amv-polar.dat

The filename has to have "dat" as extension. See Sampling grid data for further instructions of using –sample. The output (truncated in the middle) is as follows:

Typically, user wants to obtain a Cartesian array, obtained with –cCreate, abbreviated -c . That operation has to be performed separately for u and v components, and using –append command to prevent overwriting:

rack volume-widespread.h5 --pDopplerInversion 2500,90 --append data -Q AMVU -c -Q AMVV -c -o amv-cart.h5

The resulting motion fields will be in the last available dataset group, here dataset1 . This behaviour can be changed with –append dataset which puts AMVU and AMVV in separate groups.

Similarly, cartesian sampling is done basically by changing extension .h5 to .dat

rack volume-widespread.h5 --pDopplerInversion 2500,90 --append data -Q AMVU -c -Q AMVV -c --sample 30,30,skipVoid=1 --format '${i} ${j2} ${AMVU} ${AMVV} ${QIND}' -o amv-cart.dat

producing:

Accompanied utility script draw-vectors.sh creates plots of vector fields using gnuplot utility. The original data can be in text (.dat) or HDF5 (.h5) format. Optional background images can be applied. For example:

BACKGROUND=background-dark.png OUTFILE=doppler-field.png ./draw-vectors.sh amv-cart.h5
Vector field derived by Doppler dealiasing.

Dealiasing Doppler data

As shown above, Rack produces wind fields (u,v) directly, but also de-aliased Doppler fields can be computed. This is obtained by re-projecting the approximated (u,v) data back to radar beams and matching the result with the original aliased data. This approach cancels the smoothing effect apparent in the approximated (u,v) field. Dealiasing is done by setting nyquist parameter to a desired virtual maximum speed, say 50m/s:

rack amv-polar.h5 --pDopplerReproject 50 -Q VRAD -c --palette default -o pDopplerDeAlias.png
De-aliased Doppler field.

Alternatively, one may skip matching of the original and derived velocities with matchAliased=0 , resulting in smoother images.

De-aliased Doppler field, without velocity matching.

Detecting motion fields

Rack provides commands for extracting atmospheric motion vectors (AMV's). There are two approaches:

  1. Deriving Doppler wind field in measurement volumes
  2. Deriving motion field of apparent intensity changes in subsequent Cartesian images

Both approaches are based on least-squares fitting applying matrix inversion.

Motion field apparent in subsequent images

The technique is called optical flow, and is based on computing spatial and temporal differentials. Further, the implementation in Rack applies sliding window techniques which makes the approach computationally quick even with windows of size 75x75, for example.

The command, –cOpticalFlow , takes several parameters. The first two define the size of the sliding window.

rack volume.h5 --cOpticalFlow '5:5,0,null,false,false' -o cOpticalFlow.h5
% width=5:5
% resize=0 [0.0..1.0|pix]
% threshold=null [value]
% spread=false [0|1]
% smooth=false [0|1]

The third parameter is an optional threshold, for example 0 (dBZ) in case of reflectivity data. Thresholding may be needed in order to filter apparent echo area edges caused radar geometry and sensitivity.
Smoothing (1=true) is always recommended especially with radar images that have large areas of empty data ie. "undetected" echoes. Alternatively, radar data can be smoothed by some separate process or previous commands, say –iAverage or –iGaussianAverage.

The input should contain two Cartesian datasets, for example composites. An optimal time difference between the files is typically 15 or 30 minutes. Such file can be generated for example with

rack --append dataset composite-1130.h5 composite-1200.h5 -o composite-pair.h5

The actual processing is done as follows:

rack composite-pair.h5 --cOpticalFlow 25,25,resize=250,threshold=0,smooth=1 -o motion.h5

The resulting file contains horizontal motion vector (u, v) stored in quantities AMVU and AMVV . In addition, a quality field (with QIND ) is provided. In addition, if –store intermediate=1 is set, then the following fields are stored:

  1. Original two intensity fields (typically DBZH or RATE)
  2. Smoothed fields (for example DBZH_smooth)
  3. Derivatives - horz and vertical gradients, and time differential (for example DBZH_x, DBZH_y, DBZH_t)

Note that the above commands can be combined to a single command line, avoiding creation of intermediate file (composite-pair.h5 ).

As with Doppler motion, script draw-vectors.sh can be used for visualizing the result:

OUTFILE=motion.png LINEWIDTH=2 TRANSP=0 ./draw-vectors.sh motion.h5
Motion vectors extracted with optical flow.

Spreading motion fields

Both methods described above produce an incomplete motion field: Doppler dealiasing provides realiable vectors inside wide-spread precipitation whereas optical flow based motion analysis works best with visual details like edges and specks. Rack provides some methods for spreading the vector fields, with optional support of quality information. The methods are essentially image operators described in Working with images.

The spreading methods for two dimensional motion vectors are illustrated using an artificial image of "clouds" shown below. In vectors, high, intermediate, and low quality are illustrated in green, blue and pink colour, respectively. Utility script draw-vectors.sh and GnuPlot have been used in creating the visualisations below.

Artificial sparse vector field (left) and its quality field (right).

Smoothing operators

An straighforward way to spread scalar or vector data is to apply a smoothing window on data. In Rack, fast and simple smoothing is carried out using rectangular averaging window:

rack motion.h5 --iAverage 200:200 -o motion-Average.h5
Vector field spread with --iAverage, window size 200 x 200.

As a computational speed-up, the rectangular averaging window of size $ W\times H $ is internally performed by $ W\times 1 $ followed by $ 1\times H $ window averaging; the narrow windows themselves are updated (accumulated) continuously, as a "pipeline", resulting in a very fast operation. As a disadvantage, rectangular patterns may also appear in the resulting images.

Two-dimensional Gaussian bell function produces smooth fields. Also that operator can be a separated to two consecutive one-dimensional run of $ W\times 1 $ and $ 1\times H $ windows, yet without pipelining. The bell curve is principally continuous and infinite, but practically truncated to size slightly larger than those of respective rectangular averaging windows.

rack motion.h5 --iGaussianAverage 200:200 -o motion-GaussianAverage.h5
Vector field spread with --iGaussianAverage, window size 400 x 400.

Sometimes it is desired that opposite vector components do not cancel each other but the magnitudes be preserved like in a flow a fluid or particles. Using –iFlowAverage produces such vectors, pointing to directions of conventionally averaged vectors.

rack motion.h5 --iFlowAverage 200:200 -o motion-FlowAverage.h5
Vector field spread with --iFlowAverage, window size 200 x 200.

The shape of this window operator is rectangular, and basically cannot be accelerated ie. separated to two operations with narrow windows. Technically, such processing can be performed by issuing the commands explicitly, producing an approximation of the true rectangular window operation:

rack motion.h5 --iFlowAverage 200:1 --iFlowAverage 1:200 -o motion-FlowAverageApprox.h5

The problem with all the averaging operators is that they also blur areas of high-quality vectors. As a solution, the vectors of low quality in the resulting image should be overridden with vectors of higher quality of the original image. This kind of combination can be produced repeatedly using –iBlender which employs a desired averaging operator (above) with a maximum or mixing operator with the original image:

rack motion.h5 --iBlender 200:200 -o motion-Blender.h5
Vector field spread with --iBlender, window size 200 x 200, applying simple averaging and maximum-quality combination.

Distance transform based operators

An alternative way to spead vector fields is to apply distance transform based methods: data values are propagated in all the directions such that the underlying quality field is decreased as a function of distance.
Propagation continues in low-quality areas until higher quality values are encountered. This method is very fast, but as a disadvantage the resulting vector fields contain segments of constant values as no averaging is involved. The basic version is applies linear model ie. constant steps in decreasing the underlying quality field. If the data has no quality data, it can be created with –createDefaultQuality .

rack motion.h5 --iDistanceTransformFill 200:200 -o motion-DistanceTransformFill.h5
Vector field spread with --iDistanceTransformFill, window size 200 x 200.

When using distance transform approach, an area of high quality may override an area of lower quality if the areas are close enough like in the upper right corner of the sample image (the motion field of the pentagon overrides that of the rectangle). When using an exponential model, the resulting quality field has steeply decaying values - that however continue to infinity, in the limits of storage type precision.

rack motion.h5 --iDistanceTransformFillExp 200:200 -o motion-DistanceTransformFillExp.h5
Vector field spread with --iDistanceTransformFillExp, window size 200 x 200.

The linear and exponential approaches have same the computional complexity.

Composites of motion fields

It is possible to generate composites of single-radar motion fields presented above. The procedure is essenstially the same as with intensity data, explained in Cartesian conversions and composites . However, compositing has to be done separately for each parameter, in this case horizontal wind components AMVU and AMVV .

dot_inline_dotgraph_6.png

For example, assuming that –c <files> are a set of motion files to be composites:

Q=AMVU; rack --cMethod MAXW --script "-Q $Q --cAdd" <files> --cExtract dw -o motion-$Q.h5
Q=AMVV; rack --cMethod MAXW --script "-Q $Q --cAdd" <files> --cExtract dw -o motion-$Q.h5
rack --append data motion-AMV?.h5 -o <motion-composite>

Note that the input files can be the same in the above lines, if they contain both the quantities.