nvmmfusion¶

Join the detector and optical-flow branches by PTS — a GstAggregator with two named sink pads that unions both branch metas onto a single output buffer:

detection — NVMM NV12 carrying GstNvmmDetMeta (nvmminfer → nvmmtracker)
flow — NVMM NV12 carrying NvmmOpticalFlowMeta (nvmmofa)

Per output frame the detection buffer is the carrier (zero-copy — same NvBufSurface, no pixel work, no CUDA) and the flow meta is added onto it, so a single buffer downstream holds both metas at one PTS. PTS is the join key: tee copies timestamps verbatim so the queue heads pair naturally; on a mismatch (a branch dropped a frame) the older head is dropped and the join resyncs.

Either branch reaching EOS ends the fused stream (inner-join semantics).

Motion annotation¶

With both metas available at the join, fusion computes each detection's mean flow magnitude (pixels/frame) from the flow cells under its box and attaches a GstNvmmMotionMeta — entries align by index with the det meta's objects. The threshold only sets the convenience moving flag; mean_px is stored alongside so consumers can re-threshold without reconfiguring fusion. nvmmdrawdet renders movers with a >> label suffix and a double-thickness box.

Property	Type	Default	Notes
`compute-motion`	bool	`true`	Compute + attach the motion meta
`motion-threshold`	double	`1.0`	px/frame at/above which an object is `moving`

gst-launch-1.0 filesrc location=video.h264 ! h264parse ! nvv4l2decoder \
  ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! tee name=t \
  t. ! queue ! nvmminfer engine-file=yolo.engine ! nvmmtracker ! queue ! f.detection \
  t. ! queue ! nvmmofa ! queue ! f.flow \
  nvmmfusion name=f ! fakesink
# GST_DEBUG=nvmmfusion:6 logs:  fused: 8 detection(s) + flow=yes on one buffer

A downstream consumer verifies the union by reading both metas off one buffer: gst_buffer_get_nvmm_det_meta(buf) and gst_buffer_get_nvmm_optical_flow_meta(buf) are both non-NULL after fusion.