Video-Processing

Converting a GStreamer buffer PTS plus base_time into a std::chrono::steady_clock time point

From GStreamer PTS to std::chrono::steady_clock

A buffer’s PTS is running time, not wall time. Add the pipeline’s base time and you get the clock time the frame was captured at. With the monotonic system clock that is a std::chrono::steady_clock time point you can compare to IMU or telemetry timestamps directly.

Measuring GStreamer latency to an intermediate checkpoint with the latency tracer and a decoupled tee tap

GStreamer Latency to a Midpoint, Not Just the Sink

The latency tracer measures source to sink. To measure source to some point in the middle, hang a fakesink off a tee. Without queues on both branches the slow branch leaks into your number, or hangs the pipeline. Measured on GStreamer 1.24 in Docker.

Zero-copy GStreamer tracking pipeline on Jetson Orin

From a PyTorch Tracker to a Zero-Copy GStreamer Pipeline: Rebuilding SAM2.1/SAMURAI on Jetson, Step by Step

A long, hands-on account of turning a research-grade PyTorch visual tracker (SAM2.1 / SAMURAI) into a real-time, zero-copy GStreamer + CUDA + TensorRT pipeline on a Jetson Orin NX — decomposing the model into engines, exporting and parity-validating each one, porting the per-frame math to CUDA, packaging it as a GStreamer element, and squeezing 8 fps into 24 with queues and frame-skipping. Every stage is validated against a golden reference.

A cropped sensor view handing pan authority off to a physical gimbal

Two Pans, One Stick: Blending a Digital Crop Pan with a Physical Gimbal

A long-range zoom camera pans two ways at once — by sliding a crop window across the sensor, and by physically rotating the gimbal underneath it. Here’s how to make one joystick drive both so the operator never feels the seam, the maths of the handoff, and the per-axis bug that kept the gimbal rolling after the stick let go.

H.264 vs H.265 vs AV1 end-to-end latency on Jetson Orin — grouped bar chart

H.264, H.265, and AV1 on Jetson Orin: A Real Hardware Latency Benchmark

A rigorous per-stage latency benchmark across H.264, H.265, and AV1 hardware codecs on NVIDIA Jetson Orin (JetPack 6), measuring encode, wire, and decode separately at FHD and HD resolutions. AV1 wins end-to-end at 104 ms FHD / 86 ms HD. H.264 is the worst choice despite being the oldest: its nvv4l2decoder holds ~4 frames in an internal DPB buffer, adding 130–170 ms of hidden latency. Wire latency is governed by parse-element lookahead, not byte volume. Clock sync achieves ±234 µs via chrony. Full pipeline source, CSVs, and reproduction steps included.

Cross-process zero-copy NVMM IPC on Jetson — dma-buf fd passing, NvBufSurfaceImport, lock-free pool

Cross-Process Zero-Copy on Jetson: dma-buf fds, NvBufSurfaceImport, and a Cache-Line-Padded Pool

Two processes on a Jetson, one camera frame in NVMM (GPU memory), no copies. The kernel does the heavy lifting via dma-buf fds; SCM_RIGHTS carries the fd across the process boundary; NvBufSurfaceImport reconstructs the surface on the consumer side; a cache-line-padded ring of atomic ref-counts keeps fan-out coherent without locks. With benchmark numbers and a Godbolt-runnable demo of the SCM_RIGHTS pattern.

Zero-Copy Video on Jetson: Building gst-nvmm-cpp and Contributing to GStreamer

How we built a GStreamer plugin suite for zero-copy NVMM video on NVIDIA Jetson, the bugs we hit along the way, and what it takes to contribute to the GStreamer project — from filing issues to getting an MR merged.