Mutation gate

A green suite and a blind suite look identical from the outside. Nothing in this repo measured the difference until now.

The gate rewrites one operator or one numeric literal on a line the diff touched, rebuilds, and runs the suite. A mutant the suite still passes is a hole in the tests, and it blocks the commit until a test kills it or a committed waiver in .mutation-gate-waivers.toml says why none can. That file does not exist yet; the gate prints a ready-to-paste stanza when it blocks. Add line = <n> to a stanza waiving a numeric literal — the gate refuses a bare literal without one, because old = "0" alone would waive every 0 => 1 in the file. It runs from a pre-commit hook and from a Claude Code Stop hook; whichever runs first satisfies the other. Spec and the decisions behind it: PavelGuzenfeld/dotfiles#9.

Running it

pip install pre-commit && pre-commit install   # then it rides `git commit`
mutation-gate --worktree                       # on demand, unstaged changes
mutation-gate --file gst/common/kalman_box.cpp # every line of one file
mutation-gate --file analytics/image_ops.hpp --dry-run   # count, run nothing

It needs ast-grep on PATH and a built gst-nvmm-cpp:dev image:

docker build -f docker/Dockerfile.dev -t gst-nvmm-cpp:dev .

The gate configures and reuses builddir-gate/ inside that image (-Danalytics=enabled, C++14, debug). Nothing else in the tree writes there.

What it costs

Measured in gst-nvmm-cpp:dev on a 20-core x86 host, mock NvBufSurface:

Step	Wall
Clean configure + build (107 edges)	2.1 s
Rebuild after touching analytics/image_ops.hpp	0.78 s
Rebuild after touching gst/common/nvmm_buffer.hpp	0.53 s
Rebuild after touching gst/common/kalman_box.hpp	0.25 s
Suite, all 32 tests	14.3 s
Suite, --no-suite fuzz (30 tests)	2.4 s
Full gate command end to end (docker + build + 30 tests)	1.9 s

The two fuzz tests are 12.4 s of that 14.3 s and re-seed every run, so the gate excludes the fuzz suite. CI still runs them — ci.yml calls plain meson test, which takes every suite.

This is why there is no per-line test selection here. coverage.py contexts are the gate's only narrowing mechanism and there is no C++ equivalent wired, so every mutant runs the whole suite. At ~3 s a mutant that is the cheaper answer, and a selection heuristic that missed the killing test would report a false SURVIVED.

Measured kill rate

gst/common/kalman_box.cpp, every line, 165 mutants: 142 killed, 23 survived — 86%. Baseline suite 2.3 s, whole sweep ~28 min.

A surviving mutant is a hypothesis, not a verdict, so all 23 were rebuilt individually and run against a probe that drives initiate, predict, update, shift and gating_distance. 22 of them produce bit-identical output — equivalent mutants that no test can kill:

• Seven enlarge a fixed-size array declaration, const double std[8] to [9], std::array<double, 4> z{} to <double, 5>. Every loop over them is bounded by its own literal, so the extra slot is written by nobody and read by nobody.
• The other fifteen are the off-diagonal machinery of chol4 and the two triangular solves. Each coordinate in this model carries its own independent position/velocity pair, so the 4x4 block those routines factor is diagonal at every step — measured, exactly zero off-diagonal, never a rounding residue. The branches the mutants alter never execute. covariance_never_correlates_two_coordinates pins that invariant, so the day the model gains cross-coordinate coupling the suite says so.

The twenty-third was real. s > 0.0 to s > 1 in the chol4 pivot guard clamps any small pivot to zero, not just a negative one, and the suite never drove the filter to a pivot below 1. gating_distance_matches_the_closed_form_at_a_pivot_below_one does, and checks the result against the closed form for a diagonal covariance — an oracle that shares no code with the factorisation it is checking. It kills that mutant and leaves the other 22 alone.

So the honest reading of 86% is not "14% of this file is untested". It is one hole, now closed, and 22 mutants the catalogue should not have generated.

Those 22 are deliberately not in a waiver file. A waiver matches on file, line and text, so it would stop matching the moment anyone edits chol4 — which is exactly when you want to be asked again — and 22 standing suppressions cost more than this paragraph. Waive one when you touch the line and the gate stops you.

What it cannot see

The gate is only as wide as the image it builds in. gst-nvmm-cpp:dev has no TensorRT, VPI or CUDA toolkit, so meson.build never descends into nvmmofa, nvmminfer, nvmmsecondaryinfer, nvmmdrawdet, nvmmsamurai or nvmmdetgate, and probes/ has no meson target at all. A mutant in any of them survives for want of a build, not for want of a test — measured: all 70 mutants on gst/nvmmsamurai/samurai_seed_math.hpp survived for exactly that reason. Those files are listed in exclude_paths in .mutation-gate.toml and the gate announces each one it skips.

Eight files under those directories do compile in the dev image, because a test builds them directly, and the gate covers them: yolo_parser.{cpp,hpp}, secondary_cache.{cpp,hpp}, samurai_view.hpp, samurai_gmc.hpp, gmc_mask.hpp and gmc_backend.hpp. Their first full run (#80, 2026-10-01) was 235 mutants, 115 killed, 120 survived. Those survivors are untested boundaries, not build artifacts, and are tracked in #85.

The rest only build on a Jetson. To gate one there, run the gate from a scratch checkout whose test_command rsyncs the tree to an Orin and runs ninja && meson test --no-suite fuzz in a warm native build dir configured with -Danalytics=enabled -Danalytics_cuda=enabled. JetPack 6.2 ships CUDA, TensorRT and VPI on the host, so the build needs no container, and /dev/nvmap is reachable for the hwlib suite. Set mutant_timeout explicitly: a mutant there costs about 12 s (rebuild + suite on 8 cores) against a 7.6 s baseline. The first on-device run (JetPack 6.2, L4T R36.4.3):

file	mutants	killed	survived
samurai_seed_math.hpp	70	45	25
samurai_memory.hpp	59	52	7
samurai_consts.hpp	36	12	24
samurai_kernels.cu	134	93	41
analytics_kernels.cu	248	188	60
gstnvmmflowstats.cpp	26	22	4
gstnvmmofa.cpp	57	45	12
gstnvmmdrawdet.cpp	210	101	109
gstnvmminfer.cpp	119	67	52
preprocess.cpp	102	84	18
preprocess.hpp	2	1	1
trt_engine.cpp	37	18	19
trt_engine.hpp	2	0	2
gstnvmmsecondaryinfer.cpp	141	79	62
roi_preprocess.cpp	121	94	27
roi_preprocess.hpp	2	0	2
gstnvmmdetgate.cpp	90	73	17
detgate.hpp	120	107	13
gstnvmmsamurai.cpp	92	87	5
samurai_tracker.cpp	492	331	161

samurai_seed_math.hpp is the before/after: 0 of 70 killed in the dev image, 45 on-device. #91 added tests for these; its PR lists the mutants still open. The element rows (from gstnvmmflowstats.cpp down) are the on-device run of the #86 element tests (nvmm_ofa, nvmm_drawdet, nvmm_preprocess, nvmm_trt_elements), each mutant running only the test executables that link its file; the gstnvmminfer.cpp row predates its two head-shape tests. These files stay in exclude_paths because the x86 lane cannot build them, not for want of a test. The last four rows are the on-device run after #133, at 11476e7, with the test engines: before it gstnvmmdetgate.cpp, detgate.hpp, gstnvmmsamurai.cpp and samurai_tracker.cpp killed 35/88, 30/120, 25/92 and 262/492. Each mutant ran only nvmm_detgate or nvmm_samurai plus pure_cpp_headers.

analytics/ headers are reached only through the component tests (-Danalytics=enabled). The OpenCV golden oracle (-Danalytics_golden) does not build in that image either, so a line only the golden tests exercise is gated against the component tests alone.

A mutant that fails to compile makes the command non-zero and is counted KILLED. That is deliberate — it keeps the run moving — but it inflates the kill rate: an invalid mutant is not evidence a test is watching.