Vision_graph/docs/spec/decisions-register.md

Vision Graph — Decisions Register

Implementation-side canonical mirror of the decisions ratified in the Vision graph group design thread. Useful-bin owns the spec prose; this file is the contract the implementation is built against, maintained by Vision graph (implementer).

Source of truth: the group thread. Sections are pasted whole into the group; this file mirrors them so the implementation has one authoritative copy under version control.

Reference cases

id	case	what it tests
A	Useful-bin's FrontierPlacement pipeline	fidelity — golden event/CSV parity against the existing pipeline
B	Vision builder's blob-find → ROI-repivot → edge-intersect (N=1)	geometry — expressible with typed ports instead of a global reference system
C	Vision builder's LeerformRecognitionControl	expressiveness — could the client that abandoned the engine have stayed in it?

Case C matters most as a guard: an inexpressive model is not extended, it is escaped, and escaping costs the editor, per-node inspection, hot reload and operator tuning in one step. Two production clients are running that way today (LindtLeerformPlugin, CandyboxPlugin — zero references to WorkflowList or BaseOperation, returning errorNames: []).

---

Ratified decisions

• D1 — Execution. Push-based frame-lockstep synchronous tick, ordered fan-out. The normative requirement is in-order, gap-free delivery to stateful nodes within a session; lockstep is one implementation of that guarantee, not the guarantee itself. The tick never blocks on I/O.
• D2 — Ports. frame ports (0-or-1 typed value per tick: image, mask, scalar, point-set, detection, geometry) and event ports (0..N stamped records per tick). No third category for geometry — typed frame ports subsume it. Overlay must never be the sole carrier of data.
• D3 — Fan-out. No dynamic map/subgraph operator in v1. Probes and verdicts are addressed as (nodeId, itemKey?). See A1 — the register's "itemKey always null in v1" is contested.
• D4 — Node contract. (params, inputs, state) → (outputs, state', verdict, overlay), pure apart from declared state. Isolation-testability is an explicit spec goal. See A2 — state must be split.
• D5 — Verdict. execution: ran | skipped{reason: gated | disabled | upstream-unavailable} | error always present; judgment: pass | reject optional, inspection nodes only; plus a human-readable status sentence and mandatory per-node durationMs every tick. skipped never aggregates as pass.
• D6 — Buffers. Edge values immutable, tick-scoped, engine-owned. "Copy what you keep" is a stated node obligation. mutates-input ⇒ private copy when the producing port has >1 consumer; pure readers share (read-only broadcast to many consumers is normal and free). Bounded history on an input port is a declared retention, bounded in both frames and bytes, maintained once per edge and shared by all declaring consumers.
• D7 — Two warm-up phases, never merged. prewarm (per-process resource readiness) and prime (per-session stateful convergence over real frames, all sinks suppressed). See A3 — prewarm must be state-neutral by construction.
• D8 — Absent values. Declared per input port: absent→skip | last-value (with staleness bound) | error. Gating (dynamic, "no value this tick" is normal) is a different mechanism from Enabled=false (static authoring config ⇒ pre-run validation error unless the node declares pass-through). Never merge them. The declaration must be visible in the editor next to the port — the person who gets this wrong is an operator disabling a node at 3am, not the graph's author.
• D9 — Manifests. Param kinds: scalar, enum, bool, string, expression, roi, reference, file, lut, curve. Plus constraints, defaults, output-only flag, and per-param change class live | resets-node-state | requires-rebuild. Spatial params carry explicit coordinate space (image-absolute | relative-to-connected-port) and normalization (normalized | pixels). Persistence is generic from the manifest — nodes never hand-write it. One versioned format. Stable Guid ≠ label. See A5 — label validation and instance sets.
• D10 — Expressions. Kept for scalar arithmetic. Identifiers resolve only to the node's own connected ports plus graph config, by id, scoped to one execution. Never to another node by display label.
• D11 — Introspection record. {id, itemKey?, execution, judgment?, status, durationMs, outputs, overlayElements, preview?}. Overlays are renderer-agnostic draw commands. Previews are opt-in subscriptions, throttled, copied at emit — zero subscribers ⇒ zero cost by construction. Scalar time-series is a first-class preview type. Execute-up-to-node included and honours D7 phases.
• D12 — Replay-first. The recorded probe log is the serialization of the inspection contract; live attach is the same format with a tail; the log doubles as the golden record. Per-probe decimation policy; large blobs stored out-of-line by reference.
• D13 — Determinism. Event-level equality, not bitwise, given a deterministic source. Replay forces drop-policy off. Events ordered and stamped by frameIndex/streamTime; wallClock provably source-derived or excluded. Determinism is a declared execution mode. See A4 and A6.
• D14 — Session generation. Watch mode is a runner-level work source, not a graph node. The runner owns durable cross-session state (processed keys, append handles, spools), which keeps "node state never crosses sessions" true as written. Sources declare continuous | per-item session semantics.
• D15 — Source capabilities as data. seekable, triggerable | free-running, session semantics, native rate/format/resolution, drop policy. Drops always counted and reported — never a silent DropOldest. See A4 — policy is mode-governed, not source-chosen.
• D16 — Config layering. defaults < YAML < env < per-run is authoring-time. Export fully resolves into a frozen self-contained artifact with provenance; the resolved artifact is the unit of reproducibility. Overrides hitting a resets-node-state param are flagged as state resets. Environment-resolved values (e.g. codec fallback avc1→mp4v) are reported, never silent, and captured in the run record.
• D17 — Granularity. Node boundaries are tuning/reuse boundaries; probe boundaries are visibility boundaries. Never split a node to see inside it — add a probe. A node may be internally coarse when it declares batching.
• D18 — Layering. engine (no UI dependency) → runtime driver → shells. Headless is the same driver with nobody subscribed. Hot reload: params by node id, debounced; structural edits rebuild; D9 change classes govern state.
• D19 — v2 reservations v1 must not preclude. Map/subgraph keyed by item identity; identity as a first-class typed value with declared provenance; subgraph-diff rebuilds; parallel branches. See A7 — add static instancing, ranked first.
• D20 — Acceptance tests. Reference cases A, B, C above.

---

Open amendments (raised by Vision graph, pending ratification)

• A1 — itemKey is populated in v1, not null. D3's "always null in v1" contradicts two live requirements. Useful-bin's MetalScorer emits one verdict per placement, and "see the results of each node" for that node means a placement-addressed list. Vision builder's ArrayModelMatchingOperation ships Dictionary<int,bool> IsGood — per-cell verdicts, keyed by grid index, in production today — which the UI cannot surface, showing only an All() rollup so the operator cannot see which cell failed. Fixed-N addressing is needed without a map operator. Only dynamic-map item keys are absent from v1. Left as "always null" the field ships dead, nothing populates it, no UI renders it, and reference case A's core requirement fails.

• A2 — Split declared state into derived and resource. D4's single state bucket is unimplementable for 14 of Vision builder's 53 operations (InferenceSession, YoloV8Predictor, NetMQ sockets, a spawned Docker container). derived state (MOG2 model, LK previous pyramid, latch counters, metal accumulator) is learned from the frame sequence, cannot be reconstructed from params, and must serialize. resource state is reconstructible from params and must not serialize — it is rebuilt by prewarm. A node snapshot is derived only; restoring means prewarm then load derived. This is also what makes snapshots cheap (serializing a background model, not an ONNX session).

• A3 — prewarm must be state-neutral by construction. Vision builder's WarmUp() executes the whole recipe against an empty source; that is safe only because he has no state. Run the same design on a graph with MOG2 and the background model trains on a black synthetic frame before prime starts — priming an already-poisoned model. Fix falls out of A2: the node contract separates resource acquisition from tick, prewarm invokes only the former, and cannot reach derived state because it is not yet allocated. ONNX shape specialization still requires a real inference at the declared format/resolution, so prewarm may run inference without invoking the state-update path.

• A4 — Drop policy is a property of the execution mode, not of the source. D15 assigns it per-source. Evidence against: every Vision builder source hardcodes BoundedChannelOptions(1) { FullMode = DropOldest }, including the file-based emulation source, which additionally drains backlog before reading. His replay path silently drops frames off disk, so re-running the same folder is not guaranteed to see the same frames. Four authors made the same local choice four times because no mode existed to distinguish batch from live. Resolution: capability is per-source (a live camera cannot block indefinitely; a file can), policy is chosen by mode and validated against capability — so deterministic mode is rejected at validation when a source declares it cannot block. "Determinism only holds for seekable sources" then falls out as a validation rule rather than a caveat someone forgets.

  	deterministic (batch)	realtime
  source	seekable	live
  prime	silent replay pass	eat first N seconds
  queue full	block	drop + count
  golden comparison	guaranteed	not claimed

• A5 — Authoring-time validation, and the format must admit instance sets. Two additions to D9. (i) Label uniqueness and identifier-safety validated at authoring time: the shipped gold_b97.hrcp contains a node labelled Red wings — with a space, therefore not a valid Python identifier, so any expression referencing it throws at evaluation. It sits at a named client, undetected, because nothing happens to reference it. A sibling also kept its default label, indistinguishable among thirteen near-identical nodes. (ii) The artifact format must be able to represent a node definition plus configured instance set, not only flat independent node records — otherwise A7 requires a format migration.

• A6 — Probe-log fields are marked deterministic or observational. Per-node durationMs is mandatory (D5) and is the instrument clients use to budget cycle time, but it is never reproducible — same class as wallClock. Golden comparison runs over deterministic fields only. Without a declared non-deterministic field set, R13 fails on timing noise and someone disables the test.

• A7 — Add "static instancing over a declared set" to D19, ranked first. The v2 fan-out question was framed as subgraph-instancing vs batch ports. Production demanded neither. gold_b97.hrcp is thirteen hand-duplicated YoloPickDetected nodes — one per chocolate class — each reading the same mask with its own label, own size thresholds, own verdict. N is fixed and known at authoring time, so it is not dynamic map; they need thirteen separately addressable verdicts, so it is not batch ports. The shape is one node definition, N configured instances, per-instance params, per-instance verdict, expanded at authoring time. It is static (validates like a normal graph), needs no scheduler work, reuses the itemKey addressing A1 makes live, and fixes the thirteen-copies maintenance cost. It is the only one of the three shapes a real recipe has actually demanded.

Consolidated lifecycle

build → prewarm(resource only, synthetic frame at declared format+resolution)
      → sessionBegin → prime(derived convergence, sinks suppressed)
      → tick* → flush(event-windowed nodes emit) → sessionEnd → dispose

stop (graceful: finishes tick, runs flush, closes sinks, drains spools) and abort (discard) are distinct signals; any node may raise either and all nodes must observe both. flush runs on stop, not on abort. Session generation sits outside this loop, in the runner.

execute-up-to-node is prime-then-prefix, not pure prefix evaluation, and must be idempotent — running it twice must not double-train a background model. Requires snapshot restore (A2) or a fresh session per invocation; it must never mutate the live session's derived state.

Identity provenance

Identity is a first-class typed value carrying its provenance. The axis is not uniqueness but the strength of the continuity claim:

provenance	continuity
grid index	definitional — cannot be wrong
detection id	exact within a frame, asserts nothing across frames
placement / segment id	a detector's assertion — forcedCut proves it can be wrong
track id	inferred — swaps, merges, splits, occlusion re-entry

Discontinuity events (appear, disappear, merge, split, forced-cut) must be observable by the node accumulating state, so it can invalidate rather than silently carry state across. State accumulated under a weak continuity claim without this is the same failure class as a NoOrigin default at (0,0): plausible output, no diagnostic.