Vision_graph/docs/spec/decisions-register.md

# Vision Graph — Decisions Register

Implementation-side canonical mirror of the decisions ratified in the `Vision graph group`
design thread. Useful-bin owns the spec prose; this file is the contract the implementation
is built against, maintained by Vision graph (implementer).

**Source of truth:** the group thread. Sections are pasted whole into the group; this file
mirrors them so the implementation has one authoritative copy under version control.

## Reference cases

| id | case | what it tests |
|----|------|---------------|
| A | Useful-bin's FrontierPlacement pipeline | **fidelity** — golden event/CSV parity against the existing pipeline |
| B | Vision builder's `blob-find → ROI-repivot → edge-intersect` (N=1) | **geometry** — expressible with typed ports instead of a global reference system |
| C | Vision builder's `LeerformRecognitionControl` | **expressiveness** — could the client that abandoned the engine have stayed in it? |

Case C matters most as a guard: an inexpressive model is not extended, it is *escaped*, and
escaping costs the editor, per-node inspection, hot reload and operator tuning in one step.
Two production clients are running that way today (`LindtLeerformPlugin`, `CandyboxPlugin`
— zero references to `WorkflowList` or `BaseOperation`, returning `errorNames: []`).

---

## Ratified decisions

- **D1 — Execution.** Push-based frame-lockstep synchronous tick, ordered fan-out. The
  normative requirement is *in-order, gap-free delivery to stateful nodes within a session*;
  lockstep is one implementation of that guarantee, not the guarantee itself. The tick never
  blocks on I/O.
- **D2 — Ports.** `frame` ports (0-or-1 typed value per tick: image, mask, scalar, point-set,
  detection, geometry) and `event` ports (0..N stamped records per tick). No third category
  for geometry — typed `frame` ports subsume it. Overlay must never be the sole carrier of
  data.
- **D3 — Fan-out.** No dynamic map/subgraph operator in v1. Probes and verdicts are addressed
  as `(nodeId, itemKey?)`. *See A1 — the register's "itemKey always null in v1" is contested.*
- **D4 — Node contract.** `(params, inputs, state) → (outputs, state', verdict, overlay)`,
  pure apart from declared state. Isolation-testability is an explicit spec goal.
  *See A2 — `state` must be split.*
- **D5 — Verdict.** `execution: ran | skipped{reason: gated | disabled | upstream-unavailable}
  | error` always present; `judgment: pass | reject` optional, inspection nodes only; plus a
  human-readable status sentence and mandatory per-node `durationMs` every tick. `skipped`
  never aggregates as `pass`.
- **D6 — Buffers.** Edge values immutable, tick-scoped, engine-owned. "Copy what you keep" is
  a stated node obligation. `mutates-input` ⇒ private copy when the producing port has >1
  consumer; pure readers share (read-only broadcast to many consumers is normal and free).
  Bounded history on an input port is a *declared* retention, bounded in both frames and
  bytes, maintained once per edge and shared by all declaring consumers.
- **D7 — Two warm-up phases, never merged.** `prewarm` (per-process resource readiness) and
  `prime` (per-session stateful convergence over real frames, all sinks suppressed).
  *See A3 — `prewarm` must be state-neutral by construction.*
- **D8 — Absent values.** Declared **per input port**: `absent→skip | last-value (with
  staleness bound) | error`. Gating (dynamic, "no value this tick" is normal) is a *different
  mechanism* from `Enabled=false` (static authoring config ⇒ pre-run validation error unless
  the node declares pass-through). Never merge them. The declaration must be visible in the
  editor next to the port — the person who gets this wrong is an operator disabling a node at
  3am, not the graph's author.
- **D9 — Manifests.** Param kinds: scalar, enum, bool, string, expression, roi, reference,
  file, lut, curve. Plus constraints, defaults, output-only flag, and per-param change class
  `live | resets-node-state | requires-rebuild`. Spatial params carry explicit coordinate
  space (`image-absolute | relative-to-connected-port`) and normalization (`normalized |
  pixels`). Persistence is generic from the manifest — nodes never hand-write it. One
  versioned format. Stable Guid ≠ label. *See A5 — label validation and instance sets.*
- **D10 — Expressions.** Kept for scalar arithmetic. Identifiers resolve **only** to the
  node's own connected ports plus graph config, by id, scoped to one execution. Never to
  another node by display label.
- **D11 — Introspection record.** `{id, itemKey?, execution, judgment?, status, durationMs,
  outputs, overlayElements, preview?}`. Overlays are renderer-agnostic draw commands.
  Previews are opt-in subscriptions, throttled, copied at emit — zero subscribers ⇒ zero cost
  *by construction*. Scalar time-series is a first-class preview type. Execute-up-to-node
  included and honours D7 phases.
- **D12 — Replay-first.** The recorded probe log *is* the serialization of the inspection
  contract; live attach is the same format with a tail; the log doubles as the golden record.
  Per-probe decimation policy; large blobs stored out-of-line by reference.
- **D13 — Determinism.** Event-level equality, not bitwise, given a deterministic source.
  Replay *forces* drop-policy off. Events ordered and stamped by `frameIndex`/`streamTime`;
  `wallClock` provably source-derived or excluded. Determinism is a declared execution mode.
  *See A4 and A6.*
- **D14 — Session generation.** Watch mode is a runner-level work source, not a graph node.
  The runner owns durable cross-session state (processed keys, append handles, spools), which
  keeps "node state never crosses sessions" true as written. Sources declare
  `continuous | per-item` session semantics.
- **D15 — Source capabilities as data.** `seekable`, `triggerable | free-running`, session
  semantics, native rate/format/resolution, drop policy. Drops always counted and reported —
  never a silent `DropOldest`. *See A4 — policy is mode-governed, not source-chosen.*
- **D16 — Config layering.** `defaults < YAML < env < per-run` is authoring-time. Export fully
  resolves into a frozen self-contained artifact with provenance; the **resolved artifact** is
  the unit of reproducibility. Overrides hitting a `resets-node-state` param are flagged as
  state resets. Environment-resolved values (e.g. codec fallback `avc1→mp4v`) are reported,
  never silent, and captured in the run record.
- **D17 — Granularity.** Node boundaries are tuning/reuse boundaries; probe boundaries are
  visibility boundaries. Never split a node to see inside it — add a probe. A node may be
  internally coarse when it declares batching.
- **D18 — Layering.** engine (no UI dependency) → runtime driver → shells. Headless is the
  same driver with nobody subscribed. Hot reload: params by node id, debounced; structural
  edits rebuild; D9 change classes govern state.
- **D19 — v2 reservations v1 must not preclude.** Map/subgraph keyed by item identity;
  identity as a first-class typed value with declared provenance; subgraph-diff rebuilds;
  parallel branches. *See A7 — add static instancing, ranked first.*
- **D20 — Acceptance tests.** Reference cases A, B, C above.

---

## Open amendments (raised by Vision graph, pending ratification)

- **A1 — `itemKey` is populated in v1, not null.** D3's "always null in v1" contradicts two
  live requirements. Useful-bin's `MetalScorer` emits one verdict *per placement*, and "see
  the results of each node" for that node means a placement-addressed list. Vision builder's
  `ArrayModelMatchingOperation` ships `Dictionary<int,bool> IsGood` — per-cell verdicts, keyed
  by grid index, in production today — which the UI cannot surface, showing only an `All()`
  rollup so the operator cannot see which cell failed. Fixed-N addressing is needed *without*
  a map operator. Only *dynamic-map* item keys are absent from v1. Left as "always null" the
  field ships dead, nothing populates it, no UI renders it, and reference case A's core
  requirement fails.
- **A2 — Split declared state into `derived` and `resource`.** D4's single `state` bucket is
  unimplementable for 14 of Vision builder's 53 operations (`InferenceSession`,
  `YoloV8Predictor`, NetMQ sockets, a spawned Docker container). `derived` state (MOG2 model,
  LK previous pyramid, latch counters, metal accumulator) is learned from the frame sequence,
  cannot be reconstructed from params, and **must serialize**. `resource` state is
  reconstructible from params and **must not serialize** — it is rebuilt by `prewarm`. A node
  snapshot is `derived` only; restoring means `prewarm` then load `derived`. This is also what
  makes snapshots cheap (serializing a background model, not an ONNX session).
- **A3 — `prewarm` must be state-neutral by construction.** Vision builder's `WarmUp()`
  executes the whole recipe against an empty source; that is safe *only because he has no
  state*. Run the same design on a graph with MOG2 and the background model trains on a black
  synthetic frame before `prime` starts — priming an already-poisoned model. Fix falls out of
  A2: the node contract separates resource acquisition from tick, `prewarm` invokes only the
  former, and cannot reach `derived` state because it is not yet allocated. ONNX shape
  specialization still requires a real inference at the declared format/resolution, so
  `prewarm` may run inference without invoking the state-update path.
- **A4 — Drop policy is a property of the execution mode, not of the source.** D15 assigns it
  per-source. Evidence against: *every* Vision builder source hardcodes
  `BoundedChannelOptions(1) { FullMode = DropOldest }`, **including the file-based emulation
  source**, which additionally drains backlog before reading. His replay path silently drops
  frames off disk, so re-running the same folder is not guaranteed to see the same frames.
  Four authors made the same local choice four times because no mode existed to distinguish
  batch from live. Resolution: *capability* is per-source (a live camera cannot block
  indefinitely; a file can), *policy* is chosen by mode and validated against capability —
  so `deterministic` mode is rejected at validation when a source declares it cannot block.
  "Determinism only holds for seekable sources" then falls out as a validation rule rather
  than a caveat someone forgets.

  | | `deterministic` (batch) | `realtime` |
  |---|---|---|
  | source | seekable | live |
  | `prime` | silent replay pass | eat first N seconds |
  | queue full | **block** | **drop + count** |
  | golden comparison | guaranteed | not claimed |

- **A5 — Authoring-time validation, and the format must admit instance sets.** Two additions
  to D9. (i) Label uniqueness *and* identifier-safety validated at authoring time: the shipped
  `gold_b97.hrcp` contains a node labelled `Red wings` — with a space, therefore not a valid
  Python identifier, so any expression referencing it throws at evaluation. It sits at a named
  client, undetected, because nothing happens to reference it. A sibling also kept its default
  label, indistinguishable among thirteen near-identical nodes. (ii) The artifact format must
  be able to represent a node *definition plus configured instance set*, not only flat
  independent node records — otherwise A7 requires a format migration.
- **A6 — Probe-log fields are marked `deterministic` or `observational`.** Per-node
  `durationMs` is mandatory (D5) and is the instrument clients use to budget cycle time, but
  it is never reproducible — same class as `wallClock`. Golden comparison runs over
  deterministic fields only. Without a declared non-deterministic field set, R13 fails on
  timing noise and someone disables the test.
- **A7 — Add "static instancing over a declared set" to D19, ranked first.** The v2 fan-out
  question was framed as subgraph-instancing vs batch ports. Production demanded neither.
  `gold_b97.hrcp` is thirteen hand-duplicated `YoloPickDetected` nodes — one per chocolate
  class — each reading the same mask with its own label, own size thresholds, own verdict.
  N is fixed and known at authoring time, so it is not dynamic map; they need thirteen
  *separately addressable* verdicts, so it is not batch ports. The shape is one node
  definition, N configured instances, per-instance params, per-instance verdict, expanded at
  authoring time. It is static (validates like a normal graph), needs no scheduler work,
  reuses the `itemKey` addressing A1 makes live, and fixes the thirteen-copies maintenance
  cost. It is the only one of the three shapes a real recipe has actually demanded.

## Consolidated lifecycle

```
build → prewarm(resource only, synthetic frame at declared format+resolution)
      → sessionBegin → prime(derived convergence, sinks suppressed)
      → tick* → flush(event-windowed nodes emit) → sessionEnd → dispose
```

`stop` (graceful: finishes tick, runs `flush`, closes sinks, drains spools) and `abort`
(discard) are distinct signals; any node may raise either and all nodes must observe both.
`flush` runs on `stop`, not on `abort`. Session generation sits **outside** this loop, in the
runner.

`execute-up-to-node` is *prime-then-prefix*, not pure prefix evaluation, and must be
**idempotent** — running it twice must not double-train a background model. Requires snapshot
restore (A2) or a fresh session per invocation; it must never mutate the live session's
`derived` state.

## Identity provenance

Identity is a first-class typed value carrying its provenance. The axis is not uniqueness but
the **strength of the continuity claim**:

| provenance | continuity |
|---|---|
| grid index | definitional — cannot be wrong |
| detection id | exact within a frame, asserts nothing across frames |
| placement / segment id | a detector's assertion — `forcedCut` proves it can be wrong |
| track id | inferred — swaps, merges, splits, occlusion re-entry |

Discontinuity events (appear, disappear, merge, split, forced-cut) must be **observable by
the node accumulating state**, so it can invalidate rather than silently carry state across.
State accumulated under a weak continuity claim without this is the same failure class as a
`NoOrigin` default at (0,0): plausible output, no diagnostic.