diff --git a/docs/spec/decisions-register.md b/docs/spec/decisions-register.md new file mode 100644 index 0000000..4912762 --- /dev/null +++ b/docs/spec/decisions-register.md @@ -0,0 +1,207 @@ +# Vision Graph — Decisions Register + +Implementation-side canonical mirror of the decisions ratified in the `Vision graph group` +design thread. Useful-bin owns the spec prose; this file is the contract the implementation +is built against, maintained by Vision graph (implementer). + +**Source of truth:** the group thread. Sections are pasted whole into the group; this file +mirrors them so the implementation has one authoritative copy under version control. + +## Reference cases + +| id | case | what it tests | +|----|------|---------------| +| A | Useful-bin's FrontierPlacement pipeline | **fidelity** — golden event/CSV parity against the existing pipeline | +| B | Vision builder's `blob-find → ROI-repivot → edge-intersect` (N=1) | **geometry** — expressible with typed ports instead of a global reference system | +| C | Vision builder's `LeerformRecognitionControl` | **expressiveness** — could the client that abandoned the engine have stayed in it? | + +Case C matters most as a guard: an inexpressive model is not extended, it is *escaped*, and +escaping costs the editor, per-node inspection, hot reload and operator tuning in one step. +Two production clients are running that way today (`LindtLeerformPlugin`, `CandyboxPlugin` +— zero references to `WorkflowList` or `BaseOperation`, returning `errorNames: []`). + +--- + +## Ratified decisions + +- **D1 — Execution.** Push-based frame-lockstep synchronous tick, ordered fan-out. The + normative requirement is *in-order, gap-free delivery to stateful nodes within a session*; + lockstep is one implementation of that guarantee, not the guarantee itself. The tick never + blocks on I/O. +- **D2 — Ports.** `frame` ports (0-or-1 typed value per tick: image, mask, scalar, point-set, + detection, geometry) and `event` ports (0..N stamped records per tick). No third category + for geometry — typed `frame` ports subsume it. Overlay must never be the sole carrier of + data. +- **D3 — Fan-out.** No dynamic map/subgraph operator in v1. Probes and verdicts are addressed + as `(nodeId, itemKey?)`. *See A1 — the register's "itemKey always null in v1" is contested.* +- **D4 — Node contract.** `(params, inputs, state) → (outputs, state', verdict, overlay)`, + pure apart from declared state. Isolation-testability is an explicit spec goal. + *See A2 — `state` must be split.* +- **D5 — Verdict.** `execution: ran | skipped{reason: gated | disabled | upstream-unavailable} + | error` always present; `judgment: pass | reject` optional, inspection nodes only; plus a + human-readable status sentence and mandatory per-node `durationMs` every tick. `skipped` + never aggregates as `pass`. +- **D6 — Buffers.** Edge values immutable, tick-scoped, engine-owned. "Copy what you keep" is + a stated node obligation. `mutates-input` ⇒ private copy when the producing port has >1 + consumer; pure readers share (read-only broadcast to many consumers is normal and free). + Bounded history on an input port is a *declared* retention, bounded in both frames and + bytes, maintained once per edge and shared by all declaring consumers. +- **D7 — Two warm-up phases, never merged.** `prewarm` (per-process resource readiness) and + `prime` (per-session stateful convergence over real frames, all sinks suppressed). + *See A3 — `prewarm` must be state-neutral by construction.* +- **D8 — Absent values.** Declared **per input port**: `absent→skip | last-value (with + staleness bound) | error`. Gating (dynamic, "no value this tick" is normal) is a *different + mechanism* from `Enabled=false` (static authoring config ⇒ pre-run validation error unless + the node declares pass-through). Never merge them. The declaration must be visible in the + editor next to the port — the person who gets this wrong is an operator disabling a node at + 3am, not the graph's author. +- **D9 — Manifests.** Param kinds: scalar, enum, bool, string, expression, roi, reference, + file, lut, curve. Plus constraints, defaults, output-only flag, and per-param change class + `live | resets-node-state | requires-rebuild`. Spatial params carry explicit coordinate + space (`image-absolute | relative-to-connected-port`) and normalization (`normalized | + pixels`). Persistence is generic from the manifest — nodes never hand-write it. One + versioned format. Stable Guid ≠ label. *See A5 — label validation and instance sets.* +- **D10 — Expressions.** Kept for scalar arithmetic. Identifiers resolve **only** to the + node's own connected ports plus graph config, by id, scoped to one execution. Never to + another node by display label. +- **D11 — Introspection record.** `{id, itemKey?, execution, judgment?, status, durationMs, + outputs, overlayElements, preview?}`. Overlays are renderer-agnostic draw commands. + Previews are opt-in subscriptions, throttled, copied at emit — zero subscribers ⇒ zero cost + *by construction*. Scalar time-series is a first-class preview type. Execute-up-to-node + included and honours D7 phases. +- **D12 — Replay-first.** The recorded probe log *is* the serialization of the inspection + contract; live attach is the same format with a tail; the log doubles as the golden record. + Per-probe decimation policy; large blobs stored out-of-line by reference. +- **D13 — Determinism.** Event-level equality, not bitwise, given a deterministic source. + Replay *forces* drop-policy off. Events ordered and stamped by `frameIndex`/`streamTime`; + `wallClock` provably source-derived or excluded. Determinism is a declared execution mode. + *See A4 and A6.* +- **D14 — Session generation.** Watch mode is a runner-level work source, not a graph node. + The runner owns durable cross-session state (processed keys, append handles, spools), which + keeps "node state never crosses sessions" true as written. Sources declare + `continuous | per-item` session semantics. +- **D15 — Source capabilities as data.** `seekable`, `triggerable | free-running`, session + semantics, native rate/format/resolution, drop policy. Drops always counted and reported — + never a silent `DropOldest`. *See A4 — policy is mode-governed, not source-chosen.* +- **D16 — Config layering.** `defaults < YAML < env < per-run` is authoring-time. Export fully + resolves into a frozen self-contained artifact with provenance; the **resolved artifact** is + the unit of reproducibility. Overrides hitting a `resets-node-state` param are flagged as + state resets. Environment-resolved values (e.g. codec fallback `avc1→mp4v`) are reported, + never silent, and captured in the run record. +- **D17 — Granularity.** Node boundaries are tuning/reuse boundaries; probe boundaries are + visibility boundaries. Never split a node to see inside it — add a probe. A node may be + internally coarse when it declares batching. +- **D18 — Layering.** engine (no UI dependency) → runtime driver → shells. Headless is the + same driver with nobody subscribed. Hot reload: params by node id, debounced; structural + edits rebuild; D9 change classes govern state. +- **D19 — v2 reservations v1 must not preclude.** Map/subgraph keyed by item identity; + identity as a first-class typed value with declared provenance; subgraph-diff rebuilds; + parallel branches. *See A7 — add static instancing, ranked first.* +- **D20 — Acceptance tests.** Reference cases A, B, C above. + +--- + +## Open amendments (raised by Vision graph, pending ratification) + +- **A1 — `itemKey` is populated in v1, not null.** D3's "always null in v1" contradicts two + live requirements. Useful-bin's `MetalScorer` emits one verdict *per placement*, and "see + the results of each node" for that node means a placement-addressed list. Vision builder's + `ArrayModelMatchingOperation` ships `Dictionary IsGood` — per-cell verdicts, keyed + by grid index, in production today — which the UI cannot surface, showing only an `All()` + rollup so the operator cannot see which cell failed. Fixed-N addressing is needed *without* + a map operator. Only *dynamic-map* item keys are absent from v1. Left as "always null" the + field ships dead, nothing populates it, no UI renders it, and reference case A's core + requirement fails. +- **A2 — Split declared state into `derived` and `resource`.** D4's single `state` bucket is + unimplementable for 14 of Vision builder's 53 operations (`InferenceSession`, + `YoloV8Predictor`, NetMQ sockets, a spawned Docker container). `derived` state (MOG2 model, + LK previous pyramid, latch counters, metal accumulator) is learned from the frame sequence, + cannot be reconstructed from params, and **must serialize**. `resource` state is + reconstructible from params and **must not serialize** — it is rebuilt by `prewarm`. A node + snapshot is `derived` only; restoring means `prewarm` then load `derived`. This is also what + makes snapshots cheap (serializing a background model, not an ONNX session). +- **A3 — `prewarm` must be state-neutral by construction.** Vision builder's `WarmUp()` + executes the whole recipe against an empty source; that is safe *only because he has no + state*. Run the same design on a graph with MOG2 and the background model trains on a black + synthetic frame before `prime` starts — priming an already-poisoned model. Fix falls out of + A2: the node contract separates resource acquisition from tick, `prewarm` invokes only the + former, and cannot reach `derived` state because it is not yet allocated. ONNX shape + specialization still requires a real inference at the declared format/resolution, so + `prewarm` may run inference without invoking the state-update path. +- **A4 — Drop policy is a property of the execution mode, not of the source.** D15 assigns it + per-source. Evidence against: *every* Vision builder source hardcodes + `BoundedChannelOptions(1) { FullMode = DropOldest }`, **including the file-based emulation + source**, which additionally drains backlog before reading. His replay path silently drops + frames off disk, so re-running the same folder is not guaranteed to see the same frames. + Four authors made the same local choice four times because no mode existed to distinguish + batch from live. Resolution: *capability* is per-source (a live camera cannot block + indefinitely; a file can), *policy* is chosen by mode and validated against capability — + so `deterministic` mode is rejected at validation when a source declares it cannot block. + "Determinism only holds for seekable sources" then falls out as a validation rule rather + than a caveat someone forgets. + + | | `deterministic` (batch) | `realtime` | + |---|---|---| + | source | seekable | live | + | `prime` | silent replay pass | eat first N seconds | + | queue full | **block** | **drop + count** | + | golden comparison | guaranteed | not claimed | + +- **A5 — Authoring-time validation, and the format must admit instance sets.** Two additions + to D9. (i) Label uniqueness *and* identifier-safety validated at authoring time: the shipped + `gold_b97.hrcp` contains a node labelled `Red wings` — with a space, therefore not a valid + Python identifier, so any expression referencing it throws at evaluation. It sits at a named + client, undetected, because nothing happens to reference it. A sibling also kept its default + label, indistinguishable among thirteen near-identical nodes. (ii) The artifact format must + be able to represent a node *definition plus configured instance set*, not only flat + independent node records — otherwise A7 requires a format migration. +- **A6 — Probe-log fields are marked `deterministic` or `observational`.** Per-node + `durationMs` is mandatory (D5) and is the instrument clients use to budget cycle time, but + it is never reproducible — same class as `wallClock`. Golden comparison runs over + deterministic fields only. Without a declared non-deterministic field set, R13 fails on + timing noise and someone disables the test. +- **A7 — Add "static instancing over a declared set" to D19, ranked first.** The v2 fan-out + question was framed as subgraph-instancing vs batch ports. Production demanded neither. + `gold_b97.hrcp` is thirteen hand-duplicated `YoloPickDetected` nodes — one per chocolate + class — each reading the same mask with its own label, own size thresholds, own verdict. + N is fixed and known at authoring time, so it is not dynamic map; they need thirteen + *separately addressable* verdicts, so it is not batch ports. The shape is one node + definition, N configured instances, per-instance params, per-instance verdict, expanded at + authoring time. It is static (validates like a normal graph), needs no scheduler work, + reuses the `itemKey` addressing A1 makes live, and fixes the thirteen-copies maintenance + cost. It is the only one of the three shapes a real recipe has actually demanded. + +## Consolidated lifecycle + +``` +build → prewarm(resource only, synthetic frame at declared format+resolution) + → sessionBegin → prime(derived convergence, sinks suppressed) + → tick* → flush(event-windowed nodes emit) → sessionEnd → dispose +``` + +`stop` (graceful: finishes tick, runs `flush`, closes sinks, drains spools) and `abort` +(discard) are distinct signals; any node may raise either and all nodes must observe both. +`flush` runs on `stop`, not on `abort`. Session generation sits **outside** this loop, in the +runner. + +`execute-up-to-node` is *prime-then-prefix*, not pure prefix evaluation, and must be +**idempotent** — running it twice must not double-train a background model. Requires snapshot +restore (A2) or a fresh session per invocation; it must never mutate the live session's +`derived` state. + +## Identity provenance + +Identity is a first-class typed value carrying its provenance. The axis is not uniqueness but +the **strength of the continuity claim**: + +| provenance | continuity | +|---|---| +| grid index | definitional — cannot be wrong | +| detection id | exact within a frame, asserts nothing across frames | +| placement / segment id | a detector's assertion — `forcedCut` proves it can be wrong | +| track id | inferred — swaps, merges, splits, occlusion re-entry | + +Discontinuity events (appear, disappear, merge, split, forced-cut) must be **observable by +the node accumulating state**, so it can invalidate rather than silently carry state across. +State accumulated under a weak continuity claim without this is the same failure class as a +`NoOrigin` default at (0,0): plausible output, no diagnostic.