Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Vision graph
|
8bac01f2a2 |
@@ -0,0 +1,207 @@
|
||||
# Vision Graph — Decisions Register
|
||||
|
||||
Implementation-side canonical mirror of the decisions ratified in the `Vision graph group`
|
||||
design thread. Useful-bin owns the spec prose; this file is the contract the implementation
|
||||
is built against, maintained by Vision graph (implementer).
|
||||
|
||||
**Source of truth:** the group thread. Sections are pasted whole into the group; this file
|
||||
mirrors them so the implementation has one authoritative copy under version control.
|
||||
|
||||
## Reference cases
|
||||
|
||||
| id | case | what it tests |
|
||||
|----|------|---------------|
|
||||
| A | Useful-bin's FrontierPlacement pipeline | **fidelity** — golden event/CSV parity against the existing pipeline |
|
||||
| B | Vision builder's `blob-find → ROI-repivot → edge-intersect` (N=1) | **geometry** — expressible with typed ports instead of a global reference system |
|
||||
| C | Vision builder's `LeerformRecognitionControl` | **expressiveness** — could the client that abandoned the engine have stayed in it? |
|
||||
|
||||
Case C matters most as a guard: an inexpressive model is not extended, it is *escaped*, and
|
||||
escaping costs the editor, per-node inspection, hot reload and operator tuning in one step.
|
||||
Two production clients are running that way today (`LindtLeerformPlugin`, `CandyboxPlugin`
|
||||
— zero references to `WorkflowList` or `BaseOperation`, returning `errorNames: []`).
|
||||
|
||||
---
|
||||
|
||||
## Ratified decisions
|
||||
|
||||
- **D1 — Execution.** Push-based frame-lockstep synchronous tick, ordered fan-out. The
|
||||
normative requirement is *in-order, gap-free delivery to stateful nodes within a session*;
|
||||
lockstep is one implementation of that guarantee, not the guarantee itself. The tick never
|
||||
blocks on I/O.
|
||||
- **D2 — Ports.** `frame` ports (0-or-1 typed value per tick: image, mask, scalar, point-set,
|
||||
detection, geometry) and `event` ports (0..N stamped records per tick). No third category
|
||||
for geometry — typed `frame` ports subsume it. Overlay must never be the sole carrier of
|
||||
data.
|
||||
- **D3 — Fan-out.** No dynamic map/subgraph operator in v1. Probes and verdicts are addressed
|
||||
as `(nodeId, itemKey?)`. *See A1 — the register's "itemKey always null in v1" is contested.*
|
||||
- **D4 — Node contract.** `(params, inputs, state) → (outputs, state', verdict, overlay)`,
|
||||
pure apart from declared state. Isolation-testability is an explicit spec goal.
|
||||
*See A2 — `state` must be split.*
|
||||
- **D5 — Verdict.** `execution: ran | skipped{reason: gated | disabled | upstream-unavailable}
|
||||
| error` always present; `judgment: pass | reject` optional, inspection nodes only; plus a
|
||||
human-readable status sentence and mandatory per-node `durationMs` every tick. `skipped`
|
||||
never aggregates as `pass`.
|
||||
- **D6 — Buffers.** Edge values immutable, tick-scoped, engine-owned. "Copy what you keep" is
|
||||
a stated node obligation. `mutates-input` ⇒ private copy when the producing port has >1
|
||||
consumer; pure readers share (read-only broadcast to many consumers is normal and free).
|
||||
Bounded history on an input port is a *declared* retention, bounded in both frames and
|
||||
bytes, maintained once per edge and shared by all declaring consumers.
|
||||
- **D7 — Two warm-up phases, never merged.** `prewarm` (per-process resource readiness) and
|
||||
`prime` (per-session stateful convergence over real frames, all sinks suppressed).
|
||||
*See A3 — `prewarm` must be state-neutral by construction.*
|
||||
- **D8 — Absent values.** Declared **per input port**: `absent→skip | last-value (with
|
||||
staleness bound) | error`. Gating (dynamic, "no value this tick" is normal) is a *different
|
||||
mechanism* from `Enabled=false` (static authoring config ⇒ pre-run validation error unless
|
||||
the node declares pass-through). Never merge them. The declaration must be visible in the
|
||||
editor next to the port — the person who gets this wrong is an operator disabling a node at
|
||||
3am, not the graph's author.
|
||||
- **D9 — Manifests.** Param kinds: scalar, enum, bool, string, expression, roi, reference,
|
||||
file, lut, curve. Plus constraints, defaults, output-only flag, and per-param change class
|
||||
`live | resets-node-state | requires-rebuild`. Spatial params carry explicit coordinate
|
||||
space (`image-absolute | relative-to-connected-port`) and normalization (`normalized |
|
||||
pixels`). Persistence is generic from the manifest — nodes never hand-write it. One
|
||||
versioned format. Stable Guid ≠ label. *See A5 — label validation and instance sets.*
|
||||
- **D10 — Expressions.** Kept for scalar arithmetic. Identifiers resolve **only** to the
|
||||
node's own connected ports plus graph config, by id, scoped to one execution. Never to
|
||||
another node by display label.
|
||||
- **D11 — Introspection record.** `{id, itemKey?, execution, judgment?, status, durationMs,
|
||||
outputs, overlayElements, preview?}`. Overlays are renderer-agnostic draw commands.
|
||||
Previews are opt-in subscriptions, throttled, copied at emit — zero subscribers ⇒ zero cost
|
||||
*by construction*. Scalar time-series is a first-class preview type. Execute-up-to-node
|
||||
included and honours D7 phases.
|
||||
- **D12 — Replay-first.** The recorded probe log *is* the serialization of the inspection
|
||||
contract; live attach is the same format with a tail; the log doubles as the golden record.
|
||||
Per-probe decimation policy; large blobs stored out-of-line by reference.
|
||||
- **D13 — Determinism.** Event-level equality, not bitwise, given a deterministic source.
|
||||
Replay *forces* drop-policy off. Events ordered and stamped by `frameIndex`/`streamTime`;
|
||||
`wallClock` provably source-derived or excluded. Determinism is a declared execution mode.
|
||||
*See A4 and A6.*
|
||||
- **D14 — Session generation.** Watch mode is a runner-level work source, not a graph node.
|
||||
The runner owns durable cross-session state (processed keys, append handles, spools), which
|
||||
keeps "node state never crosses sessions" true as written. Sources declare
|
||||
`continuous | per-item` session semantics.
|
||||
- **D15 — Source capabilities as data.** `seekable`, `triggerable | free-running`, session
|
||||
semantics, native rate/format/resolution, drop policy. Drops always counted and reported —
|
||||
never a silent `DropOldest`. *See A4 — policy is mode-governed, not source-chosen.*
|
||||
- **D16 — Config layering.** `defaults < YAML < env < per-run` is authoring-time. Export fully
|
||||
resolves into a frozen self-contained artifact with provenance; the **resolved artifact** is
|
||||
the unit of reproducibility. Overrides hitting a `resets-node-state` param are flagged as
|
||||
state resets. Environment-resolved values (e.g. codec fallback `avc1→mp4v`) are reported,
|
||||
never silent, and captured in the run record.
|
||||
- **D17 — Granularity.** Node boundaries are tuning/reuse boundaries; probe boundaries are
|
||||
visibility boundaries. Never split a node to see inside it — add a probe. A node may be
|
||||
internally coarse when it declares batching.
|
||||
- **D18 — Layering.** engine (no UI dependency) → runtime driver → shells. Headless is the
|
||||
same driver with nobody subscribed. Hot reload: params by node id, debounced; structural
|
||||
edits rebuild; D9 change classes govern state.
|
||||
- **D19 — v2 reservations v1 must not preclude.** Map/subgraph keyed by item identity;
|
||||
identity as a first-class typed value with declared provenance; subgraph-diff rebuilds;
|
||||
parallel branches. *See A7 — add static instancing, ranked first.*
|
||||
- **D20 — Acceptance tests.** Reference cases A, B, C above.
|
||||
|
||||
---
|
||||
|
||||
## Open amendments (raised by Vision graph, pending ratification)
|
||||
|
||||
- **A1 — `itemKey` is populated in v1, not null.** D3's "always null in v1" contradicts two
|
||||
live requirements. Useful-bin's `MetalScorer` emits one verdict *per placement*, and "see
|
||||
the results of each node" for that node means a placement-addressed list. Vision builder's
|
||||
`ArrayModelMatchingOperation` ships `Dictionary<int,bool> IsGood` — per-cell verdicts, keyed
|
||||
by grid index, in production today — which the UI cannot surface, showing only an `All()`
|
||||
rollup so the operator cannot see which cell failed. Fixed-N addressing is needed *without*
|
||||
a map operator. Only *dynamic-map* item keys are absent from v1. Left as "always null" the
|
||||
field ships dead, nothing populates it, no UI renders it, and reference case A's core
|
||||
requirement fails.
|
||||
- **A2 — Split declared state into `derived` and `resource`.** D4's single `state` bucket is
|
||||
unimplementable for 14 of Vision builder's 53 operations (`InferenceSession`,
|
||||
`YoloV8Predictor`, NetMQ sockets, a spawned Docker container). `derived` state (MOG2 model,
|
||||
LK previous pyramid, latch counters, metal accumulator) is learned from the frame sequence,
|
||||
cannot be reconstructed from params, and **must serialize**. `resource` state is
|
||||
reconstructible from params and **must not serialize** — it is rebuilt by `prewarm`. A node
|
||||
snapshot is `derived` only; restoring means `prewarm` then load `derived`. This is also what
|
||||
makes snapshots cheap (serializing a background model, not an ONNX session).
|
||||
- **A3 — `prewarm` must be state-neutral by construction.** Vision builder's `WarmUp()`
|
||||
executes the whole recipe against an empty source; that is safe *only because he has no
|
||||
state*. Run the same design on a graph with MOG2 and the background model trains on a black
|
||||
synthetic frame before `prime` starts — priming an already-poisoned model. Fix falls out of
|
||||
A2: the node contract separates resource acquisition from tick, `prewarm` invokes only the
|
||||
former, and cannot reach `derived` state because it is not yet allocated. ONNX shape
|
||||
specialization still requires a real inference at the declared format/resolution, so
|
||||
`prewarm` may run inference without invoking the state-update path.
|
||||
- **A4 — Drop policy is a property of the execution mode, not of the source.** D15 assigns it
|
||||
per-source. Evidence against: *every* Vision builder source hardcodes
|
||||
`BoundedChannelOptions(1) { FullMode = DropOldest }`, **including the file-based emulation
|
||||
source**, which additionally drains backlog before reading. His replay path silently drops
|
||||
frames off disk, so re-running the same folder is not guaranteed to see the same frames.
|
||||
Four authors made the same local choice four times because no mode existed to distinguish
|
||||
batch from live. Resolution: *capability* is per-source (a live camera cannot block
|
||||
indefinitely; a file can), *policy* is chosen by mode and validated against capability —
|
||||
so `deterministic` mode is rejected at validation when a source declares it cannot block.
|
||||
"Determinism only holds for seekable sources" then falls out as a validation rule rather
|
||||
than a caveat someone forgets.
|
||||
|
||||
| | `deterministic` (batch) | `realtime` |
|
||||
|---|---|---|
|
||||
| source | seekable | live |
|
||||
| `prime` | silent replay pass | eat first N seconds |
|
||||
| queue full | **block** | **drop + count** |
|
||||
| golden comparison | guaranteed | not claimed |
|
||||
|
||||
- **A5 — Authoring-time validation, and the format must admit instance sets.** Two additions
|
||||
to D9. (i) Label uniqueness *and* identifier-safety validated at authoring time: the shipped
|
||||
`gold_b97.hrcp` contains a node labelled `Red wings` — with a space, therefore not a valid
|
||||
Python identifier, so any expression referencing it throws at evaluation. It sits at a named
|
||||
client, undetected, because nothing happens to reference it. A sibling also kept its default
|
||||
label, indistinguishable among thirteen near-identical nodes. (ii) The artifact format must
|
||||
be able to represent a node *definition plus configured instance set*, not only flat
|
||||
independent node records — otherwise A7 requires a format migration.
|
||||
- **A6 — Probe-log fields are marked `deterministic` or `observational`.** Per-node
|
||||
`durationMs` is mandatory (D5) and is the instrument clients use to budget cycle time, but
|
||||
it is never reproducible — same class as `wallClock`. Golden comparison runs over
|
||||
deterministic fields only. Without a declared non-deterministic field set, R13 fails on
|
||||
timing noise and someone disables the test.
|
||||
- **A7 — Add "static instancing over a declared set" to D19, ranked first.** The v2 fan-out
|
||||
question was framed as subgraph-instancing vs batch ports. Production demanded neither.
|
||||
`gold_b97.hrcp` is thirteen hand-duplicated `YoloPickDetected` nodes — one per chocolate
|
||||
class — each reading the same mask with its own label, own size thresholds, own verdict.
|
||||
N is fixed and known at authoring time, so it is not dynamic map; they need thirteen
|
||||
*separately addressable* verdicts, so it is not batch ports. The shape is one node
|
||||
definition, N configured instances, per-instance params, per-instance verdict, expanded at
|
||||
authoring time. It is static (validates like a normal graph), needs no scheduler work,
|
||||
reuses the `itemKey` addressing A1 makes live, and fixes the thirteen-copies maintenance
|
||||
cost. It is the only one of the three shapes a real recipe has actually demanded.
|
||||
|
||||
## Consolidated lifecycle
|
||||
|
||||
```
|
||||
build → prewarm(resource only, synthetic frame at declared format+resolution)
|
||||
→ sessionBegin → prime(derived convergence, sinks suppressed)
|
||||
→ tick* → flush(event-windowed nodes emit) → sessionEnd → dispose
|
||||
```
|
||||
|
||||
`stop` (graceful: finishes tick, runs `flush`, closes sinks, drains spools) and `abort`
|
||||
(discard) are distinct signals; any node may raise either and all nodes must observe both.
|
||||
`flush` runs on `stop`, not on `abort`. Session generation sits **outside** this loop, in the
|
||||
runner.
|
||||
|
||||
`execute-up-to-node` is *prime-then-prefix*, not pure prefix evaluation, and must be
|
||||
**idempotent** — running it twice must not double-train a background model. Requires snapshot
|
||||
restore (A2) or a fresh session per invocation; it must never mutate the live session's
|
||||
`derived` state.
|
||||
|
||||
## Identity provenance
|
||||
|
||||
Identity is a first-class typed value carrying its provenance. The axis is not uniqueness but
|
||||
the **strength of the continuity claim**:
|
||||
|
||||
| provenance | continuity |
|
||||
|---|---|
|
||||
| grid index | definitional — cannot be wrong |
|
||||
| detection id | exact within a frame, asserts nothing across frames |
|
||||
| placement / segment id | a detector's assertion — `forcedCut` proves it can be wrong |
|
||||
| track id | inferred — swaps, merges, splits, occlusion re-entry |
|
||||
|
||||
Discontinuity events (appear, disappear, merge, split, forced-cut) must be **observable by
|
||||
the node accumulating state**, so it can invalidate rather than silently carry state across.
|
||||
State accumulated under a weak continuity claim without this is the same failure class as a
|
||||
`NoOrigin` default at (0,0): plausible output, no diagnostic.
|
||||
Reference in New Issue
Block a user