The Pi journal showed 170 000 'upstream silent NxN' messages within one
second — i.e. ReceiveJsonAsync was returning null in zero time, so the
upstream WS to OpenAI was closed. The relay was spinning until idle
watchdog (30 s) fired. Add IRealtimeUpstream.IsClosed + CloseReason,
populated by OpenAIRealtimeUpstream from the close status / WS exception,
and have PumpAsync stop on close and forward
error code=upstream_closed message=close status=<X> desc=<Y>
to the device. Closes the spin AND gives us the OpenAI close reason.
The previous run showed session_started, then total silence from the
relay for the entire conversation — no events forwarded, no errors,
nothing. Two changes to disambiguate:
1. Drop the explicit audio.input.format / audio.output.format objects.
24 kHz PCM16 mono is the documented GA default for both directions
and the only thing OpenAI ever advertised; an unrecognised value
inside the format object is the most likely reason OpenAI silently
ignores incoming audio.
2. Forward 'upstream silent NxN250ms' debug envelopes every ~2 s when
ReceiveJsonAsync keeps returning null, and add a 'server_version'
field to hello_ack so the Pi journal proves which build is running.
Coolify container logs aren't accessible from the dev sandbox, so the only
log surface we can read is the Pi journal. The relay now forwards every
upstream event type to the device as {type:'debug', message:'relay evt=<T>'}
and every mid-session upstream error event as {type:'error',code:'upstream_<C>'}.
The client's Session.dispatch handles 'debug' as an INFO log line. Updates
the wake-flow integration test's receive-loop to skip over the new debug
envelopes via a ReceiveUntil helper.
GA Realtime API requires create_response:true and interrupt_response:true
inside turn_detection — without them the model never auto-generates a
reply after VAD detects speech_stopped, so the relay idles out at 30s
with no assistant audio. Add both.
Client side: when the /device WS drops (Coolify rolling deploy, etc.),
Session.run_forever reconnects but the StateMachine kept whatever state
it was in — usually LISTENING, which permanently disables the wakeword.
Add StateMachine.force_idle() and a Callbacks.on_disconnected() hook;
main.py wires it to force the state back to IDLE on every disconnect.
Also harden Session's fire-and-forget send_* helpers to swallow
ConnectionClosed instead of bubbling up as unhandled task exceptions.
Upstream returned 'beta_api_shape_disabled' on every wake. Changes:
- Drop OpenAI-Beta: realtime=v1 header from the upgrade.
- session.update payload nests audio under session.audio.{input,output}
and adds session.type='realtime'; audio.{input,output}.format is now
an object {type:audio/pcm, rate:24000} instead of 'pcm16' string.
- Default model bumped to gpt-realtime (the GA name); the relay also
promotes any stored 'gpt-4o-realtime-preview*' to gpt-realtime so
pre-Plan-3 DeviceConfig rows keep working without a manual DB edit.
Plan 3 smoke surfaced 'session_ended reason=error' on every wake with no
context. The relay now logs every upstream event type at INFO and forwards
any upstream 'error' event to the device as 'error code=upstream_<code>'
so it appears in the Pi's journal alongside the wake/session_ended pair.
Also extends the client's wakeword.predict with per-frame RMS + score
diagnostics and the bridge loop with a 5 s heartbeat for triage.
tes