Commit Graph

12 Commits

Author SHA1 Message Date
tes a42038b825 Diagnostics: forward upstream events + errors to device as 'debug' envelopes
Coolify container logs aren't accessible from the dev sandbox, so the only
log surface we can read is the Pi journal. The relay now forwards every
upstream event type to the device as {type:'debug', message:'relay evt=<T>'}
and every mid-session upstream error event as {type:'error',code:'upstream_<C>'}.

The client's Session.dispatch handles 'debug' as an INFO log line. Updates
the wake-flow integration test's receive-loop to skip over the new debug
envelopes via a ReceiveUntil helper.
2026-06-12 06:59:14 +00:00
tes 8bb0ce51b1 Plan 3 fix: GA turn_detection auto-create_response + reset state on WS reconnect
GA Realtime API requires create_response:true and interrupt_response:true
inside turn_detection — without them the model never auto-generates a
reply after VAD detects speech_stopped, so the relay idles out at 30s
with no assistant audio. Add both.

Client side: when the /device WS drops (Coolify rolling deploy, etc.),
Session.run_forever reconnects but the StateMachine kept whatever state
it was in — usually LISTENING, which permanently disables the wakeword.
Add StateMachine.force_idle() and a Callbacks.on_disconnected() hook;
main.py wires it to force the state back to IDLE on every disconnect.
Also harden Session's fire-and-forget send_* helpers to swallow
ConnectionClosed instead of bubbling up as unhandled task exceptions.
2026-06-12 06:50:00 +00:00
tes 7294a81a9a Realtime: migrate to OpenAI GA shape (beta API was retired)
Upstream returned 'beta_api_shape_disabled' on every wake. Changes:
- Drop OpenAI-Beta: realtime=v1 header from the upgrade.
- session.update payload nests audio under session.audio.{input,output}
  and adds session.type='realtime'; audio.{input,output}.format is now
  an object {type:audio/pcm, rate:24000} instead of 'pcm16' string.
- Default model bumped to gpt-realtime (the GA name); the relay also
  promotes any stored 'gpt-4o-realtime-preview*' to gpt-realtime so
  pre-Plan-3 DeviceConfig rows keep working without a manual DB edit.
2026-06-12 06:42:21 +00:00
tes d01b78b225 Relay: log upstream events + forward error envelopes to device for diagnostics
Plan 3 smoke surfaced 'session_ended reason=error' on every wake with no
context. The relay now logs every upstream event type at INFO and forwards
any upstream 'error' event to the device as 'error code=upstream_<code>'
so it appears in the Pi's journal alongside the wake/session_ended pair.

Also extends the client's wakeword.predict with per-frame RMS + score
diagnostics and the bridge loop with a 5 s heartbeat for triage.
2026-06-12 06:28:58 +00:00
tes 78373b0912 Add OpenAIRealtimeUpstream + RealtimeSessionFactory + DI registrations 2026-06-11 21:48:21 +00:00
tes 8f53f9daa9 RealtimeSession: idle-timeout watchdog (resets on speech_started) 2026-06-11 21:46:16 +00:00
tes 7dada5fdb9 RealtimeSession: execute server-side tools at response.done 2026-06-11 21:38:31 +00:00
tes b078f9aa78 RealtimeSession: persist user/assistant transcripts; emit assistant_done 2026-06-11 21:35:40 +00:00
tes 24b8acdf60 RealtimeSession: uplink pump + downlink audio.delta forwarding 2026-06-11 21:33:37 +00:00
tes a202725c39 Add RealtimeSession lifecycle (open + session.update + session_started/ended) 2026-06-11 21:30:40 +00:00
tes 35a6e8febb Add IRealtimeUpstream + RealtimeEvents helpers for upstream JSON 2026-06-11 21:25:46 +00:00
tes d3e101eb29 Add OpenAIKeyProvider + RealtimeSettings record 2026-06-11 21:23:54 +00:00