tes/Assistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
67 Commits 2 Branches 0 Tags
8bb0ce51b17dc0e206ae1358136d397f03d06ac0
Commit Graph

67 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
tes 8bb0ce51b1 Plan 3 fix: GA turn_detection auto-create_response + reset state on WS reconnect 
GA Realtime API requires create_response:true and interrupt_response:true
inside turn_detection — without them the model never auto-generates a
reply after VAD detects speech_stopped, so the relay idles out at 30s
with no assistant audio. Add both.

Client side: when the /device WS drops (Coolify rolling deploy, etc.),
Session.run_forever reconnects but the StateMachine kept whatever state
it was in — usually LISTENING, which permanently disables the wakeword.
Add StateMachine.force_idle() and a Callbacks.on_disconnected() hook;
main.py wires it to force the state back to IDLE on every disconnect.
Also harden Session's fire-and-forget send_* helpers to swallow
ConnectionClosed instead of bubbling up as unhandled task exceptions.
 2026-06-12 06:50:00 +00:00
tes 7294a81a9a Realtime: migrate to OpenAI GA shape (beta API was retired) 
Upstream returned 'beta_api_shape_disabled' on every wake. Changes:
- Drop OpenAI-Beta: realtime=v1 header from the upgrade.
- session.update payload nests audio under session.audio.{input,output}
  and adds session.type='realtime'; audio.{input,output}.format is now
  an object {type:audio/pcm, rate:24000} instead of 'pcm16' string.
- Default model bumped to gpt-realtime (the GA name); the relay also
  promotes any stored 'gpt-4o-realtime-preview*' to gpt-realtime so
  pre-Plan-3 DeviceConfig rows keep working without a manual DB edit.
 2026-06-12 06:42:21 +00:00
tes d01b78b225 Relay: log upstream events + forward error envelopes to device for diagnostics 
Plan 3 smoke surfaced 'session_ended reason=error' on every wake with no
context. The relay now logs every upstream event type at INFO and forwards
any upstream 'error' event to the device as 'error code=upstream_<code>'
so it appears in the Pi's journal alongside the wake/session_ended pair.

Also extends the client's wakeword.predict with per-frame RMS + score
diagnostics and the bridge loop with a 5 s heartbeat for triage.
 2026-06-12 06:28:58 +00:00
tes 82ea164695 README: document Plan 3 Pi client surface 2026-06-11 22:56:34 +00:00
tes 8662dbe8a7 Plan 3: main.py composition root (audio + state + wakeword + session + playback wired) 2026-06-11 22:52:01 +00:00
tes b1eeaa8af3 Plan 3: backend WS client + dispatch + tool round-trip 2026-06-11 22:51:23 +00:00
tes fa245d87a5 Plan 3: playback worker + wake/sleep wav + software volume gain 2026-06-11 22:50:13 +00:00
tes d0bd141082 Plan 3: openwakeword wrapper with threshold + cooldown 2026-06-11 22:49:40 +00:00
tes 55257b8867 Plan 3: USB device discovery + persistent audio streams 2026-06-11 22:49:20 +00:00
tes d155af4b70 Plan 3: pair CLI (POST /api/pair, write config.json) 2026-06-11 22:48:46 +00:00
tes 781ef9377a Plan 3: 24 kHz->16 kHz downsample helper for the wakeword feed 2026-06-11 22:48:16 +00:00
tes 4d83f2d9fd Plan 3: load/save ~/assistant/state/config.json with mode 0600 2026-06-11 22:47:50 +00:00
tes d75129c4f9 Plan 3: state machine (IDLE -> WAKE_PENDING -> LISTENING <-> ASSISTANT_SPEAKING -> IDLE_PENDING -> IDLE) 2026-06-11 22:47:23 +00:00
tes 908e3f4907 Plan 3: add client.log structured logger 2026-06-11 22:46:44 +00:00
tes 79d6e923a6 Plan 3: scaffold Python test harness and stub client package 2026-06-11 22:46:25 +00:00
tes 8380e3149e README: document Plan 2 device-hub surface area 2026-06-11 22:02:35 +00:00
tes 7ba9f0cf5e Plan 2: device hub + Realtime relay + tools registry 2026-06-11 22:01:30 +00:00
tes ba709d15d5 DeviceHub: wake → session lifecycle + uplink relay + tool_result routing 2026-06-11 22:00:33 +00:00
tes 9023509b01 DeviceHub: /device WS with bearer auth + hello/hello_ack + ping/pong 2026-06-11 21:55:33 +00:00
tes 4dc48bd9fd Add ActiveDevice + DeviceRegistry + HubEnvelopes + IDeviceClient/Channel implementations 2026-06-11 21:51:25 +00:00
tes 78373b0912 Add OpenAIRealtimeUpstream + RealtimeSessionFactory + DI registrations 2026-06-11 21:48:21 +00:00
tes 8f53f9daa9 RealtimeSession: idle-timeout watchdog (resets on speech_started) 2026-06-11 21:46:16 +00:00
tes a45b25038a Test end_session sequencing closes with reason=tool and skips response.create 2026-06-11 21:42:47 +00:00
tes 7dada5fdb9 RealtimeSession: execute server-side tools at response.done 2026-06-11 21:38:31 +00:00
tes b078f9aa78 RealtimeSession: persist user/assistant transcripts; emit assistant_done 2026-06-11 21:35:40 +00:00
tes 24b8acdf60 RealtimeSession: uplink pump + downlink audio.delta forwarding 2026-06-11 21:33:37 +00:00
tes a202725c39 Add RealtimeSession lifecycle (open + session.update + session_started/ended) 2026-06-11 21:30:40 +00:00
tes 8f197d1203 Add ScriptedRealtimeUpstream + FakeDeviceChannel test doubles 2026-06-11 21:27:50 +00:00
tes 35a6e8febb Add IRealtimeUpstream + RealtimeEvents helpers for upstream JSON 2026-06-11 21:25:46 +00:00
tes d3e101eb29 Add OpenAIKeyProvider + RealtimeSettings record 2026-06-11 21:23:54 +00:00
tes 212ce06f07 Add SetVolumeTool (validates 0..100, delegates to Pi via IDeviceChannel) 2026-06-11 21:22:45 +00:00
tes ebbe23c639 Add EndSessionTool (no params, returns ok; relay closes on this tool name) 2026-06-11 21:21:13 +00:00
tes 9325d21f9e Add ITool + ToolRegistry + GetCurrentTimeTool 2026-06-11 21:19:23 +00:00
tes 13d8773704 Add ConversationLog (start/append/end with monotonic per-convo index) 2026-06-11 21:13:50 +00:00
tes 99cfe60a2c Add Conversation + Turn entities and migration 2026-06-11 21:09:52 +00:00
tes 58b8a446b1 Fix Dockerfile: don't copy test csproj (excluded by .dockerignore) 2026-06-11 20:31:52 +00:00
tes a83e9f93de Add UseForwardedHeaders so /install.sh + backend_ws use https/wss behind Traefik 2026-06-11 20:17:46 +00:00
tes 68738e486d Add README with repo layout, local dev, smoke test, deploy commands 2026-06-11 20:11:39 +00:00
tes 2701b8a483 Add docker-compose.yml for Coolify deploy + gitignore deploy.json 2026-06-11 20:05:58 +00:00
tes 645a7e3722 Persist data-protection keys to /keys when present (Coolify volume) 2026-06-11 19:05:41 +00:00
tes 88339c51eb Add multi-stage Dockerfile (backend + client tarball) 2026-06-11 19:04:45 +00:00
tes a7e5e13ac8 Add Python client placeholders for bundle build (full impl in Plan 3) 2026-06-11 19:04:18 +00:00
tes 751030dcef Add InstallScriptBuilder + GET /install.sh + GET /client.tar.gz 2026-06-11 19:03:33 +00:00
tes e5ae228738 Add /api/devices CRUD (list, get, rename, revoke) with owner scoping 2026-06-11 19:00:42 +00:00
tes b5be4c6914 Add /api/pair-code (auth) + /api/pair (public) endpoints 2026-06-11 18:56:41 +00:00
tes db5c4e59a4 Add PairingService with safe alphabet and 10-min TTL 2026-06-11 18:53:32 +00:00
tes 720583de0a Add PairingCode entity + migration 2026-06-11 18:51:18 +00:00
tes b862bf6f0f Add DeviceTokenService (random token, SHA-256 hash, fixed-time verify) 2026-06-11 18:48:39 +00:00
tes 3e9c245bb3 Fix Task 6: Device FK to AspNetUsers + SystemSettings singleton Id pin 2026-06-11 18:46:42 +00:00
tes 68e7e81312 Add Device, DeviceConfig, SystemSettings entities and seed defaults 2026-06-11 18:41:27 +00:00