Compare commits

..

62 Commits

Author SHA1 Message Date
tes 1a36c8096d Diagnostics: surface upstream WS close reason + break the null-spin loop
The Pi journal showed 170 000 'upstream silent NxN' messages within one
second — i.e. ReceiveJsonAsync was returning null in zero time, so the
upstream WS to OpenAI was closed. The relay was spinning until idle
watchdog (30 s) fired. Add IRealtimeUpstream.IsClosed + CloseReason,
populated by OpenAIRealtimeUpstream from the close status / WS exception,
and have PumpAsync stop on close and forward
  error code=upstream_closed message=close status=<X> desc=<Y>
to the device. Closes the spin AND gives us the OpenAI close reason.
2026-06-12 07:17:04 +00:00
tes 12be767325 Trim GA session.update + add deploy marker + surface upstream silence
The previous run showed session_started, then total silence from the
relay for the entire conversation — no events forwarded, no errors,
nothing. Two changes to disambiguate:

1. Drop the explicit audio.input.format / audio.output.format objects.
   24 kHz PCM16 mono is the documented GA default for both directions
   and the only thing OpenAI ever advertised; an unrecognised value
   inside the format object is the most likely reason OpenAI silently
   ignores incoming audio.

2. Forward 'upstream silent NxN250ms' debug envelopes every ~2 s when
   ReceiveJsonAsync keeps returning null, and add a 'server_version'
   field to hello_ack so the Pi journal proves which build is running.
2026-06-12 07:04:53 +00:00
tes a42038b825 Diagnostics: forward upstream events + errors to device as 'debug' envelopes
Coolify container logs aren't accessible from the dev sandbox, so the only
log surface we can read is the Pi journal. The relay now forwards every
upstream event type to the device as {type:'debug', message:'relay evt=<T>'}
and every mid-session upstream error event as {type:'error',code:'upstream_<C>'}.

The client's Session.dispatch handles 'debug' as an INFO log line. Updates
the wake-flow integration test's receive-loop to skip over the new debug
envelopes via a ReceiveUntil helper.
2026-06-12 06:59:14 +00:00
tes 8bb0ce51b1 Plan 3 fix: GA turn_detection auto-create_response + reset state on WS reconnect
GA Realtime API requires create_response:true and interrupt_response:true
inside turn_detection — without them the model never auto-generates a
reply after VAD detects speech_stopped, so the relay idles out at 30s
with no assistant audio. Add both.

Client side: when the /device WS drops (Coolify rolling deploy, etc.),
Session.run_forever reconnects but the StateMachine kept whatever state
it was in — usually LISTENING, which permanently disables the wakeword.
Add StateMachine.force_idle() and a Callbacks.on_disconnected() hook;
main.py wires it to force the state back to IDLE on every disconnect.
Also harden Session's fire-and-forget send_* helpers to swallow
ConnectionClosed instead of bubbling up as unhandled task exceptions.
2026-06-12 06:50:00 +00:00
tes 7294a81a9a Realtime: migrate to OpenAI GA shape (beta API was retired)
Upstream returned 'beta_api_shape_disabled' on every wake. Changes:
- Drop OpenAI-Beta: realtime=v1 header from the upgrade.
- session.update payload nests audio under session.audio.{input,output}
  and adds session.type='realtime'; audio.{input,output}.format is now
  an object {type:audio/pcm, rate:24000} instead of 'pcm16' string.
- Default model bumped to gpt-realtime (the GA name); the relay also
  promotes any stored 'gpt-4o-realtime-preview*' to gpt-realtime so
  pre-Plan-3 DeviceConfig rows keep working without a manual DB edit.
2026-06-12 06:42:21 +00:00
tes d01b78b225 Relay: log upstream events + forward error envelopes to device for diagnostics
Plan 3 smoke surfaced 'session_ended reason=error' on every wake with no
context. The relay now logs every upstream event type at INFO and forwards
any upstream 'error' event to the device as 'error code=upstream_<code>'
so it appears in the Pi's journal alongside the wake/session_ended pair.

Also extends the client's wakeword.predict with per-frame RMS + score
diagnostics and the bridge loop with a 5 s heartbeat for triage.
2026-06-12 06:28:58 +00:00
tes 82ea164695 README: document Plan 3 Pi client surface 2026-06-11 22:56:34 +00:00
tes 8662dbe8a7 Plan 3: main.py composition root (audio + state + wakeword + session + playback wired) 2026-06-11 22:52:01 +00:00
tes b1eeaa8af3 Plan 3: backend WS client + dispatch + tool round-trip 2026-06-11 22:51:23 +00:00
tes fa245d87a5 Plan 3: playback worker + wake/sleep wav + software volume gain 2026-06-11 22:50:13 +00:00
tes d0bd141082 Plan 3: openwakeword wrapper with threshold + cooldown 2026-06-11 22:49:40 +00:00
tes 55257b8867 Plan 3: USB device discovery + persistent audio streams 2026-06-11 22:49:20 +00:00
tes d155af4b70 Plan 3: pair CLI (POST /api/pair, write config.json) 2026-06-11 22:48:46 +00:00
tes 781ef9377a Plan 3: 24 kHz->16 kHz downsample helper for the wakeword feed 2026-06-11 22:48:16 +00:00
tes 4d83f2d9fd Plan 3: load/save ~/assistant/state/config.json with mode 0600 2026-06-11 22:47:50 +00:00
tes d75129c4f9 Plan 3: state machine (IDLE -> WAKE_PENDING -> LISTENING <-> ASSISTANT_SPEAKING -> IDLE_PENDING -> IDLE) 2026-06-11 22:47:23 +00:00
tes 908e3f4907 Plan 3: add client.log structured logger 2026-06-11 22:46:44 +00:00
tes 79d6e923a6 Plan 3: scaffold Python test harness and stub client package 2026-06-11 22:46:25 +00:00
tes 8380e3149e README: document Plan 2 device-hub surface area 2026-06-11 22:02:35 +00:00
tes 7ba9f0cf5e Plan 2: device hub + Realtime relay + tools registry 2026-06-11 22:01:30 +00:00
tes ba709d15d5 DeviceHub: wake → session lifecycle + uplink relay + tool_result routing 2026-06-11 22:00:33 +00:00
tes 9023509b01 DeviceHub: /device WS with bearer auth + hello/hello_ack + ping/pong 2026-06-11 21:55:33 +00:00
tes 4dc48bd9fd Add ActiveDevice + DeviceRegistry + HubEnvelopes + IDeviceClient/Channel implementations 2026-06-11 21:51:25 +00:00
tes 78373b0912 Add OpenAIRealtimeUpstream + RealtimeSessionFactory + DI registrations 2026-06-11 21:48:21 +00:00
tes 8f53f9daa9 RealtimeSession: idle-timeout watchdog (resets on speech_started) 2026-06-11 21:46:16 +00:00
tes a45b25038a Test end_session sequencing closes with reason=tool and skips response.create 2026-06-11 21:42:47 +00:00
tes 7dada5fdb9 RealtimeSession: execute server-side tools at response.done 2026-06-11 21:38:31 +00:00
tes b078f9aa78 RealtimeSession: persist user/assistant transcripts; emit assistant_done 2026-06-11 21:35:40 +00:00
tes 24b8acdf60 RealtimeSession: uplink pump + downlink audio.delta forwarding 2026-06-11 21:33:37 +00:00
tes a202725c39 Add RealtimeSession lifecycle (open + session.update + session_started/ended) 2026-06-11 21:30:40 +00:00
tes 8f197d1203 Add ScriptedRealtimeUpstream + FakeDeviceChannel test doubles 2026-06-11 21:27:50 +00:00
tes 35a6e8febb Add IRealtimeUpstream + RealtimeEvents helpers for upstream JSON 2026-06-11 21:25:46 +00:00
tes d3e101eb29 Add OpenAIKeyProvider + RealtimeSettings record 2026-06-11 21:23:54 +00:00
tes 212ce06f07 Add SetVolumeTool (validates 0..100, delegates to Pi via IDeviceChannel) 2026-06-11 21:22:45 +00:00
tes ebbe23c639 Add EndSessionTool (no params, returns ok; relay closes on this tool name) 2026-06-11 21:21:13 +00:00
tes 9325d21f9e Add ITool + ToolRegistry + GetCurrentTimeTool 2026-06-11 21:19:23 +00:00
tes 13d8773704 Add ConversationLog (start/append/end with monotonic per-convo index) 2026-06-11 21:13:50 +00:00
tes 99cfe60a2c Add Conversation + Turn entities and migration 2026-06-11 21:09:52 +00:00
tes 58b8a446b1 Fix Dockerfile: don't copy test csproj (excluded by .dockerignore) 2026-06-11 20:31:52 +00:00
tes a83e9f93de Add UseForwardedHeaders so /install.sh + backend_ws use https/wss behind Traefik 2026-06-11 20:17:46 +00:00
tes 68738e486d Add README with repo layout, local dev, smoke test, deploy commands 2026-06-11 20:11:39 +00:00
tes 2701b8a483 Add docker-compose.yml for Coolify deploy + gitignore deploy.json 2026-06-11 20:05:58 +00:00
tes 645a7e3722 Persist data-protection keys to /keys when present (Coolify volume) 2026-06-11 19:05:41 +00:00
tes 88339c51eb Add multi-stage Dockerfile (backend + client tarball) 2026-06-11 19:04:45 +00:00
tes a7e5e13ac8 Add Python client placeholders for bundle build (full impl in Plan 3) 2026-06-11 19:04:18 +00:00
tes 751030dcef Add InstallScriptBuilder + GET /install.sh + GET /client.tar.gz 2026-06-11 19:03:33 +00:00
tes e5ae228738 Add /api/devices CRUD (list, get, rename, revoke) with owner scoping 2026-06-11 19:00:42 +00:00
tes b5be4c6914 Add /api/pair-code (auth) + /api/pair (public) endpoints 2026-06-11 18:56:41 +00:00
tes db5c4e59a4 Add PairingService with safe alphabet and 10-min TTL 2026-06-11 18:53:32 +00:00
tes 720583de0a Add PairingCode entity + migration 2026-06-11 18:51:18 +00:00
tes b862bf6f0f Add DeviceTokenService (random token, SHA-256 hash, fixed-time verify) 2026-06-11 18:48:39 +00:00
tes 3e9c245bb3 Fix Task 6: Device FK to AspNetUsers + SystemSettings singleton Id pin 2026-06-11 18:46:42 +00:00
tes 68e7e81312 Add Device, DeviceConfig, SystemSettings entities and seed defaults 2026-06-11 18:41:27 +00:00
tes a1aa5023a2 Fix Task 5 review issues: per-test ApiFactory, drop dead RoleManager param + EF Core using 2026-06-11 18:38:37 +00:00
tes 2a208e81b2 Add /api/auth/{register,login,logout,me} with first-user-is-admin rule 2026-06-11 18:34:07 +00:00
tes 8a7d01ac9e Add ASP.NET Identity (cookie auth) and role seeding 2026-06-11 18:26:34 +00:00
tes 824a27520d Fix: clean up SQLite WAL/SHM sidecars in ApiFactory.Dispose 2026-06-11 18:22:33 +00:00
tes f2bb2d984b Add ApiFactory test helper + /health sanity test 2026-06-11 18:17:48 +00:00
tes 2c16a9485b Add EF Core + SQLite with empty AppDbContext 2026-06-11 18:12:04 +00:00
tes 1eb0383834 Scaffold backend + backend.tests projects 2026-06-11 18:04:42 +00:00
tes cf055c8928 Plan 1: backend foundation + pairing + Coolify deploy
18 bite-sized tasks taking the repo from empty to a deployed ASP.NET app
where users can sign up, generate pairing codes, and pair a device via
curl. Includes data-protection keys, multi-stage Dockerfile, and the
end-to-end smoke test. Defers WS hub, Realtime relay, tools, the real
Python client, and the React UI to Plans 2-4.
2026-06-11 17:48:18 +00:00
tes 99a9950cee Smart-assistant main-build design spec
Captures the architecture for the post-prototype build: Pi Python client +
ASP.NET backend (with embedded React UI) on Coolify, SQLite, multi-turn
Realtime sessions with end_session/get_current_time/set_volume tools,
pairing-code device onboarding, and idempotent install/update flow.
2026-06-11 15:48:10 +00:00
128 changed files with 14152 additions and 2789 deletions
+13
View File
@@ -0,0 +1,13 @@
**/bin
**/obj
**/.vs
.git
.gitignore
.vscode
.idea
docs
tests
backend.tests
*.md
.dockerignore
Dockerfile
+12
View File
@@ -3,3 +3,15 @@ __pycache__/
.venv/
*.wav
.openwakeword-cache/
# .NET build outputs (don't catch root-level bin/ scripts)
backend/bin/
backend/obj/
backend.tests/bin/
backend.tests/obj/
# SQLite dev database (created on first run)
backend/*.db
backend/*.db-shm
backend/*.db-wal
deploy.json
+48
View File
@@ -0,0 +1,48 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.0.31903.59
MinimumVisualStudioVersion = 10.0.40219.1
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "backend", "backend\backend.csproj", "{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "backend.tests", "backend.tests\backend.tests.csproj", "{63423BEA-4ECF-45CF-B0E8-611AC65AE463}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Debug|x64 = Debug|x64
Debug|x86 = Debug|x86
Release|Any CPU = Release|Any CPU
Release|x64 = Release|x64
Release|x86 = Release|x86
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|Any CPU.Build.0 = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x64.ActiveCfg = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x64.Build.0 = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x86.ActiveCfg = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x86.Build.0 = Debug|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|Any CPU.ActiveCfg = Release|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|Any CPU.Build.0 = Release|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x64.ActiveCfg = Release|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x64.Build.0 = Release|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x86.ActiveCfg = Release|Any CPU
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x86.Build.0 = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|Any CPU.Build.0 = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x64.ActiveCfg = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x64.Build.0 = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x86.ActiveCfg = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x86.Build.0 = Debug|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|Any CPU.ActiveCfg = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|Any CPU.Build.0 = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x64.ActiveCfg = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x64.Build.0 = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x86.ActiveCfg = Release|Any CPU
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
-4
View File
@@ -24,7 +24,3 @@ Copy a file to the Pi:
```sh
sshpass -p 'assistant' scp <local> pi@192.168.50.115:<remote>
```
## Prototype + probe findings
`findings.md` is the canonical record of what we proved during the prototype + C# probe phase. Read it before starting any audio, wakeword, or Pi-client work — it captures hardware constraints (USB Speaker Phone is 48 kHz-native, mic and speaker are the same device), the .NET-on-Linux env-var gotcha (`Environment.SetEnvironmentVariable` does NOT reach libc `getenv()`; you must P/Invoke `setenv` for libportaudio and onnxruntime to see env knobs), and the punch list of items to redesign before the main assistant. The Python prototypes live at `tests/01-record-play/` and `tests/02-wakeword/`; the C# Test 1 port at `tests/01-record-play-cs/` (deployed via `bin/probe-cs`).
+30
View File
@@ -0,0 +1,30 @@
# syntax=docker/dockerfile:1.7
# ---- Backend build ----
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
WORKDIR /src
COPY backend/backend.csproj backend/
RUN dotnet restore backend/backend.csproj
COPY backend/ backend/
RUN dotnet publish backend/backend.csproj -c Release -o /publish
# ---- Client bundle ----
FROM alpine:3.20 AS bundle
WORKDIR /src
RUN apk add --no-cache tar gzip
COPY client/ client/
COPY requirements.txt requirements-nodeps.txt ./
RUN tar czf /client.tar.gz client/ requirements.txt requirements-nodeps.txt
# ---- Final ----
FROM mcr.microsoft.com/dotnet/aspnet:9.0
WORKDIR /app
COPY --from=build /publish/ ./
RUN mkdir -p ./wwwroot && mkdir -p /data /keys
COPY --from=bundle /client.tar.gz ./wwwroot/client.tar.gz
ENV ASPNETCORE_URLS=http://+:8080
ENV ConnectionStrings__Default="Data Source=/data/assistant.db"
ENV Install__ClientTarballPath="/app/wwwroot/client.tar.gz"
EXPOSE 8080
ENTRYPOINT ["dotnet", "backend.dll"]
+213
View File
@@ -0,0 +1,213 @@
# Smart Assistant
Voice assistant on a Raspberry Pi talking to the OpenAI Realtime API via an
ASP.NET backend deployed on Coolify.
See `docs/superpowers/specs/2026-06-11-smart-assistant-design.md` for the full
design. Implementation plans:
- Plan 1 — `docs/superpowers/plans/2026-06-11-smart-assistant-backend-foundation.md`
(Identity, devices, pairing, install endpoints).
- Plan 2 — `docs/superpowers/plans/2026-06-11-smart-assistant-device-hub-realtime-tools.md`
(device WebSocket hub, OpenAI Realtime relay, tools registry).
## Repo layout
- `backend/` — ASP.NET 9 backend (Identity, Devices, Pairing, Install).
- `backend.tests/` — xUnit integration + unit tests via `WebApplicationFactory`.
- `client/` — Python client. **Plan 1 ships placeholders only**; the full
wakeword + Realtime client lands in Plan 3.
- `Dockerfile` + `docker-compose.yml` — multi-stage build; produces one image
with the published ASP.NET app + a bundled `client.tar.gz` in
`wwwroot/client.tar.gz`.
- `deploy.json` — Coolify config (gitignored; contains the OpenAI key).
- `docs/superpowers/` — specs and plans.
## Local development
```sh
dotnet test # all backend tests
dotnet run --project backend # starts on http://localhost:5252
```
Hit `http://localhost:5252/health` to verify it's up.
## Smoke test against a live deploy
The full pairing flow can be exercised with `curl` — no UI needed yet.
```sh
DOMAIN="https://assistant.volcanic.tes.gd" # or your domain
COOKIE=$(mktemp)
# Register the first user (becomes Admin automatically) and sign in
curl -sf -c "$COOKIE" -X POST "$DOMAIN/api/auth/register" \
-H 'content-type: application/json' \
-d '{"email":"you@example.com","password":"Passw0rd!"}'
curl -sf -c "$COOKIE" -b "$COOKIE" -X POST "$DOMAIN/api/auth/login" \
-H 'content-type: application/json' \
-d '{"email":"you@example.com","password":"Passw0rd!"}'
# Generate a pairing code (auth required)
CODE=$(curl -sf -c "$COOKIE" -b "$COOKIE" -X POST "$DOMAIN/api/pair-code" \
-H 'content-type: application/json' \
-d '{"name":"kitchen"}' \
| python3 -c 'import sys,json; print(json.load(sys.stdin)["code"])')
echo "code=$CODE"
# Pair as a device (public endpoint) and capture the token
curl -sf -X POST "$DOMAIN/api/pair" \
-H 'content-type: application/json' \
-d "{\"code\":\"$CODE\",\"hostname\":\"smoke\",\"client_version\":\"0.1\"}" \
| python3 -m json.tool
# Verify the install script and client bundle are served
curl -sf "$DOMAIN/install.sh" | head -5
curl -sIf "$DOMAIN/client.tar.gz"
```
Expected: `{"ok":true}` on `/health`, JSON with `device_id`/`device_token`/`backend_ws`
from `/api/pair`, the bash shebang from `/install.sh`, and a `200 OK` with
`Content-Type: application/gzip` for the bundle.
> **Plan 1 ships a placeholder Python bundle** — running the bundled
> `client.main` will exit with the placeholder message. Plan 3 replaces it
> with the real wakeword + Realtime client.
## Deploying
Use the Coolify v4 API. The `deploy.json` file (gitignored) carries the
project + server UUIDs and the OpenAI key.
```sh
APP=$(python3 -c "import json; print(json.load(open('deploy.json'))['app']['uuid'])")
curl -sf -X POST "$COOLIFY_URL/api/v1/deploy?uuid=$APP" \
-H "Authorization: Bearer $COOLIFY_KEY"
```
Fire-and-forget; the API returns a `deployment_uuid` immediately. Don't poll
(per the global Coolify rule in `CLAUDE.md`).
## Plan 2: device hub + Realtime relay
- `wss://<domain>/device` — bearer-token WebSocket. Pair a device first to get a token.
- Handshake: client sends `{type:"hello", device_id, client_version}`, backend
replies `hello_ack` with per-device config (voice, model, system prompt, idle
timeout, enabled tools).
- Heartbeat: client sends `{type:"ping"}` every 15 s, backend replies `pong`
and updates `Devices.LastSeenAt`.
- Wake: `{type:"wake"}` opens a Realtime session against OpenAI; backend emits
`session_started` (with `conversation_id`), forwards `response.audio.delta`
as binary frames to the device, persists `response.audio_transcript.done` as
an assistant turn, persists `conversation.item.input_audio_transcription.completed`
as a user turn, and emits `assistant_done` after each `response.done`.
- Tools shipped (per-device toggle via `DeviceConfig.EnabledToolsJson`):
- `get_current_time` (server-side, returns `{"now":"<ISO-UTC>"}`),
- `end_session` (closes the session after the current `response.done`
completes; emits `session_ended(reason="tool")`),
- `set_volume` (round-trips a `tool_call`/`tool_result` envelope to the Pi;
the Pi side will run `amixer` in Plan 3).
- Idle timeout: server-side. With no `input_audio_buffer.speech_started` upstream
event for `IdleTimeoutSeconds`, the relay emits `session_ended(reason="idle")`.
The Python Pi client (Plan 3) is the production consumer of this surface. A
quick handshake-only smoke against the live deploy:
```sh
DOMAIN="https://assistant.volcanic.tes.gd"
# Reuse the curl recipe above to log in, generate a pair-code, and pair
# a "smoke" device — capture $TOKEN from the /api/pair response.
python3 - <<PY
import asyncio, json, websockets
URL, TOKEN = "wss://assistant.volcanic.tes.gd/device", "$TOKEN"
async def main():
async with websockets.connect(
URL, additional_headers={"Authorization": f"Bearer {TOKEN}"}
) as ws:
await ws.send(json.dumps({
"type":"hello","device_id":"00000000-0000-0000-0000-000000000000",
"client_version":"0.1"}))
print("ack:", json.loads(await ws.recv())["type"])
await ws.send(json.dumps({"type":"ping"}))
print("pong:", json.loads(await ws.recv())["type"])
asyncio.run(main())
PY
```
Don't send `wake` from the smoke — it would open a real OpenAI session and
burn budget. Plan 3 exercises that end-to-end.
## Plan 3: Pi client
The Python client lives in `client/`. It is shipped to the Pi inside
`client.tar.gz` (built into the backend Docker image) and installed by
`install.sh`. Modules:
- `main.py` — entry point. Sets `PA_ALSA_PLUGHW=1` before importing
sounddevice. Wires audio, state, wakeword, session, and playback.
- `state.py` — `IDLE → WAKE_PENDING → LISTENING ↔ ASSISTANT_SPEAKING →
IDLE_PENDING → IDLE`. `wakeword_enabled` is true only in `IDLE`;
`uplink_enabled` is true only in `LISTENING`.
- `audio.py` — USB device discovery, persistent 24 kHz/mono/int16 InputStream
and OutputStream, and the 24 → 16 kHz resample for the wakeword feed.
- `wakeword.py` — openwakeword `alexa` model, threshold + cooldown.
- `playback.py` — single worker thread that owns the OutputStream and
implements `set_volume` as a software gain (the USB Speaker Phone has no
hardware playback volume control; only a `PCM Playback Switch`).
- `session.py` — backend WS client, dispatch, tool round-trip,
exponential-backoff reconnect.
- `config.py` — `~/assistant/state/config.json` (mode 0600).
- `pair.py` — `python -m client.pair --backend <url> [--code <code>]`.
### Updating the Pi
For a fast iteration loop:
```sh
sshpass -p 'assistant' rsync -az --delete --exclude __pycache__ --exclude tests \
client/ pi@192.168.50.115:assistant/code/client/
sshpass -p 'assistant' ssh pi@192.168.50.115 'systemctl --user restart assistant'
sshpass -p 'assistant' ssh pi@192.168.50.115 \
'journalctl --user -u assistant -n 50 --no-pager'
```
For a production update (after a backend deploy):
```sh
sshpass -p 'assistant' ssh pi@192.168.50.115 \
'curl -fsSL https://assistant.volcanic.tes.gd/install.sh | bash'
```
### Local tests
```sh
python3 -m venv .venv
.venv/bin/pip install -r requirements-dev.txt requests websockets numpy scipy
.venv/bin/python -m pytest client/tests
```
Hardware-touching modules (`audio.py` streams, `wakeword.py`, `playback.py`,
`session.run_forever`) are smoke-tested via the live deploy. Deterministic
modules (`state.py`, `config.py`, `pair.py`, the resample helper, the session
dispatcher) are covered by the pytest suite.
### End-to-end smoke
With the unit running on the Pi:
```sh
sshpass -p 'assistant' ssh pi@192.168.50.115 'journalctl --user -u assistant -f'
```
…then say `alexa` near the Speaker Phone. Expected log sequence: `wakeword
fired`, `state IDLE -> WAKE_PENDING`, `session_started conversation_id=…`,
`state WAKE_PENDING -> LISTENING`. Speak a question; the assistant replies
through the speaker (`state LISTENING -> ASSISTANT_SPEAKING`, then
`assistant_done`). Say "bye" → `session_ended reason=tool`, then `state
LISTENING -> IDLE_PENDING -> IDLE`. The conversation row + turns are visible
via the deployed backend's DB (admin UI in Plan 4).
## What's next
- **Plan 4** — React management UI for the admin dashboard, per-device
config editor, and conversation history viewer.
+65
View File
@@ -0,0 +1,65 @@
using backend.Realtime;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Data.Sqlite;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
namespace backend.tests;
public class ApiFactory : WebApplicationFactory<Program>
{
private readonly string _dbPath = Path.Combine(Path.GetTempPath(), $"assistant-test-{Guid.NewGuid():N}.db");
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.UseEnvironment("Test");
builder.ConfigureAppConfiguration((_, cfg) =>
{
cfg.AddInMemoryCollection(new Dictionary<string, string?>
{
["ConnectionStrings:Default"] = $"Data Source={_dbPath}",
["OPENAI_API_KEY"] = "test-key",
});
});
builder.ConfigureTestServices(services =>
{
services.AddScoped<RealtimeSessionFactory, FakeRealtimeSessionFactory>();
});
}
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
SqliteConnection.ClearAllPools();
foreach (var suffix in new[] { "", "-shm", "-wal", "-journal" })
{
var p = _dbPath + suffix;
try { if (File.Exists(p)) File.Delete(p); } catch { /* best effort */ }
}
}
}
public class FakeRealtimeSessionFactory(
OpenAIKeyProvider keys,
backend.Tools.ToolRegistry registry,
backend.Conversations.ConversationLog log,
ILoggerFactory loggerFactory)
: RealtimeSessionFactory(keys, registry, log, loggerFactory)
{
public static ScriptedRealtimeUpstream Upstream { get; set; } = new();
public override Task<RealtimeSession> CreateAsync(
Guid deviceId, RealtimeSettings cfg, ISessionOutput output,
backend.Tools.IDeviceChannel channel, CancellationToken ct)
{
Upstream.Push(new System.Text.Json.Nodes.JsonObject { ["type"] = "session.updated" });
var s = new RealtimeSession(
deviceId, Upstream, output, log, registry, cfg, channel,
loggerFactory.CreateLogger<RealtimeSession>());
return Task.FromResult(s);
}
}
+51
View File
@@ -0,0 +1,51 @@
using System.Net;
using System.Net.Http.Json;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class AuthTests
{
[Fact]
public async Task Register_then_login_then_me_returns_user()
{
using var factory = new ApiFactory();
var client = factory.CreateClient();
var reg = await client.PostAsJsonAsync("/api/auth/register",
new { email = "first@example.com", password = "Passw0rd!" });
reg.StatusCode.Should().Be(HttpStatusCode.OK);
var login = await client.PostAsJsonAsync("/api/auth/login",
new { email = "first@example.com", password = "Passw0rd!" });
login.StatusCode.Should().Be(HttpStatusCode.OK);
var me = await client.GetAsync("/api/auth/me");
me.StatusCode.Should().Be(HttpStatusCode.OK);
(await me.Content.ReadAsStringAsync()).Should().Contain("first@example.com");
}
[Fact]
public async Task Me_without_login_returns_401()
{
using var factory = new ApiFactory();
var client = factory.CreateClient();
var res = await client.GetAsync("/api/auth/me");
res.StatusCode.Should().Be(HttpStatusCode.Unauthorized);
}
[Fact]
public async Task First_user_is_admin()
{
using var factory = new ApiFactory();
var client = factory.CreateClient();
await client.PostAsJsonAsync("/api/auth/register",
new { email = "admin@example.com", password = "Passw0rd!" });
await client.PostAsJsonAsync("/api/auth/login",
new { email = "admin@example.com", password = "Passw0rd!" });
var me = await client.GetAsync("/api/auth/me");
(await me.Content.ReadAsStringAsync()).Should().Contain("\"isAdmin\":true");
}
}
+71
View File
@@ -0,0 +1,71 @@
using backend.Conversations;
using backend.Data;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class ConversationLogTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
private async Task<(AppDbContext db, ConversationLog log, Guid deviceId)> ArrangeAsync()
{
var scope = _factory.Services.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var deviceId = Guid.NewGuid();
return (db, log, deviceId);
}
[Fact]
public async Task Start_creates_conversation_row()
{
var (db, log, deviceId) = await ArrangeAsync();
var convo = await log.StartAsync(deviceId, default);
var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
row.DeviceId.Should().Be(deviceId);
row.EndedAt.Should().BeNull();
row.EndReason.Should().Be(EndReason.Unknown);
}
[Fact]
public async Task Turns_are_persisted_with_monotonic_index_per_conversation()
{
var (db, log, deviceId) = await ArrangeAsync();
var convo = await log.StartAsync(deviceId, default);
await log.AppendUserAsync(convo.Id, "hi", default);
await log.AppendAssistantAsync(convo.Id, "hello there", default);
await log.AppendToolAsync(convo.Id, "get_current_time", "{}", "{\"now\":\"2026-06-11T00:00:00Z\"}", default);
var turns = await db.Turns
.Where(t => t.ConversationId == convo.Id)
.OrderBy(t => t.Index)
.ToListAsync();
turns.Should().HaveCount(3);
turns.Select(t => t.Index).Should().Equal(0, 1, 2);
turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant, TurnRole.Tool);
turns[0].Text.Should().Be("hi");
turns[1].Text.Should().Be("hello there");
turns[2].ToolName.Should().Be("get_current_time");
}
[Fact]
public async Task End_sets_ended_at_and_reason()
{
var (db, log, deviceId) = await ArrangeAsync();
var convo = await log.StartAsync(deviceId, default);
await log.EndAsync(convo.Id, EndReason.Idle, default);
var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
row.EndedAt.Should().NotBeNull();
row.EndReason.Should().Be(EndReason.Idle);
}
}
+91
View File
@@ -0,0 +1,91 @@
using System.Net.Http.Json;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class DeviceHubHandshakeTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
private async Task<string> PairAndGetTokenAsync(string email)
{
var http = _factory.CreateClient();
await http.PostAsJsonAsync("/api/auth/register",
new { email, password = "Passw0rd!" });
await http.PostAsJsonAsync("/api/auth/login",
new { email, password = "Passw0rd!" });
var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "ws-test" });
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "smoke", client_version = "0.1" });
var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
return pairBody!["device_token"].ToString()!;
}
[Fact]
public async Task Connect_without_token_fails_upgrade()
{
var client = _factory.Server.CreateWebSocketClient();
var uri = new Uri(_factory.Server.BaseAddress, "device");
var act = async () => await client.ConnectAsync(uri, CancellationToken.None);
await act.Should().ThrowAsync<Exception>();
}
[Fact]
public async Task Hello_ack_includes_config()
{
var token = await PairAndGetTokenAsync("ws1@example.com");
var client = _factory.Server.CreateWebSocketClient();
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
var uri = new Uri(_factory.Server.BaseAddress, "device");
var ws = await client.ConnectAsync(uri, CancellationToken.None);
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
var ack = await ReceiveJson(ws);
ack.GetProperty("type").GetString().Should().Be("hello_ack");
ack.GetProperty("config").GetProperty("voice").GetString().Should().NotBeNullOrEmpty();
ack.GetProperty("config").GetProperty("model").GetString().Should().NotBeNullOrEmpty();
ack.GetProperty("config").GetProperty("enabled_tools").GetArrayLength().Should().BeGreaterThan(0);
}
[Fact]
public async Task Ping_replies_with_pong()
{
var token = await PairAndGetTokenAsync("ws2@example.com");
var client = _factory.Server.CreateWebSocketClient();
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
var uri = new Uri(_factory.Server.BaseAddress, "device");
var ws = await client.ConnectAsync(uri, CancellationToken.None);
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
await ReceiveJson(ws);
await SendJson(ws, new { type = "ping" });
var pong = await ReceiveJson(ws);
pong.GetProperty("type").GetString().Should().Be("pong");
}
private static async Task SendJson(WebSocket ws, object payload)
{
var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
}
private static async Task<JsonElement> ReceiveJson(WebSocket ws)
{
var buf = new byte[8192];
using var stream = new MemoryStream();
while (true)
{
var r = await ws.ReceiveAsync(buf, CancellationToken.None);
stream.Write(buf, 0, r.Count);
if (r.EndOfMessage) break;
}
return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
}
}
+129
View File
@@ -0,0 +1,129 @@
using System.Net.Http.Json;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using FluentAssertions;
using Xunit;
namespace backend.tests;
[Collection("WakeFlow")] // tests touch FakeRealtimeSessionFactory.Upstream static — serialize
public class DeviceHubWakeFlowTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
private async Task<string> PairAndGetTokenAsync(string email)
{
var http = _factory.CreateClient();
await http.PostAsJsonAsync("/api/auth/register",
new { email, password = "Passw0rd!" });
await http.PostAsJsonAsync("/api/auth/login",
new { email, password = "Passw0rd!" });
var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "wake-test" });
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "smoke", client_version = "0.1" });
var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
return pairBody!["device_token"].ToString()!;
}
[Fact]
public async Task Wake_opens_session_started_then_assistant_done_then_session_ended_on_cancel()
{
FakeRealtimeSessionFactory.Upstream = new ScriptedRealtimeUpstream();
var token = await PairAndGetTokenAsync("wake1@example.com");
var client = _factory.Server.CreateWebSocketClient();
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
var uri = new Uri(_factory.Server.BaseAddress, "device");
var ws = await client.ConnectAsync(uri, CancellationToken.None);
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
await ReceiveJson(ws); // ack
await SendJson(ws, new { type = "wake", at = 0L });
var started = await ReceiveJson(ws);
started.GetProperty("type").GetString().Should().Be("session_started");
FakeRealtimeSessionFactory.Upstream.Push(new JsonObject
{
["type"] = "response.audio_transcript.done",
["transcript"] = "hello!",
});
FakeRealtimeSessionFactory.Upstream.Push(new JsonObject { ["type"] = "response.done" });
var done = await ReceiveUntil(ws, "assistant_done");
done.GetProperty("type").GetString().Should().Be("assistant_done");
await SendJson(ws, new { type = "cancel" });
var ended = await ReceiveUntil(ws, "session_ended");
ended.GetProperty("type").GetString().Should().Be("session_ended");
}
[Fact]
public async Task Binary_uplink_after_wake_reaches_upstream_as_input_audio_buffer_append()
{
FakeRealtimeSessionFactory.Upstream = new ScriptedRealtimeUpstream();
var token = await PairAndGetTokenAsync("wake2@example.com");
var client = _factory.Server.CreateWebSocketClient();
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
var uri = new Uri(_factory.Server.BaseAddress, "device");
var ws = await client.ConnectAsync(uri, CancellationToken.None);
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
await ReceiveJson(ws);
await SendJson(ws, new { type = "wake", at = 0L });
await ReceiveJson(ws); // session_started
var frame = new byte[3840];
for (int i = 0; i < frame.Length; i++) frame[i] = 0x55;
await ws.SendAsync(frame, WebSocketMessageType.Binary, true, CancellationToken.None);
// Allow time for the pump's 250 ms poll + send.
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(2);
while (DateTime.UtcNow < deadline)
{
if (FakeRealtimeSessionFactory.Upstream.Sent.Any(e =>
(string?)e["type"] == "input_audio_buffer.append")) break;
await Task.Delay(50);
}
FakeRealtimeSessionFactory.Upstream.Sent
.Any(e => (string?)e["type"] == "input_audio_buffer.append")
.Should().BeTrue();
await SendJson(ws, new { type = "cancel" });
await ReceiveJson(ws);
}
private static async Task SendJson(WebSocket ws, object payload)
{
var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
}
private static async Task<JsonElement> ReceiveJson(WebSocket ws)
{
var buf = new byte[16384];
using var stream = new MemoryStream();
while (true)
{
var r = await ws.ReceiveAsync(buf, CancellationToken.None);
if (r.MessageType == WebSocketMessageType.Binary) { stream.Write(buf, 0, r.Count); continue; }
stream.Write(buf, 0, r.Count);
if (r.EndOfMessage) break;
}
return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
}
// The relay forwards an upstream-event "debug" envelope for every event
// it sees, so envelopes the test cares about are interleaved with debugs.
private static async Task<JsonElement> ReceiveUntil(WebSocket ws, string type)
{
for (int i = 0; i < 50; i++)
{
var env = await ReceiveJson(ws);
if (env.GetProperty("type").GetString() == type) return env;
}
throw new TimeoutException($"never saw envelope type={type}");
}
}
+25
View File
@@ -0,0 +1,25 @@
using System.Net.WebSockets;
using backend.DeviceHub;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class DeviceRegistryTests
{
[Fact]
public void Register_unregister_round_trip()
{
var reg = new DeviceRegistry();
var id = Guid.NewGuid();
var dev = new ActiveDevice(id, new ClientWebSocket());
reg.TryRegister(dev).Should().BeTrue();
reg.TryRegister(dev).Should().BeFalse();
reg.IsOnline(id).Should().BeTrue();
reg.OnlineCount.Should().Be(1);
reg.Unregister(id).Should().BeTrue();
reg.IsOnline(id).Should().BeFalse();
reg.OnlineCount.Should().Be(0);
}
}
+36
View File
@@ -0,0 +1,36 @@
using backend.Devices;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class DeviceTokenServiceTests
{
private readonly DeviceTokenService _svc = new();
[Fact]
public void Generate_returns_url_safe_token_of_expected_length()
{
var (plaintext, hash) = _svc.Generate();
plaintext.Should().MatchRegex("^[A-Za-z0-9_-]+$");
plaintext.Length.Should().BeGreaterThanOrEqualTo(40);
hash.Should().NotBeNullOrEmpty();
hash.Should().NotBe(plaintext);
}
[Fact]
public void Hash_is_deterministic_for_same_input()
{
var (plaintext, hash) = _svc.Generate();
_svc.Hash(plaintext).Should().Be(hash);
}
[Fact]
public void Verify_accepts_matching_plaintext_and_rejects_others()
{
var (plaintext, hash) = _svc.Generate();
_svc.Verify(plaintext, hash).Should().BeTrue();
_svc.Verify(plaintext + "x", hash).Should().BeFalse();
_svc.Verify("totally-different", hash).Should().BeFalse();
}
}
+77
View File
@@ -0,0 +1,77 @@
using System.Net;
using System.Net.Http.Json;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class DevicesEndpointsTests
{
private static async Task<(HttpClient signedIn, string code)> SetupWithPairCode(ApiFactory factory, string email = "owner@example.com")
{
var client = factory.CreateClient();
await client.PostAsJsonAsync("/api/auth/register",
new { email, password = "Passw0rd!" });
await client.PostAsJsonAsync("/api/auth/login",
new { email, password = "Passw0rd!" });
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
var body = await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
return (client, body!["code"].ToString()!);
}
[Fact]
public async Task User_sees_only_their_own_devices()
{
using var factory = new ApiFactory();
var (alice, codeA) = await SetupWithPairCode(factory, "alice@example.com");
await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code = codeA, hostname = "rpi", client_version = "0.1" });
var (bob, codeB) = await SetupWithPairCode(factory, "bob@example.com");
await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code = codeB, hostname = "rpi", client_version = "0.1" });
var aliceList = await alice.GetStringAsync("/api/devices");
var bobList = await bob.GetStringAsync("/api/devices");
aliceList.Should().Contain("kitchen");
bobList.Should().Contain("kitchen");
aliceList.Should().NotBe(bobList);
}
[Fact]
public async Task Rename_device_updates_name()
{
using var factory = new ApiFactory();
var (client, code) = await SetupWithPairCode(factory);
await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "rpi", client_version = "0.1" });
var listJson = await client.GetFromJsonAsync<List<Dictionary<string, object>>>("/api/devices");
var id = listJson![0]["id"].ToString();
var patchRes = await client.PatchAsJsonAsync($"/api/devices/{id}", new { name = "living-room" });
patchRes.StatusCode.Should().Be(HttpStatusCode.OK);
var after = await client.GetStringAsync($"/api/devices/{id}");
after.Should().Contain("living-room");
}
[Fact]
public async Task Revoke_device_sets_is_revoked_true()
{
using var factory = new ApiFactory();
var (client, code) = await SetupWithPairCode(factory);
await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "rpi", client_version = "0.1" });
var listJson = await client.GetFromJsonAsync<List<Dictionary<string, object>>>("/api/devices");
var id = listJson![0]["id"].ToString();
var del = await client.DeleteAsync($"/api/devices/{id}");
del.StatusCode.Should().Be(HttpStatusCode.OK);
var after = await client.GetStringAsync($"/api/devices/{id}");
after.Should().Contain("\"isRevoked\":true");
}
}
+35
View File
@@ -0,0 +1,35 @@
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class EndSessionToolTests
{
[Fact]
public async Task Returns_ok_true()
{
var tool = new EndSessionTool();
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());
var res = await tool.ExecuteAsync(
new ToolInvocation("call_1", JsonDocument.Parse("{}").RootElement), ctx, default);
res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
}
[Fact]
public void Name_and_runs_during_response_match_spec()
{
var tool = new EndSessionTool();
tool.Name.Should().Be("end_session");
tool.RunsDuringResponse.Should().BeFalse();
}
private class NoopChannel : IDeviceChannel
{
public Task<JsonElement> CallPiToolAsync(string n, JsonElement a, CancellationToken ct)
=> throw new NotSupportedException();
}
}
+32
View File
@@ -0,0 +1,32 @@
using System.Collections.Concurrent;
using System.Text.Json;
using backend.DeviceHub;
using backend.Tools;
namespace backend.tests;
public class FakeDeviceChannel : IDeviceChannel, IDeviceClient
{
public List<HubEnvelope> SentEnvelopes { get; } = new();
public List<byte[]> SentBinary { get; } = new();
public ConcurrentQueue<JsonElement> PiToolReplies { get; } = new();
public Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
{
SentEnvelopes.Add(envelope);
return Task.CompletedTask;
}
public Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
{
SentBinary.Add(bytes.ToArray());
return Task.CompletedTask;
}
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
{
if (PiToolReplies.TryDequeue(out var reply))
return Task.FromResult(reply);
return Task.FromResult(JsonDocument.Parse("""{"ok":true}""").RootElement);
}
}
+29
View File
@@ -0,0 +1,29 @@
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class GetCurrentTimeToolTests
{
[Fact]
public async Task Returns_now_field_with_iso_utc_timestamp()
{
var tool = new GetCurrentTimeTool();
var args = JsonDocument.Parse("{}").RootElement;
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());
var res = await tool.ExecuteAsync(new ToolInvocation("call_1", args), ctx, default);
res.Output.TryGetProperty("now", out var now).Should().BeTrue();
now.GetString().Should().MatchRegex(@"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}");
now.GetString()!.Should().EndWith("Z");
}
private class NoopChannel : IDeviceChannel
{
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
=> throw new NotSupportedException();
}
}
+19
View File
@@ -0,0 +1,19 @@
using System.Net;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class HealthTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task Health_returns_ok()
{
var client = _factory.CreateClient();
var res = await client.GetAsync("/health");
res.StatusCode.Should().Be(HttpStatusCode.OK);
(await res.Content.ReadAsStringAsync()).Should().Contain("\"ok\":true");
}
}
+55
View File
@@ -0,0 +1,55 @@
using System.Net;
using FluentAssertions;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Xunit;
namespace backend.tests;
public class InstallEndpointsTests
{
[Fact]
public async Task Install_sh_returns_shell_script_with_request_host()
{
using var factory = new ApiFactory();
var client = factory.CreateClient();
var res = await client.GetAsync("/install.sh");
res.StatusCode.Should().Be(HttpStatusCode.OK);
res.Content.Headers.ContentType!.MediaType.Should().Be("text/x-shellscript");
var body = await res.Content.ReadAsStringAsync();
body.Should().StartWith("#!/usr/bin/env bash");
body.Should().Contain("BACKEND_URL=\"http://localhost\"");
}
[Fact]
public async Task Client_tarball_returns_octet_stream()
{
var tmp = Path.Combine(Path.GetTempPath(), $"client-test-{Guid.NewGuid():N}.tar.gz");
File.WriteAllBytes(tmp, new byte[] { 0x1f, 0x8b }); // tiny gzip-ish marker
try
{
using var factory = new ApiFactory();
var client = factory.WithWebHostBuilder(b =>
{
b.ConfigureAppConfiguration((_, cfg) =>
{
cfg.AddInMemoryCollection(new Dictionary<string, string?>
{
["Install:ClientTarballPath"] = tmp,
});
});
}).CreateClient();
var res = await client.GetAsync("/client.tar.gz");
res.StatusCode.Should().Be(HttpStatusCode.OK);
res.Content.Headers.ContentType!.MediaType.Should().Be("application/gzip");
(await res.Content.ReadAsByteArrayAsync()).Length.Should().Be(2);
}
finally
{
if (File.Exists(tmp)) File.Delete(tmp);
}
}
}
@@ -0,0 +1,18 @@
using backend.Install;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class InstallScriptBuilderTests
{
[Fact]
public void Build_inlines_the_backend_url_into_the_script()
{
var script = new InstallScriptBuilder().Build("https://example.test");
script.Should().Contain("BACKEND_URL=\"https://example.test\"");
script.Should().Contain("#!/usr/bin/env bash");
script.Should().Contain("$BACKEND_URL/client.tar.gz");
script.Should().Contain("client.pair");
}
}
+73
View File
@@ -0,0 +1,73 @@
using System.Net;
using System.Net.Http.Json;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class PairingEndpointsTests
{
private static async Task<HttpClient> SignedInClient(ApiFactory factory, string email = "owner@example.com")
{
var client = factory.CreateClient();
await client.PostAsJsonAsync("/api/auth/register",
new { email, password = "Passw0rd!" });
await client.PostAsJsonAsync("/api/auth/login",
new { email, password = "Passw0rd!" });
return client;
}
[Fact]
public async Task Anonymous_user_cannot_request_a_pairing_code()
{
using var factory = new ApiFactory();
var client = factory.CreateClient();
var res = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
res.StatusCode.Should().Be(HttpStatusCode.Unauthorized);
}
[Fact]
public async Task Pair_code_then_pair_returns_token_and_device_id()
{
using var factory = new ApiFactory();
var client = await SignedInClient(factory);
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
codeRes.StatusCode.Should().Be(HttpStatusCode.OK);
var codeBody = await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
var code = codeBody!["code"].ToString()!;
var pairRes = await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "rpi", client_version = "0.1" });
pairRes.StatusCode.Should().Be(HttpStatusCode.OK);
var body = await pairRes.Content.ReadAsStringAsync();
body.Should().Contain("device_id");
body.Should().Contain("device_token");
body.Should().Contain("backend_ws");
}
[Fact]
public async Task Pair_with_invalid_code_returns_404()
{
using var factory = new ApiFactory();
var res = await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code = "ZZZZZZZZ", hostname = "rpi", client_version = "0.1" });
res.StatusCode.Should().Be(HttpStatusCode.NotFound);
}
[Fact]
public async Task Pair_with_consumed_code_returns_404()
{
using var factory = new ApiFactory();
var client = await SignedInClient(factory);
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
var first = await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "rpi", client_version = "0.1" });
first.StatusCode.Should().Be(HttpStatusCode.OK);
var second = await factory.CreateClient().PostAsJsonAsync("/api/pair",
new { code, hostname = "rpi", client_version = "0.1" });
second.StatusCode.Should().Be(HttpStatusCode.NotFound);
}
}
+28
View File
@@ -0,0 +1,28 @@
using backend.Pairing;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class PairingServiceTests
{
[Fact]
public void Generated_codes_use_only_safe_alphabet_and_are_8_chars()
{
var svc = new PairingService();
for (var i = 0; i < 50; i++)
{
var code = svc.GenerateCode();
code.Length.Should().Be(8);
code.Should().MatchRegex("^[A-HJ-KM-NP-Z2-9]+$");
}
}
[Fact]
public void Generated_codes_are_not_all_identical()
{
var svc = new PairingService();
var codes = Enumerable.Range(0, 20).Select(_ => svc.GenerateCode()).ToHashSet();
codes.Count.Should().BeGreaterThan(15);
}
}
@@ -0,0 +1,81 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionAudioTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task Uplink_bytes_become_input_audio_buffer_append()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(Array.Empty<ITool>());
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var cts = new CancellationTokenSource();
var run = session.RunAsync(cts.Token);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
var frame = new byte[3840];
for (int i = 0; i < frame.Length; i++) frame[i] = (byte)(i % 256);
await session.WriteUplinkAsync(frame, default);
await Task.Delay(100);
var appended = upstream.Sent
.FirstOrDefault(e => (string?)e["type"] == "input_audio_buffer.append");
appended.Should().NotBeNull();
var b64 = (string?)appended!["audio"];
Convert.FromBase64String(b64!).Length.Should().Be(3840);
cts.Cancel();
await run;
}
[Fact]
public async Task Audio_delta_from_upstream_becomes_binary_to_device()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(Array.Empty<ITool>());
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var cts = new CancellationTokenSource();
var run = session.RunAsync(cts.Token);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
var payload = new byte[3840];
for (int i = 0; i < payload.Length; i++) payload[i] = (byte)((i + 1) % 256);
upstream.Push(new JsonObject
{
["type"] = "response.audio.delta",
["delta"] = Convert.ToBase64String(payload),
});
await Task.Delay(100);
sink.Binary.Should().NotBeEmpty();
sink.Binary[0].Length.Should().Be(3840);
sink.Binary[0][0].Should().Be(1);
cts.Cancel();
await run;
}
}
@@ -0,0 +1,54 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionEndSessionTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task End_session_call_closes_session_with_reason_tool_and_skips_response_create()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(new ITool[] { new EndSessionTool() });
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "end_session" });
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var run = session.RunAsync(default);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
upstream.Push(new JsonObject
{
["type"] = "response.audio.delta",
["delta"] = Convert.ToBase64String(new byte[3840]),
});
upstream.Push(new JsonObject
{
["type"] = "response.function_call_arguments.done",
["call_id"] = "c1",
["name"] = "end_session",
["arguments"] = "{}",
});
upstream.Push(new JsonObject { ["type"] = "response.done" });
await run;
sink.Binary.Should().NotBeEmpty();
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
((string?)ended["reason"]).Should().Be("tool");
upstream.Sent.Should().NotContain(e => (string?)e["type"] == "response.create");
}
}
@@ -0,0 +1,63 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionIdleTimeoutTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task Session_ends_with_reason_idle_when_no_speech_for_timeout()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(Array.Empty<ITool>());
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 1, new HashSet<string>());
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var run = session.RunAsync(default);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
await run.WaitAsync(TimeSpan.FromSeconds(4));
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
((string?)ended["reason"]).Should().Be("idle");
}
[Fact]
public async Task Speech_started_resets_the_idle_timer()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(Array.Empty<ITool>());
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 2, new HashSet<string>());
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var run = session.RunAsync(default);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
await Task.Delay(1000);
upstream.Push(new JsonObject { ["type"] = "input_audio_buffer.speech_started" });
await Task.Delay(1500);
sink.Envelopes.Any(e => (string?)e["type"] == "session_ended").Should().BeFalse();
await run.WaitAsync(TimeSpan.FromSeconds(4));
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
((string?)ended["reason"]).Should().Be("idle");
}
}
@@ -0,0 +1,92 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionLifecycleTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task Open_emits_session_update_then_session_started_then_session_ended_on_cancel()
{
using var scope = _factory.Services.CreateScope();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var registry = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings(
Model: "gpt-4o-realtime-preview",
Voice: "alloy",
SystemPrompt: "test",
IdleTimeoutSeconds: 30,
EnabledTools: new HashSet<string> { "get_current_time" });
var session = new RealtimeSession(
deviceId: Guid.NewGuid(),
upstream, sink, log, registry, cfg,
channel: new FakeDeviceChannel(),
logger: NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var cts = new CancellationTokenSource();
var runTask = session.RunAsync(cts.Token);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
cts.Cancel();
await runTask;
upstream.Sent.Select(e => (string?)e["type"]).Should().StartWith("session.update");
sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_started");
sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_ended");
}
}
internal class NullLogger : Microsoft.Extensions.Logging.ILogger
{
public static readonly NullLogger Instance = new();
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(Microsoft.Extensions.Logging.LogLevel logLevel) => false;
public void Log<TState>(Microsoft.Extensions.Logging.LogLevel l, Microsoft.Extensions.Logging.EventId e,
TState s, Exception? ex, Func<TState, Exception?, string> f) { }
}
internal class RecordingSink : ISessionOutput
{
public List<JsonObject> Envelopes { get; } = new();
public List<byte[]> Binary { get; } = new();
public Task WriteEnvelopeAsync(JsonObject e, CancellationToken ct)
{
lock (Envelopes) Envelopes.Add(e);
_gate.Release();
return Task.CompletedTask;
}
public Task WriteBinaryAsync(ReadOnlyMemory<byte> b, CancellationToken ct)
{
Binary.Add(b.ToArray());
return Task.CompletedTask;
}
private readonly SemaphoreSlim _gate = new(0, int.MaxValue);
public async Task WaitForAsync(string type, TimeSpan timeout)
{
var deadline = DateTime.UtcNow + timeout;
while (DateTime.UtcNow < deadline)
{
lock (Envelopes)
{
if (Envelopes.Any(e => (string?)e["type"] == type)) return;
}
await _gate.WaitAsync(timeout);
}
throw new TimeoutException($"never saw {type}");
}
}
@@ -0,0 +1,67 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionToolsTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task Get_current_time_call_returns_function_call_output_and_persists_tool_turn()
{
using var scope = _factory.Services.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "get_current_time" });
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var cts = new CancellationTokenSource();
var run = session.RunAsync(cts.Token);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
upstream.Push(new JsonObject
{
["type"] = "response.function_call_arguments.done",
["call_id"] = "call_abc",
["name"] = "get_current_time",
["arguments"] = "{}",
});
upstream.Push(new JsonObject { ["type"] = "response.done" });
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(2);
while (DateTime.UtcNow < deadline && !upstream.Sent.Any(e =>
(string?)e["type"] == "conversation.item.create"))
{
await Task.Delay(20);
}
var item = upstream.Sent.First(e => (string?)e["type"] == "conversation.item.create");
((string?)item["item"]!["type"]).Should().Be("function_call_output");
((string?)item["item"]!["call_id"]).Should().Be("call_abc");
((string?)item["item"]!["output"]).Should().Contain("\"now\"");
upstream.Sent.Should().Contain(e => (string?)e["type"] == "response.create");
var convoId = session.ConversationId!.Value;
var toolTurn = await db.Turns.FirstAsync(t =>
t.ConversationId == convoId && t.Role == TurnRole.Tool);
toolTurn.ToolName.Should().Be("get_current_time");
toolTurn.ToolResultJson.Should().Contain("\"now\"");
cts.Cancel();
await run;
}
}
@@ -0,0 +1,60 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;
namespace backend.tests;
public class RealtimeSessionTranscriptTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
private readonly ApiFactory _factory = factory;
[Fact]
public async Task User_and_assistant_transcripts_are_persisted_and_response_done_emits_assistant_done()
{
using var scope = _factory.Services.CreateScope();
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
var reg = new ToolRegistry(Array.Empty<ITool>());
var upstream = new ScriptedRealtimeUpstream();
var sink = new RecordingSink();
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
new FakeDeviceChannel(), NullLogger.Instance);
upstream.Push(new JsonObject { ["type"] = "session.updated" });
var cts = new CancellationTokenSource();
var run = session.RunAsync(cts.Token);
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
upstream.Push(new JsonObject
{
["type"] = "conversation.item.input_audio_transcription.completed",
["transcript"] = "hello",
});
upstream.Push(new JsonObject
{
["type"] = "response.audio_transcript.done",
["transcript"] = "hi there",
});
upstream.Push(new JsonObject { ["type"] = "response.done" });
await sink.WaitForAsync("assistant_done", TimeSpan.FromSeconds(2));
var convoId = session.ConversationId!.Value;
var turns = await db.Turns
.Where(t => t.ConversationId == convoId)
.OrderBy(t => t.Index).ToListAsync();
turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant);
turns[0].Text.Should().Be("hello");
turns[1].Text.Should().Be("hi there");
cts.Cancel();
await run;
}
}
+55
View File
@@ -0,0 +1,55 @@
using System.Text.Json.Nodes;
using System.Threading.Channels;
using backend.Realtime;
namespace backend.tests;
public class ScriptedRealtimeUpstream : IRealtimeUpstream
{
private readonly Channel<JsonObject> _toRelay =
Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleReader = true });
private readonly Channel<JsonObject> _fromRelay =
Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleWriter = true });
public IList<JsonObject> Sent { get; } = new List<JsonObject>();
public bool IsClosed => _toRelay.Reader.Completion.IsCompleted;
public string? CloseReason => IsClosed ? "scripted upstream closed" : null;
public Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
{
Sent.Add(envelope);
return _fromRelay.Writer.WriteAsync(envelope, ct).AsTask();
}
public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
{
try
{
return await _toRelay.Reader.ReadAsync(ct);
}
catch (ChannelClosedException)
{
return null;
}
}
public void Push(JsonObject evt) => _toRelay.Writer.TryWrite(evt);
public void CloseUpstream() => _toRelay.Writer.TryComplete();
public async Task<JsonObject> WaitForSentAsync(string type, CancellationToken ct)
{
await foreach (var item in _fromRelay.Reader.ReadAllAsync(ct))
{
if ((string?)item["type"] == type) return item;
}
throw new TimeoutException($"never saw outbound {type}");
}
public ValueTask DisposeAsync()
{
_toRelay.Writer.TryComplete();
_fromRelay.Writer.TryComplete();
return ValueTask.CompletedTask;
}
}
+52
View File
@@ -0,0 +1,52 @@
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class SetVolumeToolTests
{
[Fact]
public async Task Forwards_call_to_pi_and_returns_its_result()
{
var fake = new RecordingChannel(
JsonDocument.Parse("""{"ok":true}""").RootElement);
var tool = new SetVolumeTool();
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);
var args = JsonDocument.Parse("""{"level":42}""").RootElement;
var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);
fake.LastName.Should().Be("set_volume");
fake.LastArgs.GetProperty("level").GetInt32().Should().Be(42);
res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
}
[Fact]
public async Task Rejects_out_of_range_level_without_calling_pi()
{
var fake = new RecordingChannel(JsonDocument.Parse("""{"ok":true}""").RootElement);
var tool = new SetVolumeTool();
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);
var args = JsonDocument.Parse("""{"level":150}""").RootElement;
var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);
fake.LastName.Should().BeNull();
res.Output.GetProperty("ok").GetBoolean().Should().BeFalse();
res.Output.GetProperty("error").GetString().Should().Contain("0..100");
}
private class RecordingChannel(JsonElement reply) : IDeviceChannel
{
public string? LastName { get; private set; }
public JsonElement LastArgs { get; private set; }
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
{
LastName = name;
LastArgs = args;
return Task.FromResult(reply);
}
}
}
+29
View File
@@ -0,0 +1,29 @@
using backend.Tools;
using FluentAssertions;
using Xunit;
namespace backend.tests;
public class ToolRegistryTests
{
private readonly ITool[] _all = new ITool[]
{
new GetCurrentTimeTool(),
};
[Fact]
public void Get_returns_tool_by_exact_name()
{
var reg = new ToolRegistry(_all);
reg.Get("get_current_time").Should().NotBeNull();
reg.Get("does_not_exist").Should().BeNull();
}
[Fact]
public void Filter_returns_only_enabled_tools_preserving_order()
{
var reg = new ToolRegistry(_all);
var enabled = new HashSet<string> { "get_current_time", "ghost" };
reg.EnabledFor(enabled).Select(t => t.Name).Should().Equal("get_current_time");
}
}
+27
View File
@@ -0,0 +1,27 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="coverlet.collector" Version="6.0.2" />
<PackageReference Include="FluentAssertions" Version="8.10.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="9.0.8" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />
</ItemGroup>
<ItemGroup>
<Using Include="Xunit" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\backend\backend.csproj" />
</ItemGroup>
</Project>
+7
View File
@@ -0,0 +1,7 @@
using Microsoft.AspNetCore.Identity;
namespace backend.Auth;
public class ApplicationUser : IdentityUser<Guid>
{
}
+62
View File
@@ -0,0 +1,62 @@
using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Mvc;
namespace backend.Auth;
public record RegisterRequest(string Email, string Password);
public record LoginRequest(string Email, string Password);
public static class AuthEndpoints
{
public static IEndpointRouteBuilder MapAuthEndpoints(this IEndpointRouteBuilder app)
{
var grp = app.MapGroup("/api/auth");
grp.MapPost("/register", async (
[FromBody] RegisterRequest req,
UserManager<ApplicationUser> users) =>
{
var user = new ApplicationUser { UserName = req.Email, Email = req.Email };
var create = await users.CreateAsync(user, req.Password);
if (!create.Succeeded)
return Results.BadRequest(new { errors = create.Errors.Select(e => e.Description) });
var anyAdmin = await users.GetUsersInRoleAsync(Roles.Admin);
var role = anyAdmin.Count == 0 ? Roles.Admin : Roles.User;
await users.AddToRoleAsync(user, role);
return Results.Ok(new { id = user.Id, email = user.Email, role });
});
grp.MapPost("/login", async (
[FromBody] LoginRequest req,
SignInManager<ApplicationUser> signIn) =>
{
var res = await signIn.PasswordSignInAsync(req.Email, req.Password,
isPersistent: true, lockoutOnFailure: false);
return res.Succeeded ? Results.Ok() : Results.Unauthorized();
});
grp.MapPost("/logout", async (SignInManager<ApplicationUser> signIn) =>
{
await signIn.SignOutAsync();
return Results.Ok();
}).RequireAuthorization();
grp.MapGet("/me", async (
UserManager<ApplicationUser> users,
HttpContext http) =>
{
var user = await users.GetUserAsync(http.User);
if (user is null) return Results.Unauthorized();
var rolesList = await users.GetRolesAsync(user);
return Results.Ok(new
{
id = user.Id,
email = user.Email,
isAdmin = rolesList.Contains(Roles.Admin),
});
}).RequireAuthorization();
return app;
}
}
+51
View File
@@ -0,0 +1,51 @@
using backend.Data;
using Microsoft.AspNetCore.Identity;
namespace backend.Auth;
public static class IdentitySetup
{
public static IServiceCollection AddAppIdentity(this IServiceCollection services)
{
services.AddIdentity<ApplicationUser, IdentityRole<Guid>>(opt =>
{
opt.Password.RequiredLength = 8;
opt.Password.RequireNonAlphanumeric = false;
opt.User.RequireUniqueEmail = true;
opt.SignIn.RequireConfirmedAccount = false;
})
.AddEntityFrameworkStores<AppDbContext>()
.AddDefaultTokenProviders();
services.ConfigureApplicationCookie(opt =>
{
opt.Cookie.HttpOnly = true;
opt.Cookie.SameSite = SameSiteMode.Lax;
opt.Cookie.SecurePolicy = CookieSecurePolicy.SameAsRequest;
opt.ExpireTimeSpan = TimeSpan.FromDays(30);
opt.SlidingExpiration = true;
opt.Events.OnRedirectToLogin = ctx =>
{
ctx.Response.StatusCode = 401;
return Task.CompletedTask;
};
opt.Events.OnRedirectToAccessDenied = ctx =>
{
ctx.Response.StatusCode = 403;
return Task.CompletedTask;
};
});
return services;
}
public static async Task SeedRolesAsync(IServiceProvider sp)
{
var roleMgr = sp.GetRequiredService<RoleManager<IdentityRole<Guid>>>();
foreach (var name in Roles.All)
{
if (!await roleMgr.RoleExistsAsync(name))
await roleMgr.CreateAsync(new IdentityRole<Guid>(name));
}
}
}
+9
View File
@@ -0,0 +1,9 @@
namespace backend.Auth;
public static class Roles
{
public const string Admin = "Admin";
public const string User = "User";
public static readonly string[] All = { Admin, User };
}
+19
View File
@@ -0,0 +1,19 @@
namespace backend.Conversations;
public enum EndReason
{
Unknown = 0,
Tool = 1,
Idle = 2,
Error = 3,
}
public class Conversation
{
public Guid Id { get; set; } = Guid.NewGuid();
public Guid DeviceId { get; set; }
public DateTimeOffset StartedAt { get; set; }
public DateTimeOffset? EndedAt { get; set; }
public EndReason EndReason { get; set; } = EndReason.Unknown;
public List<Turn> Turns { get; set; } = new();
}
+63
View File
@@ -0,0 +1,63 @@
using backend.Data;
using Microsoft.EntityFrameworkCore;
namespace backend.Conversations;
public class ConversationLog(AppDbContext db)
{
public async Task<Conversation> StartAsync(Guid deviceId, CancellationToken ct)
{
var c = new Conversation
{
DeviceId = deviceId,
StartedAt = DateTimeOffset.UtcNow,
};
db.Conversations.Add(c);
await db.SaveChangesAsync(ct);
return c;
}
public Task AppendUserAsync(Guid conversationId, string text, CancellationToken ct)
=> AppendAsync(conversationId, TurnRole.User, text: text, ct: ct);
public Task AppendAssistantAsync(Guid conversationId, string text, CancellationToken ct)
=> AppendAsync(conversationId, TurnRole.Assistant, text: text, ct: ct);
public Task AppendToolAsync(
Guid conversationId, string toolName, string argsJson, string resultJson, CancellationToken ct)
=> AppendAsync(conversationId, TurnRole.Tool,
toolName: toolName, argsJson: argsJson, resultJson: resultJson, ct: ct);
public async Task EndAsync(Guid conversationId, EndReason reason, CancellationToken ct)
{
var row = await db.Conversations.FirstAsync(c => c.Id == conversationId, ct);
row.EndedAt = DateTimeOffset.UtcNow;
row.EndReason = reason;
await db.SaveChangesAsync(ct);
}
private async Task AppendAsync(
Guid conversationId, TurnRole role,
string? text = null, string? toolName = null,
string? argsJson = null, string? resultJson = null,
CancellationToken ct = default)
{
var next = await db.Turns
.Where(t => t.ConversationId == conversationId)
.Select(t => (int?)t.Index)
.MaxAsync(ct) ?? -1;
db.Turns.Add(new Turn
{
ConversationId = conversationId,
Index = next + 1,
Role = role,
Text = text,
ToolName = toolName,
ToolArgsJson = argsJson,
ToolResultJson = resultJson,
CreatedAt = DateTimeOffset.UtcNow,
});
await db.SaveChangesAsync(ct);
}
}
+21
View File
@@ -0,0 +1,21 @@
namespace backend.Conversations;
public enum TurnRole
{
User = 0,
Assistant = 1,
Tool = 2,
}
public class Turn
{
public Guid Id { get; set; } = Guid.NewGuid();
public Guid ConversationId { get; set; }
public int Index { get; set; }
public TurnRole Role { get; set; }
public string? Text { get; set; }
public string? ToolName { get; set; }
public string? ToolArgsJson { get; set; }
public string? ToolResultJson { get; set; }
public DateTimeOffset CreatedAt { get; set; }
}
+51
View File
@@ -0,0 +1,51 @@
using backend.Auth;
using backend.Conversations;
using backend.Devices;
using backend.Pairing;
using backend.Settings;
using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;
namespace backend.Data;
public class AppDbContext(DbContextOptions<AppDbContext> options)
: IdentityDbContext<ApplicationUser, IdentityRole<Guid>, Guid>(options)
{
public DbSet<Device> Devices => Set<Device>();
public DbSet<DeviceConfig> DeviceConfigs => Set<DeviceConfig>();
public DbSet<SystemSettings> SystemSettings => Set<SystemSettings>();
public DbSet<PairingCode> PairingCodes => Set<PairingCode>();
public DbSet<Conversation> Conversations => Set<Conversation>();
public DbSet<Turn> Turns => Set<Turn>();
protected override void OnModelCreating(ModelBuilder b)
{
base.OnModelCreating(b);
b.Entity<Device>()
.HasOne<ApplicationUser>()
.WithMany()
.HasForeignKey(d => d.OwnerUserId)
.OnDelete(DeleteBehavior.Cascade);
b.Entity<Device>().HasIndex(d => d.TokenHash).IsUnique();
b.Entity<Device>()
.HasOne(d => d.Config)
.WithOne()
.HasForeignKey<DeviceConfig>(c => c.DeviceId)
.OnDelete(DeleteBehavior.Cascade);
b.Entity<DeviceConfig>().HasKey(c => c.DeviceId);
b.Entity<SystemSettings>().HasKey(s => s.Id);
b.Entity<SystemSettings>().Property(s => s.Id).ValueGeneratedNever();
b.Entity<PairingCode>().HasKey(p => p.Code);
b.Entity<PairingCode>().HasIndex(p => p.ExpiresAt);
b.Entity<Conversation>().HasIndex(c => c.DeviceId);
b.Entity<Conversation>()
.HasMany(c => c.Turns)
.WithOne()
.HasForeignKey(t => t.ConversationId)
.OnDelete(DeleteBehavior.Cascade);
b.Entity<Turn>().HasIndex(t => new { t.ConversationId, t.Index }).IsUnique();
}
}
+112
View File
@@ -0,0 +1,112 @@
using System.Collections.Concurrent;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Realtime;
using backend.Tools;
namespace backend.DeviceHub;
public interface IDeviceClient
{
Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope;
Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
}
public class ActiveDevice : IDeviceChannel, IDeviceClient, ISessionOutput, IAsyncDisposable
{
public Guid DeviceId { get; }
public WebSocket Ws => _ws;
public RealtimeSession? CurrentSession { get; set; }
public CancellationTokenSource? SessionCts { get; set; }
private readonly WebSocket _ws;
private readonly SemaphoreSlim _sendLock = new(1, 1);
private readonly ConcurrentDictionary<string, TaskCompletionSource<JsonElement>> _pendingTools = new();
public ActiveDevice(Guid deviceId, WebSocket ws)
{
DeviceId = deviceId;
_ws = ws;
}
public async Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
{
var json = JsonSerializer.Serialize(envelope, envelope.GetType());
await _sendLock.WaitAsync(ct);
try
{
await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
WebSocketMessageType.Text, endOfMessage: true, ct);
}
finally { _sendLock.Release(); }
}
public async Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
{
await _sendLock.WaitAsync(ct);
try
{
await _ws.SendAsync(bytes, WebSocketMessageType.Binary, endOfMessage: true, ct);
}
finally { _sendLock.Release(); }
}
public Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct)
{
var json = envelope.ToJsonString();
return WriteRawTextAsync(json, ct);
}
private async Task WriteRawTextAsync(string json, CancellationToken ct)
{
await _sendLock.WaitAsync(ct);
try
{
await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
WebSocketMessageType.Text, endOfMessage: true, ct);
}
finally { _sendLock.Release(); }
}
public Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
=> SendBinaryAsync(bytes, ct);
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
{
var callId = "srv_" + Guid.NewGuid().ToString("N")[..12];
var tcs = new TaskCompletionSource<JsonElement>(TaskCreationOptions.RunContinuationsAsynchronously);
_pendingTools[callId] = tcs;
var envelope = new ToolCallEnvelope("tool_call", callId, name, args);
_ = SendEnvelopeAsync(envelope, ct);
ct.Register(() => tcs.TrySetCanceled());
return tcs.Task;
}
public void CompleteToolResult(ToolResultEnvelope env)
{
if (!_pendingTools.TryRemove(env.CallId, out var tcs)) return;
if (env.Ok && env.Result is JsonElement r) tcs.TrySetResult(r);
else
{
var msg = (env.Error ?? "unknown").Replace("\\", "\\\\").Replace("\"", "\\\"");
var doc = JsonDocument.Parse($$"""{"ok":false,"error":"{{msg}}"}""");
tcs.TrySetResult(doc.RootElement.Clone());
}
}
public async ValueTask DisposeAsync()
{
try
{
if (_ws.State == WebSocketState.Open)
await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None);
}
catch { }
_ws.Dispose();
_sendLock.Dispose();
}
}
+21
View File
@@ -0,0 +1,21 @@
using backend.Data;
using backend.Devices;
using Microsoft.EntityFrameworkCore;
namespace backend.DeviceHub;
public class DeviceAuth(AppDbContext db, DeviceTokenService tokens)
{
public async Task<Device?> AuthenticateAsync(string? authHeader, CancellationToken ct)
{
if (string.IsNullOrEmpty(authHeader)) return null;
const string prefix = "Bearer ";
if (!authHeader.StartsWith(prefix, StringComparison.Ordinal)) return null;
var token = authHeader[prefix.Length..].Trim();
if (string.IsNullOrEmpty(token)) return null;
var hash = tokens.Hash(token);
var device = await db.Devices.Include(d => d.Config)
.FirstOrDefaultAsync(d => d.TokenHash == hash && !d.IsRevoked, ct);
return device;
}
}
+207
View File
@@ -0,0 +1,207 @@
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Devices;
using backend.Realtime;
using backend.Tools;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
namespace backend.DeviceHub;
public static class DeviceHubEndpoint
{
public static IEndpointRouteBuilder MapDeviceHub(this IEndpointRouteBuilder app)
{
app.Map("/device", async (
HttpContext ctx,
DeviceAuth auth,
DeviceRegistry reg,
AppDbContext db,
RealtimeSessionFactory sessions,
IServiceScopeFactory scopes) =>
{
if (!ctx.WebSockets.IsWebSocketRequest)
{
ctx.Response.StatusCode = StatusCodes.Status400BadRequest;
return;
}
var device = await auth.AuthenticateAsync(
ctx.Request.Headers.Authorization.ToString(), ctx.RequestAborted);
if (device is null)
{
ctx.Response.StatusCode = StatusCodes.Status401Unauthorized;
return;
}
using var ws = await ctx.WebSockets.AcceptWebSocketAsync();
await using var active = new ActiveDevice(device.Id, ws);
if (!reg.TryRegister(active))
{
await ws.CloseAsync(WebSocketCloseStatus.PolicyViolation,
"device already connected", ctx.RequestAborted);
return;
}
try
{
await HandleAsync(active, device, db, sessions, scopes, ctx.RequestAborted);
}
finally
{
reg.Unregister(device.Id);
}
});
return app;
}
private static async Task HandleAsync(
ActiveDevice active, Device device, AppDbContext db,
RealtimeSessionFactory sessions, IServiceScopeFactory scopes,
CancellationToken ct)
{
var buffer = new byte[8192];
var ws = active.Ws;
while (ws.State == WebSocketState.Open && !ct.IsCancellationRequested)
{
using var msg = new MemoryStream();
WebSocketReceiveResult res;
while (true)
{
res = await ws.ReceiveAsync(buffer, ct);
if (res.MessageType == WebSocketMessageType.Close) return;
msg.Write(buffer, 0, res.Count);
if (res.EndOfMessage) break;
}
if (res.MessageType == WebSocketMessageType.Binary)
{
if (msg.Length == 3840 && active.CurrentSession is not null)
{
await active.CurrentSession.WriteUplinkAsync(msg.ToArray(), ct);
}
continue;
}
var text = Encoding.UTF8.GetString(msg.ToArray());
using var doc = JsonDocument.Parse(text);
var type = doc.RootElement.GetProperty("type").GetString();
switch (type)
{
case "hello":
await SendHelloAckAsync(active, device, ct);
break;
case "ping":
device.LastSeenAt = DateTimeOffset.UtcNow;
await db.SaveChangesAsync(ct);
await active.SendEnvelopeAsync(new PongEnvelope("pong"), ct);
break;
case "wake":
if (active.CurrentSession is null)
await StartSessionAsync(active, device, sessions, scopes, ct);
break;
case "cancel":
active.SessionCts?.Cancel();
break;
case "playback_done":
break; // v1: client coordination only
case "tool_result":
var env = JsonSerializer.Deserialize<ToolResultEnvelope>(text)!;
active.CompleteToolResult(env);
break;
}
}
}
private static async Task StartSessionAsync(
ActiveDevice active, Device device,
RealtimeSessionFactory sessions, IServiceScopeFactory scopes,
CancellationToken parentCt)
{
var enabledTools = JsonSerializer.Deserialize<string[]>(device.Config?.EnabledToolsJson ?? "[]")
?? Array.Empty<string>();
// Promote any deprecated Beta-era model name to the GA name. Plan 4's
// admin UI will let users set a specific value; until then, anything
// that mentions the old preview slug routes to gpt-realtime.
var model = device.Config?.Model ?? "gpt-realtime";
if (model.StartsWith("gpt-4o-realtime-preview", StringComparison.Ordinal))
model = "gpt-realtime";
var cfg = new RealtimeSettings(
Model: model,
Voice: device.Config?.Voice ?? "alloy",
SystemPrompt: device.Config?.SystemPrompt ?? "",
IdleTimeoutSeconds: device.Config?.IdleTimeoutSeconds ?? 30,
EnabledTools: new HashSet<string>(enabledTools));
active.SessionCts = CancellationTokenSource.CreateLinkedTokenSource(parentCt);
var ct = active.SessionCts.Token;
var sessionScope = scopes.CreateAsyncScope();
var scopedSessions = sessionScope.ServiceProvider.GetRequiredService<RealtimeSessionFactory>();
RealtimeSession session;
try
{
session = await scopedSessions.CreateAsync(
device.Id, cfg, active, active, ct);
}
catch (Exception ex)
{
await active.SendEnvelopeAsync(
new ErrorEnvelope("error", "upstream", ex.Message, true), parentCt);
await sessionScope.DisposeAsync();
return;
}
active.CurrentSession = session;
_ = Task.Run(async () =>
{
try { await session.RunAsync(ct); }
finally
{
active.CurrentSession = null;
active.SessionCts?.Dispose();
active.SessionCts = null;
await sessionScope.DisposeAsync();
}
}, ct);
}
// Bump on every backend change so we can verify a Coolify deploy landed
// by looking at the hello_ack config.server_version field in the Pi journal.
public const string ServerVersion = "plan3-debug-2026-06-12-09-closeprobe";
private static async Task SendHelloAckAsync(ActiveDevice active, Device device, CancellationToken ct)
{
var enabled = device.Config?.EnabledToolsJson ?? "[]";
var config = JsonNode.Parse($$"""
{
"voice": "{{Escape(device.Config?.Voice)}}",
"model": "{{Escape(device.Config?.Model)}}",
"system_prompt": "{{Escape(device.Config?.SystemPrompt)}}",
"idle_timeout_seconds": {{device.Config?.IdleTimeoutSeconds ?? 30}},
"enabled_tools": {{enabled}},
"server_version": "{{ServerVersion}}"
}
""")!.AsObject();
await active.WriteEnvelopeAsync(new JsonObject
{
["type"] = "hello_ack",
["config"] = config,
}, ct);
}
private static string Escape(string? s) =>
(s ?? "").Replace("\\", "\\\\").Replace("\"", "\\\"");
}
+21
View File
@@ -0,0 +1,21 @@
using System.Collections.Concurrent;
namespace backend.DeviceHub;
public class DeviceRegistry
{
private readonly ConcurrentDictionary<Guid, ActiveDevice> _online = new();
public bool TryRegister(ActiveDevice dev)
=> _online.TryAdd(dev.DeviceId, dev);
public bool Unregister(Guid deviceId)
=> _online.TryRemove(deviceId, out _);
public ActiveDevice? Get(Guid deviceId)
=> _online.TryGetValue(deviceId, out var d) ? d : null;
public bool IsOnline(Guid deviceId) => _online.ContainsKey(deviceId);
public int OnlineCount => _online.Count;
}
+42
View File
@@ -0,0 +1,42 @@
using System.Text.Json;
using System.Text.Json.Serialization;
namespace backend.DeviceHub;
public record HubEnvelope([property: JsonPropertyName("type")] string Type);
public record HelloEnvelope(string Type,
[property: JsonPropertyName("device_id")] Guid DeviceId,
[property: JsonPropertyName("client_version")] string ClientVersion) : HubEnvelope(Type);
public record HelloAckEnvelope(string Type,
[property: JsonPropertyName("config")] JsonElement Config) : HubEnvelope(Type);
public record PongEnvelope(string Type) : HubEnvelope(Type);
public record WakeEnvelope(string Type,
[property: JsonPropertyName("at")] long? At) : HubEnvelope(Type);
public record SessionStartedEnvelope(string Type,
[property: JsonPropertyName("conversation_id")] Guid ConversationId) : HubEnvelope(Type);
public record SessionEndedEnvelope(string Type,
[property: JsonPropertyName("reason")] string Reason) : HubEnvelope(Type);
public record AssistantDoneEnvelope(string Type) : HubEnvelope(Type);
public record ToolCallEnvelope(string Type,
[property: JsonPropertyName("call_id")] string CallId,
[property: JsonPropertyName("name")] string Name,
[property: JsonPropertyName("arguments")] JsonElement Arguments) : HubEnvelope(Type);
public record ToolResultEnvelope(string Type,
[property: JsonPropertyName("call_id")] string CallId,
[property: JsonPropertyName("ok")] bool Ok,
[property: JsonPropertyName("result")] JsonElement? Result,
[property: JsonPropertyName("error")] string? Error) : HubEnvelope(Type);
public record ErrorEnvelope(string Type,
[property: JsonPropertyName("code")] string Code,
[property: JsonPropertyName("message")] string Message,
[property: JsonPropertyName("fatal")] bool? Fatal) : HubEnvelope(Type);
+13
View File
@@ -0,0 +1,13 @@
namespace backend.Devices;
public class Device
{
public Guid Id { get; set; } = Guid.NewGuid();
public Guid OwnerUserId { get; set; }
public string Name { get; set; } = "";
public DateTimeOffset PairedAt { get; set; }
public DateTimeOffset? LastSeenAt { get; set; }
public string TokenHash { get; set; } = "";
public bool IsRevoked { get; set; }
public DeviceConfig? Config { get; set; }
}
+11
View File
@@ -0,0 +1,11 @@
namespace backend.Devices;
public class DeviceConfig
{
public Guid DeviceId { get; set; }
public string SystemPrompt { get; set; } = "";
public string Voice { get; set; } = "";
public string Model { get; set; } = "";
public int IdleTimeoutSeconds { get; set; } = 30;
public string EnabledToolsJson { get; set; } = "[\"end_session\",\"get_current_time\",\"set_volume\"]";
}
+36
View File
@@ -0,0 +1,36 @@
using System.Security.Cryptography;
using System.Text;
namespace backend.Devices;
public class DeviceTokenService
{
public (string Plaintext, string Hash) Generate()
{
Span<byte> bytes = stackalloc byte[32];
RandomNumberGenerator.Fill(bytes);
var plaintext = Base64UrlEncode(bytes);
return (plaintext, Hash(plaintext));
}
public string Hash(string plaintext)
{
Span<byte> hash = stackalloc byte[32];
SHA256.HashData(Encoding.UTF8.GetBytes(plaintext), hash);
return Convert.ToHexString(hash);
}
public bool Verify(string plaintext, string expectedHash)
{
var actual = Hash(plaintext);
return CryptographicOperations.FixedTimeEquals(
Encoding.UTF8.GetBytes(actual),
Encoding.UTF8.GetBytes(expectedHash));
}
private static string Base64UrlEncode(ReadOnlySpan<byte> bytes)
=> Convert.ToBase64String(bytes)
.TrimEnd('=')
.Replace('+', '-')
.Replace('/', '_');
}
+83
View File
@@ -0,0 +1,83 @@
using System.Security.Claims;
using backend.Data;
using Microsoft.AspNetCore.Mvc;
using Microsoft.EntityFrameworkCore;
namespace backend.Devices;
public record RenameRequest(string Name);
public static class DevicesEndpoints
{
public static IEndpointRouteBuilder MapDevicesEndpoints(this IEndpointRouteBuilder app)
{
var grp = app.MapGroup("/api/devices").RequireAuthorization();
grp.MapGet("/", async (AppDbContext db, HttpContext http, CancellationToken ct) =>
{
var uid = GetUserId(http);
var devs = await db.Devices
.Where(d => d.OwnerUserId == uid)
.Select(d => new
{
id = d.Id,
name = d.Name,
pairedAt = d.PairedAt,
lastSeenAt = d.LastSeenAt,
isRevoked = d.IsRevoked,
})
.ToListAsync(ct);
return Results.Ok(devs);
});
grp.MapGet("/{id:guid}", async (Guid id, AppDbContext db, HttpContext http, CancellationToken ct) =>
{
var uid = GetUserId(http);
var d = await db.Devices.Include(x => x.Config)
.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
return d is null
? Results.NotFound()
: Results.Ok(new
{
id = d.Id,
name = d.Name,
pairedAt = d.PairedAt,
lastSeenAt = d.LastSeenAt,
isRevoked = d.IsRevoked,
config = d.Config,
});
});
grp.MapPatch("/{id:guid}", async (
Guid id, [FromBody] RenameRequest req,
AppDbContext db, HttpContext http, CancellationToken ct) =>
{
var uid = GetUserId(http);
var d = await db.Devices.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
if (d is null) return Results.NotFound();
if (string.IsNullOrWhiteSpace(req.Name))
return Results.BadRequest(new { error = "name is required" });
d.Name = req.Name.Trim();
await db.SaveChangesAsync(ct);
return Results.Ok();
});
grp.MapDelete("/{id:guid}", async (Guid id, AppDbContext db, HttpContext http, CancellationToken ct) =>
{
var uid = GetUserId(http);
var d = await db.Devices.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
if (d is null) return Results.NotFound();
d.IsRevoked = true;
await db.SaveChangesAsync(ct);
return Results.Ok();
});
return app;
}
private static Guid GetUserId(HttpContext http)
{
var s = http.User.FindFirstValue(ClaimTypes.NameIdentifier);
return Guid.TryParse(s, out var g) ? g : Guid.Empty;
}
}
+28
View File
@@ -0,0 +1,28 @@
using Microsoft.Extensions.Configuration;
namespace backend.Install;
public static class InstallEndpoints
{
public static IEndpointRouteBuilder MapInstallEndpoints(this IEndpointRouteBuilder app)
{
app.MapGet("/install.sh", (HttpContext http, InstallScriptBuilder builder) =>
{
var scheme = http.Request.Scheme;
var host = http.Request.Host.ToString();
var script = builder.Build($"{scheme}://{host}");
return Results.Text(script, "text/x-shellscript");
});
app.MapGet("/client.tar.gz", (IConfiguration cfg, IWebHostEnvironment env) =>
{
var path = cfg["Install:ClientTarballPath"]
?? Path.Combine(env.WebRootPath ?? "wwwroot", "client.tar.gz");
if (!File.Exists(path))
return Results.NotFound(new { error = "client.tar.gz not found at " + path });
return Results.File(path, "application/gzip", "client.tar.gz");
});
return app;
}
}
+70
View File
@@ -0,0 +1,70 @@
namespace backend.Install;
public class InstallScriptBuilder
{
public string Build(string backendUrl)
{
return $$"""
#!/usr/bin/env bash
# Smart Assistant installer (idempotent). Re-run to upgrade.
set -euo pipefail
BACKEND_URL="{{backendUrl}}"
ASSISTANT_HOME="$HOME/assistant"
RESET=0
for arg in "$@"; do
case "$arg" in
--reset) RESET=1 ;;
esac
done
echo "Installing system deps (sudo password may be prompted)..."
sudo apt-get update -qq
sudo apt-get install -y --no-install-recommends python3-venv libportaudio2 alsa-utils
mkdir -p "$ASSISTANT_HOME/code" "$ASSISTANT_HOME/state"
if [ ! -d "$ASSISTANT_HOME/.venv" ]; then
python3 -m venv "$ASSISTANT_HOME/.venv"
fi
echo "Downloading client bundle from $BACKEND_URL ..."
curl -fsSL "$BACKEND_URL/client.tar.gz" | tar -xz -C "$ASSISTANT_HOME/code"
"$ASSISTANT_HOME/.venv/bin/pip" install --quiet -r "$ASSISTANT_HOME/code/requirements.txt"
"$ASSISTANT_HOME/.venv/bin/pip" install --quiet --no-deps -r "$ASSISTANT_HOME/code/requirements-nodeps.txt"
if [ "$RESET" -eq 1 ]; then
rm -f "$ASSISTANT_HOME/state/config.json"
fi
if [ ! -f "$ASSISTANT_HOME/state/config.json" ]; then
echo
read -r -p "Pairing code: " CODE
"$ASSISTANT_HOME/.venv/bin/python" -m client.pair --backend "$BACKEND_URL" --code "$CODE"
fi
mkdir -p "$HOME/.config/systemd/user"
cat > "$HOME/.config/systemd/user/assistant.service" <<UNIT
[Unit]
Description=Smart Assistant Pi client
After=network-online.target
[Service]
Environment=PA_ALSA_PLUGHW=1
ExecStart=$ASSISTANT_HOME/.venv/bin/python -m client.main
WorkingDirectory=$ASSISTANT_HOME/code
Restart=on-failure
RestartSec=5
[Install]
WantedBy=default.target
UNIT
loginctl enable-linger "$USER" 2>/dev/null || true
systemctl --user daemon-reload
systemctl --user enable --now assistant
echo "Done. systemctl --user status assistant"
""";
}
}
+24
View File
@@ -0,0 +1,24 @@
// <auto-generated />
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611181007_Initial")]
partial class Initial
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,22 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class Initial : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
}
}
}
+267
View File
@@ -0,0 +1,267 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611182529_AddIdentity")]
partial class AddIdentity
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,222 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class AddIdentity : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "AspNetRoles",
columns: table => new
{
Id = table.Column<Guid>(type: "TEXT", nullable: false),
Name = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
NormalizedName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
ConcurrencyStamp = table.Column<string>(type: "TEXT", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetRoles", x => x.Id);
});
migrationBuilder.CreateTable(
name: "AspNetUsers",
columns: table => new
{
Id = table.Column<Guid>(type: "TEXT", nullable: false),
UserName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
NormalizedUserName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
Email = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
NormalizedEmail = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
EmailConfirmed = table.Column<bool>(type: "INTEGER", nullable: false),
PasswordHash = table.Column<string>(type: "TEXT", nullable: true),
SecurityStamp = table.Column<string>(type: "TEXT", nullable: true),
ConcurrencyStamp = table.Column<string>(type: "TEXT", nullable: true),
PhoneNumber = table.Column<string>(type: "TEXT", nullable: true),
PhoneNumberConfirmed = table.Column<bool>(type: "INTEGER", nullable: false),
TwoFactorEnabled = table.Column<bool>(type: "INTEGER", nullable: false),
LockoutEnd = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
LockoutEnabled = table.Column<bool>(type: "INTEGER", nullable: false),
AccessFailedCount = table.Column<int>(type: "INTEGER", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetUsers", x => x.Id);
});
migrationBuilder.CreateTable(
name: "AspNetRoleClaims",
columns: table => new
{
Id = table.Column<int>(type: "INTEGER", nullable: false)
.Annotation("Sqlite:Autoincrement", true),
RoleId = table.Column<Guid>(type: "TEXT", nullable: false),
ClaimType = table.Column<string>(type: "TEXT", nullable: true),
ClaimValue = table.Column<string>(type: "TEXT", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetRoleClaims", x => x.Id);
table.ForeignKey(
name: "FK_AspNetRoleClaims_AspNetRoles_RoleId",
column: x => x.RoleId,
principalTable: "AspNetRoles",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateTable(
name: "AspNetUserClaims",
columns: table => new
{
Id = table.Column<int>(type: "INTEGER", nullable: false)
.Annotation("Sqlite:Autoincrement", true),
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
ClaimType = table.Column<string>(type: "TEXT", nullable: true),
ClaimValue = table.Column<string>(type: "TEXT", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetUserClaims", x => x.Id);
table.ForeignKey(
name: "FK_AspNetUserClaims_AspNetUsers_UserId",
column: x => x.UserId,
principalTable: "AspNetUsers",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateTable(
name: "AspNetUserLogins",
columns: table => new
{
LoginProvider = table.Column<string>(type: "TEXT", nullable: false),
ProviderKey = table.Column<string>(type: "TEXT", nullable: false),
ProviderDisplayName = table.Column<string>(type: "TEXT", nullable: true),
UserId = table.Column<Guid>(type: "TEXT", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetUserLogins", x => new { x.LoginProvider, x.ProviderKey });
table.ForeignKey(
name: "FK_AspNetUserLogins_AspNetUsers_UserId",
column: x => x.UserId,
principalTable: "AspNetUsers",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateTable(
name: "AspNetUserRoles",
columns: table => new
{
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
RoleId = table.Column<Guid>(type: "TEXT", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetUserRoles", x => new { x.UserId, x.RoleId });
table.ForeignKey(
name: "FK_AspNetUserRoles_AspNetRoles_RoleId",
column: x => x.RoleId,
principalTable: "AspNetRoles",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
table.ForeignKey(
name: "FK_AspNetUserRoles_AspNetUsers_UserId",
column: x => x.UserId,
principalTable: "AspNetUsers",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateTable(
name: "AspNetUserTokens",
columns: table => new
{
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
LoginProvider = table.Column<string>(type: "TEXT", nullable: false),
Name = table.Column<string>(type: "TEXT", nullable: false),
Value = table.Column<string>(type: "TEXT", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_AspNetUserTokens", x => new { x.UserId, x.LoginProvider, x.Name });
table.ForeignKey(
name: "FK_AspNetUserTokens_AspNetUsers_UserId",
column: x => x.UserId,
principalTable: "AspNetUsers",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "IX_AspNetRoleClaims_RoleId",
table: "AspNetRoleClaims",
column: "RoleId");
migrationBuilder.CreateIndex(
name: "RoleNameIndex",
table: "AspNetRoles",
column: "NormalizedName",
unique: true);
migrationBuilder.CreateIndex(
name: "IX_AspNetUserClaims_UserId",
table: "AspNetUserClaims",
column: "UserId");
migrationBuilder.CreateIndex(
name: "IX_AspNetUserLogins_UserId",
table: "AspNetUserLogins",
column: "UserId");
migrationBuilder.CreateIndex(
name: "IX_AspNetUserRoles_RoleId",
table: "AspNetUserRoles",
column: "RoleId");
migrationBuilder.CreateIndex(
name: "EmailIndex",
table: "AspNetUsers",
column: "NormalizedEmail");
migrationBuilder.CreateIndex(
name: "UserNameIndex",
table: "AspNetUsers",
column: "NormalizedUserName",
unique: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "AspNetRoleClaims");
migrationBuilder.DropTable(
name: "AspNetUserClaims");
migrationBuilder.DropTable(
name: "AspNetUserLogins");
migrationBuilder.DropTable(
name: "AspNetUserRoles");
migrationBuilder.DropTable(
name: "AspNetUserTokens");
migrationBuilder.DropTable(
name: "AspNetRoles");
migrationBuilder.DropTable(
name: "AspNetUsers");
}
}
}
@@ -0,0 +1,372 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611184053_AddDevicesAndSettings")]
partial class AddDevicesAndSettings
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<bool>("IsRevoked")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LastSeenAt")
.HasColumnType("TEXT");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("PairedAt")
.HasColumnType("TEXT");
b.Property<string>("TokenHash")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("OwnerUserId");
b.HasIndex("TokenHash")
.IsUnique();
b.ToTable("Devices");
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<string>("EnabledToolsJson")
.IsRequired()
.HasColumnType("TEXT");
b.Property<int>("IdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("Model")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("SystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("Voice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("DeviceId");
b.ToTable("DeviceConfigs");
});
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<int>("DefaultIdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("DefaultModel")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultSystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultVoice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.ToTable("SystemSettings");
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.HasOne("backend.Devices.Device", null)
.WithOne("Config")
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Navigation("Config");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,94 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class AddDevicesAndSettings : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "Devices",
columns: table => new
{
Id = table.Column<Guid>(type: "TEXT", nullable: false),
OwnerUserId = table.Column<Guid>(type: "TEXT", nullable: false),
Name = table.Column<string>(type: "TEXT", nullable: false),
PairedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
LastSeenAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
TokenHash = table.Column<string>(type: "TEXT", nullable: false),
IsRevoked = table.Column<bool>(type: "INTEGER", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_Devices", x => x.Id);
});
migrationBuilder.CreateTable(
name: "SystemSettings",
columns: table => new
{
Id = table.Column<int>(type: "INTEGER", nullable: false)
.Annotation("Sqlite:Autoincrement", true),
DefaultSystemPrompt = table.Column<string>(type: "TEXT", nullable: false),
DefaultVoice = table.Column<string>(type: "TEXT", nullable: false),
DefaultModel = table.Column<string>(type: "TEXT", nullable: false),
DefaultIdleTimeoutSeconds = table.Column<int>(type: "INTEGER", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_SystemSettings", x => x.Id);
});
migrationBuilder.CreateTable(
name: "DeviceConfigs",
columns: table => new
{
DeviceId = table.Column<Guid>(type: "TEXT", nullable: false),
SystemPrompt = table.Column<string>(type: "TEXT", nullable: false),
Voice = table.Column<string>(type: "TEXT", nullable: false),
Model = table.Column<string>(type: "TEXT", nullable: false),
IdleTimeoutSeconds = table.Column<int>(type: "INTEGER", nullable: false),
EnabledToolsJson = table.Column<string>(type: "TEXT", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_DeviceConfigs", x => x.DeviceId);
table.ForeignKey(
name: "FK_DeviceConfigs_Devices_DeviceId",
column: x => x.DeviceId,
principalTable: "Devices",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "IX_Devices_OwnerUserId",
table: "Devices",
column: "OwnerUserId");
migrationBuilder.CreateIndex(
name: "IX_Devices_TokenHash",
table: "Devices",
column: "TokenHash",
unique: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "DeviceConfigs");
migrationBuilder.DropTable(
name: "SystemSettings");
migrationBuilder.DropTable(
name: "Devices");
}
}
}
@@ -0,0 +1,380 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611184536_AddDeviceOwnerFkAndPinSettingsId")]
partial class AddDeviceOwnerFkAndPinSettingsId
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<bool>("IsRevoked")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LastSeenAt")
.HasColumnType("TEXT");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("PairedAt")
.HasColumnType("TEXT");
b.Property<string>("TokenHash")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("OwnerUserId");
b.HasIndex("TokenHash")
.IsUnique();
b.ToTable("Devices");
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<string>("EnabledToolsJson")
.IsRequired()
.HasColumnType("TEXT");
b.Property<int>("IdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("Model")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("SystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("Voice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("DeviceId");
b.ToTable("DeviceConfigs");
});
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
{
b.Property<int>("Id")
.HasColumnType("INTEGER");
b.Property<int>("DefaultIdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("DefaultModel")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultSystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultVoice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.ToTable("SystemSettings");
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("OwnerUserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.HasOne("backend.Devices.Device", null)
.WithOne("Config")
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Navigation("Config");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,48 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class AddDeviceOwnerFkAndPinSettingsId : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AlterColumn<int>(
name: "Id",
table: "SystemSettings",
type: "INTEGER",
nullable: false,
oldClrType: typeof(int),
oldType: "INTEGER")
.OldAnnotation("Sqlite:Autoincrement", true);
migrationBuilder.AddForeignKey(
name: "FK_Devices_AspNetUsers_OwnerUserId",
table: "Devices",
column: "OwnerUserId",
principalTable: "AspNetUsers",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropForeignKey(
name: "FK_Devices_AspNetUsers_OwnerUserId",
table: "Devices");
migrationBuilder.AlterColumn<int>(
name: "Id",
table: "SystemSettings",
type: "INTEGER",
nullable: false,
oldClrType: typeof(int),
oldType: "INTEGER")
.Annotation("Sqlite:Autoincrement", true);
}
}
}
@@ -0,0 +1,405 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611185048_AddPairingCodes")]
partial class AddPairingCodes
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<bool>("IsRevoked")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LastSeenAt")
.HasColumnType("TEXT");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("PairedAt")
.HasColumnType("TEXT");
b.Property<string>("TokenHash")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("OwnerUserId");
b.HasIndex("TokenHash")
.IsUnique();
b.ToTable("Devices");
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<string>("EnabledToolsJson")
.IsRequired()
.HasColumnType("TEXT");
b.Property<int>("IdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("Model")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("SystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("Voice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("DeviceId");
b.ToTable("DeviceConfigs");
});
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
{
b.Property<string>("Code")
.HasColumnType("TEXT");
b.Property<DateTimeOffset?>("ConsumedAt")
.HasColumnType("TEXT");
b.Property<string>("DeviceName")
.IsRequired()
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("ExpiresAt")
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.HasKey("Code");
b.HasIndex("ExpiresAt");
b.ToTable("PairingCodes");
});
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
{
b.Property<int>("Id")
.HasColumnType("INTEGER");
b.Property<int>("DefaultIdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("DefaultModel")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultSystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultVoice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.ToTable("SystemSettings");
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("OwnerUserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.HasOne("backend.Devices.Device", null)
.WithOne("Config")
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Navigation("Config");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,42 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class AddPairingCodes : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "PairingCodes",
columns: table => new
{
Code = table.Column<string>(type: "TEXT", nullable: false),
OwnerUserId = table.Column<Guid>(type: "TEXT", nullable: false),
DeviceName = table.Column<string>(type: "TEXT", nullable: false),
ExpiresAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
ConsumedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_PairingCodes", x => x.Code);
});
migrationBuilder.CreateIndex(
name: "IX_PairingCodes_ExpiresAt",
table: "PairingCodes",
column: "ExpiresAt");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "PairingCodes");
}
}
}
@@ -0,0 +1,482 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
[Migration("20260611210852_AddConversationsAndTurns")]
partial class AddConversationsAndTurns
{
/// <inheritdoc />
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("backend.Conversations.Conversation", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<int>("EndReason")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("EndedAt")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("StartedAt")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("DeviceId");
b.ToTable("Conversations");
});
modelBuilder.Entity("backend.Conversations.Turn", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<Guid>("ConversationId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("CreatedAt")
.HasColumnType("TEXT");
b.Property<int>("Index")
.HasColumnType("INTEGER");
b.Property<int>("Role")
.HasColumnType("INTEGER");
b.Property<string>("Text")
.HasColumnType("TEXT");
b.Property<string>("ToolArgsJson")
.HasColumnType("TEXT");
b.Property<string>("ToolName")
.HasColumnType("TEXT");
b.Property<string>("ToolResultJson")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("ConversationId", "Index")
.IsUnique();
b.ToTable("Turns");
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<bool>("IsRevoked")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LastSeenAt")
.HasColumnType("TEXT");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("PairedAt")
.HasColumnType("TEXT");
b.Property<string>("TokenHash")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("OwnerUserId");
b.HasIndex("TokenHash")
.IsUnique();
b.ToTable("Devices");
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<string>("EnabledToolsJson")
.IsRequired()
.HasColumnType("TEXT");
b.Property<int>("IdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("Model")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("SystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("Voice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("DeviceId");
b.ToTable("DeviceConfigs");
});
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
{
b.Property<string>("Code")
.HasColumnType("TEXT");
b.Property<DateTimeOffset?>("ConsumedAt")
.HasColumnType("TEXT");
b.Property<string>("DeviceName")
.IsRequired()
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("ExpiresAt")
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.HasKey("Code");
b.HasIndex("ExpiresAt");
b.ToTable("PairingCodes");
});
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
{
b.Property<int>("Id")
.HasColumnType("INTEGER");
b.Property<int>("DefaultIdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("DefaultModel")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultSystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultVoice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.ToTable("SystemSettings");
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Conversations.Turn", b =>
{
b.HasOne("backend.Conversations.Conversation", null)
.WithMany("Turns")
.HasForeignKey("ConversationId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("OwnerUserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.HasOne("backend.Devices.Device", null)
.WithOne("Config")
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Conversations.Conversation", b =>
{
b.Navigation("Turns");
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Navigation("Config");
});
#pragma warning restore 612, 618
}
}
}
@@ -0,0 +1,76 @@
using System;
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace backend.Migrations
{
/// <inheritdoc />
public partial class AddConversationsAndTurns : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "Conversations",
columns: table => new
{
Id = table.Column<Guid>(type: "TEXT", nullable: false),
DeviceId = table.Column<Guid>(type: "TEXT", nullable: false),
StartedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
EndedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
EndReason = table.Column<int>(type: "INTEGER", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_Conversations", x => x.Id);
});
migrationBuilder.CreateTable(
name: "Turns",
columns: table => new
{
Id = table.Column<Guid>(type: "TEXT", nullable: false),
ConversationId = table.Column<Guid>(type: "TEXT", nullable: false),
Index = table.Column<int>(type: "INTEGER", nullable: false),
Role = table.Column<int>(type: "INTEGER", nullable: false),
Text = table.Column<string>(type: "TEXT", nullable: true),
ToolName = table.Column<string>(type: "TEXT", nullable: true),
ToolArgsJson = table.Column<string>(type: "TEXT", nullable: true),
ToolResultJson = table.Column<string>(type: "TEXT", nullable: true),
CreatedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false)
},
constraints: table =>
{
table.PrimaryKey("PK_Turns", x => x.Id);
table.ForeignKey(
name: "FK_Turns_Conversations_ConversationId",
column: x => x.ConversationId,
principalTable: "Conversations",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "IX_Conversations_DeviceId",
table: "Conversations",
column: "DeviceId");
migrationBuilder.CreateIndex(
name: "IX_Turns_ConversationId_Index",
table: "Turns",
columns: new[] { "ConversationId", "Index" },
unique: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "Turns");
migrationBuilder.DropTable(
name: "Conversations");
}
}
}
@@ -0,0 +1,479 @@
// <auto-generated />
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
using backend.Data;
#nullable disable
namespace backend.Migrations
{
[DbContext(typeof(AppDbContext))]
partial class AppDbContextModelSnapshot : ModelSnapshot
{
protected override void BuildModel(ModelBuilder modelBuilder)
{
#pragma warning disable 612, 618
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedName")
.IsUnique()
.HasDatabaseName("RoleNameIndex");
b.ToTable("AspNetRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("RoleId");
b.ToTable("AspNetRoleClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.Property<int>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER");
b.Property<string>("ClaimType")
.HasColumnType("TEXT");
b.Property<string>("ClaimValue")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("UserId");
b.ToTable("AspNetUserClaims", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("ProviderKey")
.HasColumnType("TEXT");
b.Property<string>("ProviderDisplayName")
.HasColumnType("TEXT");
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.HasKey("LoginProvider", "ProviderKey");
b.HasIndex("UserId");
b.ToTable("AspNetUserLogins", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<Guid>("RoleId")
.HasColumnType("TEXT");
b.HasKey("UserId", "RoleId");
b.HasIndex("RoleId");
b.ToTable("AspNetUserRoles", (string)null);
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.Property<Guid>("UserId")
.HasColumnType("TEXT");
b.Property<string>("LoginProvider")
.HasColumnType("TEXT");
b.Property<string>("Name")
.HasColumnType("TEXT");
b.Property<string>("Value")
.HasColumnType("TEXT");
b.HasKey("UserId", "LoginProvider", "Name");
b.ToTable("AspNetUserTokens", (string)null);
});
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<int>("AccessFailedCount")
.HasColumnType("INTEGER");
b.Property<string>("ConcurrencyStamp")
.IsConcurrencyToken()
.HasColumnType("TEXT");
b.Property<string>("Email")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<bool>("EmailConfirmed")
.HasColumnType("INTEGER");
b.Property<bool>("LockoutEnabled")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LockoutEnd")
.HasColumnType("TEXT");
b.Property<string>("NormalizedEmail")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("NormalizedUserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.Property<string>("PasswordHash")
.HasColumnType("TEXT");
b.Property<string>("PhoneNumber")
.HasColumnType("TEXT");
b.Property<bool>("PhoneNumberConfirmed")
.HasColumnType("INTEGER");
b.Property<string>("SecurityStamp")
.HasColumnType("TEXT");
b.Property<bool>("TwoFactorEnabled")
.HasColumnType("INTEGER");
b.Property<string>("UserName")
.HasMaxLength(256)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("NormalizedEmail")
.HasDatabaseName("EmailIndex");
b.HasIndex("NormalizedUserName")
.IsUnique()
.HasDatabaseName("UserNameIndex");
b.ToTable("AspNetUsers", (string)null);
});
modelBuilder.Entity("backend.Conversations.Conversation", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<int>("EndReason")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("EndedAt")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("StartedAt")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("DeviceId");
b.ToTable("Conversations");
});
modelBuilder.Entity("backend.Conversations.Turn", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<Guid>("ConversationId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("CreatedAt")
.HasColumnType("TEXT");
b.Property<int>("Index")
.HasColumnType("INTEGER");
b.Property<int>("Role")
.HasColumnType("INTEGER");
b.Property<string>("Text")
.HasColumnType("TEXT");
b.Property<string>("ToolArgsJson")
.HasColumnType("TEXT");
b.Property<string>("ToolName")
.HasColumnType("TEXT");
b.Property<string>("ToolResultJson")
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("ConversationId", "Index")
.IsUnique();
b.ToTable("Turns");
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("TEXT");
b.Property<bool>("IsRevoked")
.HasColumnType("INTEGER");
b.Property<DateTimeOffset?>("LastSeenAt")
.HasColumnType("TEXT");
b.Property<string>("Name")
.IsRequired()
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("PairedAt")
.HasColumnType("TEXT");
b.Property<string>("TokenHash")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("OwnerUserId");
b.HasIndex("TokenHash")
.IsUnique();
b.ToTable("Devices");
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.Property<Guid>("DeviceId")
.HasColumnType("TEXT");
b.Property<string>("EnabledToolsJson")
.IsRequired()
.HasColumnType("TEXT");
b.Property<int>("IdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("Model")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("SystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("Voice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("DeviceId");
b.ToTable("DeviceConfigs");
});
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
{
b.Property<string>("Code")
.HasColumnType("TEXT");
b.Property<DateTimeOffset?>("ConsumedAt")
.HasColumnType("TEXT");
b.Property<string>("DeviceName")
.IsRequired()
.HasColumnType("TEXT");
b.Property<DateTimeOffset>("ExpiresAt")
.HasColumnType("TEXT");
b.Property<Guid>("OwnerUserId")
.HasColumnType("TEXT");
b.HasKey("Code");
b.HasIndex("ExpiresAt");
b.ToTable("PairingCodes");
});
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
{
b.Property<int>("Id")
.HasColumnType("INTEGER");
b.Property<int>("DefaultIdleTimeoutSeconds")
.HasColumnType("INTEGER");
b.Property<string>("DefaultModel")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultSystemPrompt")
.IsRequired()
.HasColumnType("TEXT");
b.Property<string>("DefaultVoice")
.IsRequired()
.HasColumnType("TEXT");
b.HasKey("Id");
b.ToTable("SystemSettings");
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
{
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
.WithMany()
.HasForeignKey("RoleId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("UserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Conversations.Turn", b =>
{
b.HasOne("backend.Conversations.Conversation", null)
.WithMany("Turns")
.HasForeignKey("ConversationId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.HasOne("backend.Auth.ApplicationUser", null)
.WithMany()
.HasForeignKey("OwnerUserId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
{
b.HasOne("backend.Devices.Device", null)
.WithOne("Config")
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("backend.Conversations.Conversation", b =>
{
b.Navigation("Turns");
});
modelBuilder.Entity("backend.Devices.Device", b =>
{
b.Navigation("Config");
});
#pragma warning restore 612, 618
}
}
}
+10
View File
@@ -0,0 +1,10 @@
namespace backend.Pairing;
public class PairingCode
{
public string Code { get; set; } = ""; // 8 chars, primary key
public Guid OwnerUserId { get; set; }
public string DeviceName { get; set; } = "";
public DateTimeOffset ExpiresAt { get; set; }
public DateTimeOffset? ConsumedAt { get; set; }
}
+47
View File
@@ -0,0 +1,47 @@
using System.Security.Claims;
using Microsoft.AspNetCore.Mvc;
namespace backend.Pairing;
public record IssueCodeRequest(string Name);
public record PairRequest(string Code, string Hostname, string Client_version);
public static class PairingEndpoints
{
public static IEndpointRouteBuilder MapPairingEndpoints(this IEndpointRouteBuilder app)
{
app.MapPost("/api/pair-code", async (
[FromBody] IssueCodeRequest req,
PairingService svc,
HttpContext http,
CancellationToken ct) =>
{
var uidStr = http.User.FindFirstValue(ClaimTypes.NameIdentifier);
if (!Guid.TryParse(uidStr, out var uid)) return Results.Unauthorized();
if (string.IsNullOrWhiteSpace(req.Name))
return Results.BadRequest(new { error = "name is required" });
var pc = await svc.IssueAsync(uid, req.Name.Trim(), ct);
return Results.Ok(new { code = pc.Code, expires_at = pc.ExpiresAt });
}).RequireAuthorization();
app.MapPost("/api/pair", async (
[FromBody] PairRequest req,
PairingService svc,
HttpContext http,
CancellationToken ct) =>
{
var result = await svc.ConsumeAsync(req.Code, ct);
if (result is null) return Results.NotFound(new { error = "code invalid or expired" });
var scheme = http.Request.Scheme == "https" ? "wss" : "ws";
var host = http.Request.Host.ToString();
return Results.Ok(new
{
device_id = result.DeviceId,
device_token = result.DeviceToken,
backend_ws = $"{scheme}://{host}/device",
});
});
return app;
}
}
+84
View File
@@ -0,0 +1,84 @@
using System.Security.Cryptography;
using backend.Data;
using backend.Devices;
using Microsoft.EntityFrameworkCore;
namespace backend.Pairing;
public record PairResult(Guid DeviceId, string DeviceToken);
public class PairingService
{
private const string Alphabet = "ABCDEFGHJKMNPQRSTUVWXYZ23456789"; // no O, 0, I, 1, L
private const int CodeLength = 8;
public static readonly TimeSpan CodeTtl = TimeSpan.FromMinutes(10);
private readonly AppDbContext? _db;
private readonly DeviceTokenService? _tokens;
public PairingService() { }
public PairingService(AppDbContext db, DeviceTokenService tokens)
{
_db = db;
_tokens = tokens;
}
public string GenerateCode()
{
Span<byte> bytes = stackalloc byte[CodeLength];
RandomNumberGenerator.Fill(bytes);
Span<char> chars = stackalloc char[CodeLength];
for (var i = 0; i < CodeLength; i++)
chars[i] = Alphabet[bytes[i] % Alphabet.Length];
return new string(chars);
}
public async Task<PairingCode> IssueAsync(Guid ownerUserId, string deviceName, CancellationToken ct)
{
if (_db is null) throw new InvalidOperationException("PairingService not constructed with DB context.");
var entity = new PairingCode
{
Code = GenerateCode(),
OwnerUserId = ownerUserId,
DeviceName = deviceName,
ExpiresAt = DateTimeOffset.UtcNow.Add(CodeTtl),
};
_db.PairingCodes.Add(entity);
await _db.SaveChangesAsync(ct);
return entity;
}
public async Task<PairResult?> ConsumeAsync(string code, CancellationToken ct)
{
if (_db is null || _tokens is null)
throw new InvalidOperationException("PairingService not constructed with DB context + token service.");
var now = DateTimeOffset.UtcNow;
var pc = await _db.PairingCodes.FirstOrDefaultAsync(p => p.Code == code, ct);
if (pc is null || pc.ConsumedAt is not null || pc.ExpiresAt < now)
return null;
var (plaintext, hash) = _tokens.Generate();
var defaults = await _db.SystemSettings.FirstAsync(ct);
var device = new Device
{
OwnerUserId = pc.OwnerUserId,
Name = pc.DeviceName,
PairedAt = now,
TokenHash = hash,
};
device.Config = new DeviceConfig
{
DeviceId = device.Id,
SystemPrompt = defaults.DefaultSystemPrompt,
Voice = defaults.DefaultVoice,
Model = defaults.DefaultModel,
IdleTimeoutSeconds = defaults.DefaultIdleTimeoutSeconds,
};
_db.Devices.Add(device);
pc.ConsumedAt = now;
await _db.SaveChangesAsync(ct);
return new PairResult(device.Id, plaintext);
}
}
+79
View File
@@ -0,0 +1,79 @@
using backend.Auth;
using backend.Data;
using backend.DeviceHub;
using backend.Devices;
using backend.Install;
using backend.Pairing;
using Microsoft.AspNetCore.DataProtection;
using Microsoft.AspNetCore.HttpOverrides;
using Microsoft.EntityFrameworkCore;
var builder = WebApplication.CreateBuilder(args);
var keysDir = Environment.GetEnvironmentVariable("DataProtection__KeysDir") ?? "/keys";
if (Directory.Exists(keysDir))
{
builder.Services.AddDataProtection()
.PersistKeysToFileSystem(new DirectoryInfo(keysDir))
.SetApplicationName("smart-assistant");
}
builder.Services.Configure<ForwardedHeadersOptions>(opt =>
{
opt.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
opt.KnownNetworks.Clear();
opt.KnownProxies.Clear();
});
builder.Services.AddDbContext<AppDbContext>(opt =>
opt.UseSqlite(builder.Configuration.GetConnectionString("Default")));
builder.Services.AddAppIdentity();
builder.Services.AddAuthorization();
builder.Services.AddScoped<backend.Devices.DeviceTokenService>();
builder.Services.AddScoped<backend.Pairing.PairingService>();
builder.Services.AddSingleton<backend.Install.InstallScriptBuilder>();
builder.Services.AddScoped<backend.Conversations.ConversationLog>();
builder.Services.AddSingleton<backend.Realtime.OpenAIKeyProvider>();
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.GetCurrentTimeTool>();
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.EndSessionTool>();
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.SetVolumeTool>();
builder.Services.AddSingleton<backend.Tools.ToolRegistry>();
builder.Services.AddScoped<backend.Realtime.RealtimeSessionFactory>();
builder.Services.AddSingleton<backend.DeviceHub.DeviceRegistry>();
builder.Services.AddScoped<backend.DeviceHub.DeviceAuth>();
var app = builder.Build();
using (var scope = app.Services.CreateScope())
{
var sp = scope.ServiceProvider;
var db = sp.GetRequiredService<AppDbContext>();
db.Database.Migrate();
await IdentitySetup.SeedRolesAsync(sp);
if (db.SystemSettings.Find(1) is null)
{
db.SystemSettings.Add(new backend.Settings.SystemSettings { Id = 1 });
await db.SaveChangesAsync();
}
}
app.UseForwardedHeaders();
app.UseAuthentication();
app.UseAuthorization();
app.MapAuthEndpoints();
app.MapPairingEndpoints();
app.MapDevicesEndpoints();
app.MapInstallEndpoints();
app.MapGet("/health", () => Results.Ok(new { ok = true }));
app.UseWebSockets();
app.MapDeviceHub();
app.Run();
public partial class Program { }
+23
View File
@@ -0,0 +1,23 @@
{
"$schema": "https://json.schemastore.org/launchsettings.json",
"profiles": {
"http": {
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"applicationUrl": "http://localhost:5252",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},
"https": {
"commandName": "Project",
"dotnetRunMessages": true,
"launchBrowser": true,
"applicationUrl": "https://localhost:7061;http://localhost:5252",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
}
}
}
+13
View File
@@ -0,0 +1,13 @@
using System.Text.Json.Nodes;
namespace backend.Realtime;
public interface IRealtimeUpstream : IAsyncDisposable
{
Task SendJsonAsync(JsonObject envelope, CancellationToken ct);
Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct);
// Once a receive returns null because the WebSocket transitioned out of
// the Open state, these surface why. Both stay null until that happens.
bool IsClosed { get; }
string? CloseReason { get; }
}
+21
View File
@@ -0,0 +1,21 @@
namespace backend.Realtime;
public class OpenAIKeyProvider
{
private readonly string? _key;
public OpenAIKeyProvider(IConfiguration cfg)
{
_key = cfg["OPENAI_API_KEY"] ?? cfg["OpenAI:ApiKey"];
}
public bool HasKey => !string.IsNullOrWhiteSpace(_key);
public string GetRequired()
{
if (string.IsNullOrWhiteSpace(_key))
throw new InvalidOperationException(
"OPENAI_API_KEY is not configured. Set the env var on the Coolify app.");
return _key!;
}
}
@@ -0,0 +1,77 @@
using System.Net.WebSockets;
using System.Text;
using System.Text.Json.Nodes;
namespace backend.Realtime;
public class OpenAIRealtimeUpstream : IRealtimeUpstream
{
private readonly ClientWebSocket _ws = new();
private string? _closeReason;
public bool IsClosed => _ws.State != WebSocketState.Open && _ws.State != WebSocketState.Connecting;
public string? CloseReason => _closeReason;
public async Task ConnectAsync(string apiKey, string model, CancellationToken ct)
{
_ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
// The GA Realtime API does NOT take the OpenAI-Beta: realtime=v1 header.
// Sending it returns upstream error code=beta_api_shape_disabled.
var uri = new Uri($"wss://api.openai.com/v1/realtime?model={Uri.EscapeDataString(model)}");
await _ws.ConnectAsync(uri, ct);
}
public async Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
{
var json = envelope.ToJsonString();
var bytes = Encoding.UTF8.GetBytes(json);
await _ws.SendAsync(bytes, WebSocketMessageType.Text, endOfMessage: true, ct);
}
public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
{
var buffer = new byte[8192];
using var stream = new MemoryStream();
while (true)
{
WebSocketReceiveResult result;
try
{
result = await _ws.ReceiveAsync(buffer, ct);
}
catch (WebSocketException exc)
{
_closeReason ??= $"WebSocketException:{exc.WebSocketErrorCode}:{exc.Message}";
return null;
}
catch (OperationCanceledException)
{
return null;
}
if (result.MessageType == WebSocketMessageType.Close)
{
var status = _ws.CloseStatus?.ToString() ?? "<none>";
var desc = _ws.CloseStatusDescription ?? "<no description>";
_closeReason ??= $"close status={status} desc={desc}";
return null;
}
stream.Write(buffer, 0, result.Count);
if (result.EndOfMessage)
{
var json = Encoding.UTF8.GetString(stream.ToArray());
return JsonNode.Parse(json)?.AsObject();
}
}
}
public async ValueTask DisposeAsync()
{
if (_ws.State == WebSocketState.Open)
{
try { await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None); }
catch { }
}
_ws.Dispose();
}
}
+92
View File
@@ -0,0 +1,92 @@
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Tools;
namespace backend.Realtime;
public static class RealtimeEvents
{
// GA Realtime API shape (the Beta API shape was retired in 2026 with
// upstream error "beta_api_shape_disabled"). Key differences:
// - session.type = "realtime" is required.
// - voice / audio formats / transcription / turn_detection now nested
// under session.audio.{input,output}.
// - No OpenAI-Beta header on the upgrade (see OpenAIRealtimeUpstream).
public static JsonObject SessionUpdate(RealtimeSettings cfg, IEnumerable<ITool> enabledTools)
{
var tools = new JsonArray();
foreach (var t in enabledTools)
{
tools.Add(new JsonObject
{
["type"] = "function",
["name"] = t.Name,
["description"] = t.Description,
["parameters"] = JsonNode.Parse(t.ParameterSchema.GetRawText())!,
});
}
// Minimal GA session.update. We drop explicit audio.input/output.format
// objects (the GA shape accepts them but OpenAI defaults to 24 kHz PCM16
// mono on both directions, which is what we use). Removing the explicit
// format objects matches the documented minimal session config and
// eliminates a known source of OpenAI silently ignoring the audio.
return new JsonObject
{
["type"] = "session.update",
["session"] = new JsonObject
{
["type"] = "realtime",
["model"] = cfg.Model,
["instructions"] = cfg.SystemPrompt,
["audio"] = new JsonObject
{
["input"] = new JsonObject
{
["transcription"] = new JsonObject { ["model"] = "whisper-1" },
["turn_detection"] = new JsonObject
{
["type"] = "server_vad",
["threshold"] = 0.5,
["prefix_padding_ms"] = 300,
["silence_duration_ms"] = 500,
["create_response"] = true,
["interrupt_response"] = true,
},
},
["output"] = new JsonObject
{
["voice"] = cfg.Voice,
},
},
["tools"] = tools,
["tool_choice"] = "auto",
},
};
}
public static JsonObject InputAudioBufferAppend(ReadOnlySpan<byte> pcm16)
{
return new JsonObject
{
["type"] = "input_audio_buffer.append",
["audio"] = Convert.ToBase64String(pcm16),
};
}
public static JsonObject FunctionCallOutput(string callId, string outputJson)
{
return new JsonObject
{
["type"] = "conversation.item.create",
["item"] = new JsonObject
{
["type"] = "function_call_output",
["call_id"] = callId,
["output"] = outputJson,
},
};
}
public static JsonObject ResponseCreate() => new() { ["type"] = "response.create" };
}
+358
View File
@@ -0,0 +1,358 @@
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Tools;
using Microsoft.Extensions.Logging;
namespace backend.Realtime;
public interface ISessionOutput
{
Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct);
Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
}
public class RealtimeSession
{
private readonly Guid _deviceId;
private readonly IRealtimeUpstream _upstream;
private readonly ISessionOutput _output;
private readonly ConversationLog _log;
private readonly ToolRegistry _registry;
private readonly RealtimeSettings _cfg;
private readonly IDeviceChannel _channel;
private readonly ILogger _logger;
private Conversation? _conversation;
private EndReason _endReason = EndReason.Unknown;
private bool _endRequested;
private DateTime _lastSpeechAt = DateTime.UtcNow;
private readonly List<(string CallId, string Name, string ArgsJson)> _pendingCalls = new();
private readonly System.Threading.Channels.Channel<byte[]> _uplink =
System.Threading.Channels.Channel.CreateBounded<byte[]>(
new System.Threading.Channels.BoundedChannelOptions(64)
{
FullMode = System.Threading.Channels.BoundedChannelFullMode.DropOldest,
});
public ValueTask WriteUplinkAsync(byte[] frame, CancellationToken ct)
=> _uplink.Writer.WriteAsync(frame, ct);
public RealtimeSession(
Guid deviceId,
IRealtimeUpstream upstream,
ISessionOutput output,
ConversationLog log,
ToolRegistry registry,
RealtimeSettings cfg,
IDeviceChannel channel,
ILogger logger)
{
_deviceId = deviceId;
_upstream = upstream;
_output = output;
_log = log;
_registry = registry;
_cfg = cfg;
_channel = channel;
_logger = logger;
}
public Guid? ConversationId => _conversation?.Id;
public async Task RunAsync(CancellationToken ct)
{
try
{
var enabled = _registry.EnabledFor(_cfg.EnabledTools).ToList();
_logger.LogInformation(
"relay opening session: model={Model} voice={Voice} tools=[{Tools}]",
_cfg.Model, _cfg.Voice, string.Join(",", enabled.Select(t => t.Name)));
await _upstream.SendJsonAsync(RealtimeEvents.SessionUpdate(_cfg, enabled), ct);
var ack = await WaitForUpstreamTypeAsync("session.updated", ct);
if (ack is null)
{
_logger.LogWarning("upstream closed before session.updated");
_endReason = EndReason.Error;
return;
}
_conversation = await _log.StartAsync(_deviceId, ct);
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "session_started",
["conversation_id"] = _conversation.Id.ToString(),
}, ct);
_lastSpeechAt = DateTime.UtcNow;
var uplinkTask = Task.Run(() => PumpUplinkAsync(ct), ct);
var idleTask = Task.Run(() => WatchdogAsync(ct), ct);
await PumpAsync(ct);
_uplink.Writer.TryComplete();
await uplinkTask;
await idleTask;
}
catch (OperationCanceledException)
{
if (_endReason == EndReason.Unknown) _endReason = EndReason.Error;
}
catch (Exception ex)
{
_logger.LogError(ex, "RealtimeSession crashed");
_endReason = EndReason.Error;
}
finally
{
await _upstream.DisposeAsync();
if (_conversation is not null)
{
await _log.EndAsync(_conversation.Id, _endReason, CancellationToken.None);
}
var endEnv = new JsonObject
{
["type"] = "session_ended",
["reason"] = _endReason.ToString().ToLowerInvariant(),
};
await _output.WriteEnvelopeAsync(endEnv, CancellationToken.None);
}
}
private async Task PumpAsync(CancellationToken ct)
{
int nullPolls = 0;
while (!ct.IsCancellationRequested && !_endRequested)
{
var evt = await ReceiveOrIdleAsync(ct);
if (evt is null && _endRequested) return;
if (evt is null && ct.IsCancellationRequested) return;
if (evt is null)
{
// If the upstream WS is closed, stop spinning — surface the
// close reason to the device and bail with EndReason=Error.
if (_upstream.IsClosed)
{
var reason = _upstream.CloseReason ?? "upstream closed";
_logger.LogWarning("upstream closed mid-session: {Reason}", reason);
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "error",
["code"] = "upstream_closed",
["message"] = reason,
["fatal"] = false,
}, ct);
_endReason = EndReason.Error;
return;
}
// Surface "alive but silent" every ~2s so we can tell that
// from "the upstream WS was closed". Coolify logs aren't
// reachable from the dev sandbox.
if (++nullPolls % 8 == 0)
{
_logger.LogInformation("upstream silent ({Polls} x 250ms polls)", nullPolls);
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "debug",
["message"] = "upstream silent " + nullPolls + "x250ms",
}, ct);
}
continue;
}
nullPolls = 0;
var type = (string?)evt["type"];
_logger.LogInformation("relay evt={EvtType}", type);
// Forward the event type to the device so it appears in the Pi
// journal — Coolify logs aren't accessible from the dev sandbox.
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "debug",
["message"] = "relay evt=" + (type ?? "<null>"),
}, ct);
switch (type)
{
case "error":
{
var err = evt["error"] as JsonObject;
var code = (string?)err?["code"] ?? "unknown";
var msg = (string?)err?["message"] ?? evt.ToJsonString();
_logger.LogWarning("upstream mid-session error code={Code} message={Message}", code, msg);
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "error",
["code"] = "upstream_" + code,
["message"] = msg,
["fatal"] = false,
}, ct);
break;
}
case "input_audio_buffer.speech_started":
_lastSpeechAt = DateTime.UtcNow;
break;
case "response.audio.delta":
await ForwardAudioDeltaAsync(evt, ct);
break;
case "conversation.item.input_audio_transcription.completed":
{
var text = (string?)evt["transcript"];
if (_conversation is not null && !string.IsNullOrEmpty(text))
await _log.AppendUserAsync(_conversation.Id, text!, ct);
break;
}
case "response.audio_transcript.done":
{
var text = (string?)evt["transcript"];
if (_conversation is not null && !string.IsNullOrEmpty(text))
await _log.AppendAssistantAsync(_conversation.Id, text!, ct);
break;
}
case "response.function_call_arguments.done":
{
var callId = (string?)evt["call_id"] ?? "";
var name = (string?)evt["name"] ?? "";
var args = (string?)evt["arguments"] ?? "{}";
_pendingCalls.Add((callId, name, args));
break;
}
case "response.done":
{
await _output.WriteEnvelopeAsync(
new JsonObject { ["type"] = "assistant_done" }, ct);
if (_pendingCalls.Count == 0) break;
var calls = _pendingCalls.ToList();
_pendingCalls.Clear();
if (calls.Any(c => c.Name == "end_session"))
{
_endReason = EndReason.Tool;
_endRequested = true;
break;
}
foreach (var c in calls)
{
await ExecuteToolAsync(c.CallId, c.Name, c.ArgsJson, ct);
}
break;
}
}
}
}
private async Task ExecuteToolAsync(string callId, string name, string argsJson, CancellationToken ct)
{
var tool = _registry.Get(name);
string outputJson;
if (tool is null)
{
outputJson = $$"""{"ok":false,"error":"unknown tool: {{name}}"}""";
}
else
{
try
{
using var argsDoc = System.Text.Json.JsonDocument.Parse(argsJson);
var ctx = new DeviceContext(_deviceId, _conversation!.Id, _channel);
var res = await tool.ExecuteAsync(
new ToolInvocation(callId, argsDoc.RootElement.Clone()), ctx, ct);
outputJson = res.Output.GetRawText();
}
catch (Exception ex)
{
outputJson = System.Text.Json.JsonSerializer.Serialize(new
{
ok = false,
error = ex.Message,
});
}
}
if (_conversation is not null)
{
await _log.AppendToolAsync(_conversation.Id, name, argsJson, outputJson, ct);
}
await _upstream.SendJsonAsync(RealtimeEvents.FunctionCallOutput(callId, outputJson), ct);
await _upstream.SendJsonAsync(RealtimeEvents.ResponseCreate(), ct);
}
private async Task ForwardAudioDeltaAsync(JsonObject evt, CancellationToken ct)
{
var b64 = (string?)evt["delta"];
if (string.IsNullOrEmpty(b64)) return;
var bytes = Convert.FromBase64String(b64);
await _output.WriteBinaryAsync(bytes, ct);
}
private async Task WatchdogAsync(CancellationToken ct)
{
try
{
var timeout = TimeSpan.FromSeconds(_cfg.IdleTimeoutSeconds);
while (!ct.IsCancellationRequested && !_endRequested)
{
await Task.Delay(250, ct);
if (DateTime.UtcNow - _lastSpeechAt > timeout)
{
_endReason = EndReason.Idle;
_endRequested = true;
return;
}
}
}
catch (OperationCanceledException) { }
}
private async Task<JsonObject?> ReceiveOrIdleAsync(CancellationToken ct)
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
var receive = _upstream.ReceiveJsonAsync(timeout.Token);
var delay = Task.Delay(250, timeout.Token);
var done = await Task.WhenAny(receive, delay);
if (done == receive) return await receive;
timeout.Cancel();
try { await receive; } catch { /* ignore */ }
return null;
}
private async Task PumpUplinkAsync(CancellationToken ct)
{
try
{
await foreach (var frame in _uplink.Reader.ReadAllAsync(ct))
{
await _upstream.SendJsonAsync(RealtimeEvents.InputAudioBufferAppend(frame), ct);
}
}
catch (OperationCanceledException) { }
catch (Exception ex) { _logger.LogWarning(ex, "uplink pump errored"); }
}
private async Task<JsonObject?> WaitForUpstreamTypeAsync(string type, CancellationToken ct)
{
while (true)
{
var evt = await _upstream.ReceiveJsonAsync(ct);
if (evt is null) return null;
var evtType = (string?)evt["type"];
_logger.LogInformation("upstream evt={EvtType}", evtType);
if (evtType == "error")
{
var err = evt["error"] as JsonObject;
var code = (string?)err?["code"] ?? "unknown";
var msg = (string?)err?["message"] ?? evt.ToJsonString();
_logger.LogWarning("upstream error code={Code} message={Message}", code, msg);
await _output.WriteEnvelopeAsync(new JsonObject
{
["type"] = "error",
["code"] = "upstream_" + code,
["message"] = msg,
["fatal"] = false,
}, ct);
_endReason = EndReason.Error;
return null;
}
if (evtType == type) return evt;
}
}
}
@@ -0,0 +1,37 @@
using backend.Conversations;
using backend.Tools;
using Microsoft.Extensions.Logging;
namespace backend.Realtime;
public class RealtimeSessionFactory
{
protected readonly OpenAIKeyProvider keys;
protected readonly ToolRegistry registry;
protected readonly ConversationLog log;
protected readonly ILoggerFactory loggerFactory;
public RealtimeSessionFactory(
OpenAIKeyProvider keys, ToolRegistry registry,
ConversationLog log, ILoggerFactory loggerFactory)
{
this.keys = keys;
this.registry = registry;
this.log = log;
this.loggerFactory = loggerFactory;
}
public virtual async Task<RealtimeSession> CreateAsync(
Guid deviceId,
RealtimeSettings cfg,
ISessionOutput output,
IDeviceChannel channel,
CancellationToken ct)
{
var upstream = new OpenAIRealtimeUpstream();
await upstream.ConnectAsync(keys.GetRequired(), cfg.Model, ct);
return new RealtimeSession(
deviceId, upstream, output, log, registry, cfg, channel,
loggerFactory.CreateLogger<RealtimeSession>());
}
}
+8
View File
@@ -0,0 +1,8 @@
namespace backend.Realtime;
public record RealtimeSettings(
string Model,
string Voice,
string SystemPrompt,
int IdleTimeoutSeconds,
IReadOnlySet<string> EnabledTools);
+11
View File
@@ -0,0 +1,11 @@
namespace backend.Settings;
public class SystemSettings
{
public int Id { get; set; } = 1;
public string DefaultSystemPrompt { get; set; } =
"You are a helpful voice assistant. Keep responses concise.";
public string DefaultVoice { get; set; } = "alloy";
public string DefaultModel { get; set; } = "gpt-realtime";
public int DefaultIdleTimeoutSeconds { get; set; } = 30;
}
+20
View File
@@ -0,0 +1,20 @@
using System.Text.Json;
namespace backend.Tools;
public class EndSessionTool : ITool
{
public string Name => "end_session";
public string Description =>
"Call this when the user has said goodbye or otherwise indicates the conversation is over.";
public bool RunsDuringResponse => false;
public JsonElement ParameterSchema { get; } =
JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;
public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
{
var output = JsonDocument.Parse("""{"ok":true}""").RootElement;
return Task.FromResult(new ToolResult(output));
}
}
+20
View File
@@ -0,0 +1,20 @@
using System.Text.Json;
namespace backend.Tools;
public class GetCurrentTimeTool : ITool
{
public string Name => "get_current_time";
public string Description => "Returns the current UTC time as an ISO-8601 string.";
public bool RunsDuringResponse => false;
public JsonElement ParameterSchema { get; } =
JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;
public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
{
var now = DateTimeOffset.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ");
var output = JsonDocument.Parse($$"""{"now":"{{now}}"}""").RootElement;
return Task.FromResult(new ToolResult(output));
}
}
+8
View File
@@ -0,0 +1,8 @@
using System.Text.Json;
namespace backend.Tools;
public interface IDeviceChannel
{
Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct);
}
+19
View File
@@ -0,0 +1,19 @@
using System.Text.Json;
namespace backend.Tools;
public record DeviceContext(Guid DeviceId, Guid ConversationId, IDeviceChannel Channel);
public record ToolInvocation(string CallId, JsonElement Arguments);
public record ToolResult(JsonElement Output);
public interface ITool
{
string Name { get; }
string Description { get; }
JsonElement ParameterSchema { get; }
bool RunsDuringResponse { get; }
Task<ToolResult> ExecuteAsync(
ToolInvocation invocation,
DeviceContext device,
CancellationToken ct);
}
+34
View File
@@ -0,0 +1,34 @@
using System.Text.Json;
namespace backend.Tools;
public class SetVolumeTool : ITool
{
public string Name => "set_volume";
public string Description =>
"Sets the assistant's audio output volume on the device. Level is 0..100.";
public bool RunsDuringResponse => true;
public JsonElement ParameterSchema { get; } = JsonDocument.Parse(
"""
{
"type":"object",
"properties":{"level":{"type":"integer","minimum":0,"maximum":100}},
"required":["level"]
}
""").RootElement;
public async Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
{
if (!invocation.Arguments.TryGetProperty("level", out var lvlEl)
|| !lvlEl.TryGetInt32(out var level)
|| level < 0 || level > 100)
{
var err = JsonDocument.Parse("""{"ok":false,"error":"level must be an integer in 0..100"}""").RootElement;
return new ToolResult(err);
}
var piReply = await device.Channel.CallPiToolAsync(Name, invocation.Arguments, ct);
return new ToolResult(piReply);
}
}
+20
View File
@@ -0,0 +1,20 @@
namespace backend.Tools;
public class ToolRegistry
{
private readonly Dictionary<string, ITool> _byName;
private readonly List<ITool> _all;
public ToolRegistry(IEnumerable<ITool> tools)
{
_all = tools.ToList();
_byName = _all.ToDictionary(t => t.Name);
}
public IReadOnlyList<ITool> All => _all;
public ITool? Get(string name) => _byName.GetValueOrDefault(name);
public IEnumerable<ITool> EnabledFor(IReadOnlySet<string> enabled)
=> _all.Where(t => enabled.Contains(t.Name));
}
+8
View File
@@ -0,0 +1,8 @@
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
}
}
+12
View File
@@ -0,0 +1,12 @@
{
"ConnectionStrings": {
"Default": "Data Source=./assistant.db"
},
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*"
}
+19
View File
@@ -0,0 +1,19 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.DataProtection" Version="9.*" />
<PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="9.*" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="9.*">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="9.*" />
</ItemGroup>
</Project>
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
PI_HOST="${PI_HOST:-192.168.50.115}"
PI_USER="${PI_USER:-pi}"
PI_PASS="${PI_PASS:-assistant}"
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-out}"
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
cd "$repo_root"
echo ">> dotnet publish (linux-arm64, self-contained)"
dotnet publish tests/01-record-play-cs \
-c Release -r linux-arm64 --self-contained \
-o "$PUBLISH_DIR"
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs/ (wipe-and-replace)"
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs && mkdir ~/probe-cs'
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
"$PI_USER@$PI_HOST:~/probe-cs/"
echo ">> ssh + run on Pi"
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'cd ~/probe-cs && chmod +x Probe && ./Probe'
-25
View File
@@ -1,25 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
PI_HOST="${PI_HOST:-192.168.50.115}"
PI_USER="${PI_USER:-pi}"
PI_PASS="${PI_PASS:-assistant}"
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-2-out}"
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
cd "$repo_root"
echo ">> dotnet publish (linux-arm64, self-contained)"
dotnet publish tests/02-wakeword-cs \
-c Release -r linux-arm64 --self-contained \
-o "$PUBLISH_DIR"
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-2/ (wipe-and-replace)"
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-2 && mkdir ~/probe-cs-2'
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
"$PI_USER@$PI_HOST:~/probe-cs-2/"
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-2 && chmod +x Probe && ./Probe'
View File
+139
View File
@@ -0,0 +1,139 @@
"""Audio helpers for the Pi client.
This module is split into two layers:
* Deterministic helpers (no hardware) — `resample_24k_to_16k`. Tested.
* `AudioStreams` — owns the persistent PortAudio InputStream / OutputStream.
Smoke-tested by `main.py` running on the Pi.
"""
import queue
import sys
from dataclasses import dataclass, field
from typing import Optional
import numpy as np
from scipy.signal import resample_poly
from client.log import get_logger
_log = get_logger("audio")
SAMPLE_RATE = 24_000
FRAME_SAMPLES = 1920 # 80 ms at 24 kHz
CHANNELS = 1
DTYPE = "int16"
BYTES_PER_FRAME = FRAME_SAMPLES * 2 # int16 -> 2 bytes
def resample_24k_to_16k(frame: np.ndarray) -> np.ndarray:
"""Downsample a 24 kHz int16 mono frame to 16 kHz int16 mono (2:3 poly).
Input length is preserved as `len(frame) * 2 // 3`. For the 1920-sample
frames produced by our 80 ms blocksize at 24 kHz, this yields 1280 samples
(80 ms at 16 kHz), which is exactly what openwakeword expects.
"""
if frame.dtype != np.int16:
raise TypeError(f"expected int16, got {frame.dtype}")
if frame.size == 0:
raise ValueError("empty input")
floats = frame.astype(np.float32)
resampled = resample_poly(floats, up=2, down=3)
return np.clip(resampled, -32768, 32767).astype(np.int16)
def find_usb_device(sd) -> int:
"""Return the PortAudio device index of the USB Speaker Phone.
`sd` is the imported `sounddevice` module — passed in so the deferred
import of sounddevice (which requires PA_ALSA_PLUGHW already set in env)
stays in `main.py`.
"""
devices = sd.query_devices()
for idx, dev in enumerate(devices):
name = dev["name"].lower()
if "usb" in name and dev["max_input_channels"] >= 1:
return idx
print("USB audio device not found. Devices:", file=sys.stderr)
print(devices, file=sys.stderr)
raise SystemExit("no USB audio device found")
@dataclass
class AudioStreams:
"""Wrapper around persistent 24 kHz/mono/int16 InputStream + OutputStream.
The InputStream is callback-driven; each 1920-sample frame is pushed onto
`mic_queue` (a `queue.Queue[bytes]`). The bridge thread drains it. The
OutputStream is opened but writes are driven by `PlaybackWorker`.
"""
sd: object # sounddevice module
device_index: int
mic_queue: queue.Queue
overflow_counter: int = 0
_in_stream: Optional[object] = field(default=None, repr=False)
_out_stream: Optional[object] = field(default=None, repr=False)
@classmethod
def open(cls, sd, mic_queue: queue.Queue) -> "AudioStreams":
device = find_usb_device(sd)
_log.info("opening audio on device %d: %r", device, sd.query_devices(device)["name"])
streams = cls(sd=sd, device_index=device, mic_queue=mic_queue)
def _on_input(indata, frames, time_info, status): # PortAudio thread
if status:
streams.overflow_counter += 1
_log.warning("input status: %s", status)
mono = indata[:, 0].tobytes()
try:
mic_queue.put_nowait(mono)
except queue.Full:
try:
mic_queue.get_nowait()
except queue.Empty:
pass
try:
mic_queue.put_nowait(mono)
except queue.Full:
pass
_log.warning("mic_queue full; dropped oldest frame")
streams._in_stream = sd.InputStream(
samplerate=SAMPLE_RATE,
channels=CHANNELS,
dtype=DTYPE,
device=device,
blocksize=FRAME_SAMPLES,
callback=_on_input,
)
streams._out_stream = sd.OutputStream(
samplerate=SAMPLE_RATE,
channels=CHANNELS,
dtype=DTYPE,
device=device,
blocksize=FRAME_SAMPLES,
)
return streams
def start(self) -> None:
self._in_stream.start()
self._out_stream.start()
def write(self, pcm16_bytes: bytes) -> None:
"""Blocking write to the OutputStream (from the playback worker thread)."""
if not pcm16_bytes:
return
buf = np.frombuffer(pcm16_bytes, dtype=np.int16)
self._out_stream.write(buf)
def close(self) -> None:
for s in (self._in_stream, self._out_stream):
if s is not None:
try:
s.stop()
except Exception:
pass
try:
s.close()
except Exception:
pass
+49
View File
@@ -0,0 +1,49 @@
"""Read / write the device config file at ~/assistant/state/config.json."""
import json
import os
from dataclasses import asdict, dataclass
from pathlib import Path
class ConfigMissingError(RuntimeError):
"""Raised when config.json is missing or malformed."""
@dataclass(frozen=True)
class Config:
device_id: str
device_token: str
backend_ws: str
def config_path() -> Path:
return Path(os.environ["HOME"]) / "assistant" / "state" / "config.json"
def load_config() -> Config:
path = config_path()
if not path.exists():
raise ConfigMissingError(f"{path} not found; run `python -m client.pair` first")
try:
raw = json.loads(path.read_text())
except json.JSONDecodeError as exc:
raise ConfigMissingError(f"{path} is not valid JSON: {exc}") from exc
try:
return Config(
device_id=str(raw["device_id"]),
device_token=str(raw["device_token"]),
backend_ws=str(raw["backend_ws"]),
)
except KeyError as exc:
raise ConfigMissingError(f"{path} missing field: {exc}") from exc
def save_config(cfg: Config) -> None:
path = config_path()
path.parent.mkdir(parents=True, exist_ok=True)
# Write to a temp file then atomically rename, so a partial write never
# leaves a half-baked config on disk.
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(asdict(cfg), indent=2) + "\n")
os.chmod(tmp, 0o600)
os.replace(tmp, path)
+28
View File
@@ -0,0 +1,28 @@
"""Structured stdout logging suitable for journald capture."""
import logging
import os
import sys
_CONFIGURED = False
def configure(level: str | int | None = None) -> None:
"""Configure root logging once. Safe to call multiple times."""
global _CONFIGURED
if _CONFIGURED:
return
if level is None:
level = os.environ.get("ASSISTANT_LOG_LEVEL", "INFO").upper()
logging.basicConfig(
level=level,
format="%(asctime)s %(levelname)-5s %(name)s %(message)s",
datefmt="%H:%M:%S",
stream=sys.stdout,
)
_CONFIGURED = True
def get_logger(name: str) -> logging.Logger:
"""Return a logger named `name`; auto-configure on first call."""
configure()
return logging.getLogger(name)
+192
View File
@@ -0,0 +1,192 @@
"""Smart Assistant Pi client entry point.
CRITICAL: set PA_ALSA_PLUGHW BEFORE importing sounddevice (see findings.md §1).
"""
import os
os.environ.setdefault("PA_ALSA_PLUGHW", "1")
import asyncio
import queue
import signal
import sys
import threading
from pathlib import Path
import numpy as np
import sounddevice as sd # safe to import now
from client.audio import (
AudioStreams,
FRAME_SAMPLES,
resample_24k_to_16k,
)
from client.config import ConfigMissingError, load_config
from client.log import get_logger
from client.playback import PlaybackWorker
from client.session import Session
from client.state import StateMachine
from client.wakeword import WakewordDetector
_log = get_logger("main")
ASSISTANT_STATE_DIR = Path(os.environ["HOME"]) / "assistant" / "state"
WAKE_WAV = ASSISTANT_STATE_DIR / "wake.wav"
SLEEP_WAV = ASSISTANT_STATE_DIR / "sleep.wav"
def _build_bridge_thread(
mic_queue: queue.Queue,
sm: StateMachine,
wakeword: WakewordDetector,
session: Session,
playback: PlaybackWorker,
stop_event: threading.Event,
) -> threading.Thread:
def run():
frame_counter = 0
recent_rms_max = 0.0
while not stop_event.is_set():
try:
pcm16_bytes = mic_queue.get(timeout=0.25)
except queue.Empty:
continue
frame_counter += 1
raw = np.frombuffer(pcm16_bytes, dtype=np.int16)
rms = float(np.sqrt(np.mean(raw.astype(np.float32) ** 2)))
if rms > recent_rms_max:
recent_rms_max = rms
# Every ~5 s emit a heartbeat so we can see if the InputStream is
# actually producing frames and what state we're routing to.
if frame_counter % 62 == 0:
_log.info(
"bridge frames=%d state=%s wake_en=%s up_en=%s qsize=%d rms_max_5s=%.0f",
frame_counter, sm.state.name,
sm.wakeword_enabled, sm.uplink_enabled, mic_queue.qsize(),
recent_rms_max,
)
recent_rms_max = 0.0
if sm.wakeword_enabled:
frame = np.frombuffer(pcm16_bytes, dtype=np.int16)
if frame.size != FRAME_SAMPLES:
continue
feed = resample_24k_to_16k(frame)
if wakeword.predict(feed):
if sm.wake_fired():
if WAKE_WAV.exists():
playback.enqueue_wav(WAKE_WAV, "wake")
else:
# Even with no wav, fire the done marker so the
# playback_done(for=wake) envelope is sent.
playback.enqueue_done_marker("wake")
session.call_soon_threadsafe_send_wake()
elif sm.uplink_enabled:
session.call_soon_threadsafe_send_binary(pcm16_bytes)
# else: ASSISTANT_SPEAKING / WAKE_PENDING / IDLE_PENDING -> drop
t = threading.Thread(target=run, name="bridge", daemon=True)
return t
def _make_callbacks(sm: StateMachine, playback: PlaybackWorker):
class _CB:
def on_session_started(self, conversation_id: str) -> None:
_log.info("session_started conversation_id=%s", conversation_id)
sm.session_started()
def on_assistant_audio(self, pcm16: bytes) -> None:
sm.assistant_audio_arrived()
playback.enqueue_audio(pcm16)
def on_assistant_done(self) -> None:
_log.info("assistant_done")
sm.assistant_done()
def on_session_ended(self, reason: str) -> None:
_log.info("session_ended reason=%s", reason)
sm.session_ended()
if SLEEP_WAV.exists():
playback.enqueue_wav(SLEEP_WAV, "sleep")
else:
playback.enqueue_done_marker("sleep")
def on_disconnected(self) -> None:
# The conversation is gone with the WS — reset to IDLE so the
# wakeword detector can re-arm. Any pending assistant audio left
# in the playback queue still drains, but we drop a "sleep" done
# marker only if we were mid-session.
_log.info("on_disconnected: forcing state to IDLE")
sm.force_idle()
return _CB()
def main() -> None:
try:
cfg = load_config()
except ConfigMissingError as exc:
raise SystemExit(str(exc)) from exc
_log.info("device_id=%s backend=%s", cfg.device_id, cfg.backend_ws)
mic_queue: queue.Queue = queue.Queue(maxsize=64)
streams = AudioStreams.open(sd, mic_queue=mic_queue)
streams.start()
sm = StateMachine()
session_ref: list[Session] = []
def on_playback_done(label: str) -> None:
_log.info("playback_done for=%s", label)
if not session_ref:
return
session = session_ref[0]
session.call_soon_threadsafe_send_playback_done(label)
if label == "sleep":
sm.sleep_played()
playback = PlaybackWorker(streams, on_done=on_playback_done)
playback.start()
wakeword = WakewordDetector()
callbacks = _make_callbacks(sm, playback)
session = Session(
backend_ws=cfg.backend_ws,
device_token=cfg.device_token,
device_id=cfg.device_id,
callbacks=callbacks,
)
session_ref.append(session)
def set_volume_handler(args: dict) -> dict:
level = int(args.get("level", -1))
playback.set_volume(level)
return {"ok": True, "level": level}
session.register_tool("set_volume", set_volume_handler)
stop_event = threading.Event()
bridge = _build_bridge_thread(mic_queue, sm, wakeword, session, playback, stop_event)
bridge.start()
def _shutdown(signum, _frame):
_log.info("signal %d received; shutting down", signum)
stop_event.set()
try:
playback.stop()
finally:
streams.close()
sys.exit(0)
signal.signal(signal.SIGTERM, _shutdown)
signal.signal(signal.SIGINT, _shutdown)
try:
asyncio.run(session.run_forever())
finally:
stop_event.set()
playback.stop()
streams.close()
if __name__ == "__main__":
main()
+68
View File
@@ -0,0 +1,68 @@
"""Pair this Pi with the backend.
Usage:
python -m client.pair --backend https://backend.example.com [--code XYZ12345]
Called from install.sh after it prompts the user for the code.
"""
import argparse
import socket
import requests
from client.config import Config, save_config
from client.log import get_logger
_log = get_logger("pair")
CLIENT_VERSION = "0.1.0"
def pair(*, backend: str, code: str | None, hostname: str) -> None:
backend = backend.rstrip("/")
if not code:
code = input("Pairing code: ").strip()
if not code:
raise SystemExit("no pairing code supplied")
url = f"{backend}/api/pair"
payload = {"code": code, "hostname": hostname, "client_version": CLIENT_VERSION}
_log.info("POST %s code=%s hostname=%s", url, code, hostname)
try:
resp = requests.post(url, json=payload, timeout=30)
except requests.RequestException as exc:
raise SystemExit(f"pair request failed: {exc}") from exc
if resp.status_code != 200:
try:
err = resp.json().get("error", resp.text)
except ValueError:
err = resp.text
raise SystemExit(f"/api/pair returned {resp.status_code}: {err}")
body = resp.json()
cfg = Config(
device_id=str(body["device_id"]),
device_token=str(body["device_token"]),
backend_ws=str(body["backend_ws"]),
)
save_config(cfg)
_log.info("paired device %s; config saved", cfg.device_id)
def main() -> None:
parser = argparse.ArgumentParser(description="Pair this Pi with the backend.")
parser.add_argument("--backend", required=True, help="Backend base URL (https://...)")
parser.add_argument("--code", default=None, help="Pairing code (prompted if omitted)")
parser.add_argument(
"--hostname",
default=socket.gethostname(),
help="Hostname reported to the backend (defaults to socket.gethostname())",
)
args = parser.parse_args()
pair(backend=args.backend, code=args.code, hostname=args.hostname)
if __name__ == "__main__":
main()
+117
View File
@@ -0,0 +1,117 @@
"""Playback worker thread + wake/sleep cue + software volume.
Items pushed to the queue:
("audio", bytes) - PCM16 LE 24 kHz mono data to write
("wav", (Path, label)) - WAV file to read and play; triggers on_done(label) after
("done_marker", str) - synthetic marker: after this is popped, call on_done(label)
("stop", None) - shut the worker down
Volume is applied as a software gain in [0..1.0] on int16 buffers because the
USB Speaker Phone has no hardware playback volume control (only `PCM Playback
Switch`, a mute).
"""
import queue
import threading
import wave
from pathlib import Path
from typing import Callable, Tuple
import numpy as np
from client.audio import SAMPLE_RATE
from client.log import get_logger
_log = get_logger("playback")
class PlaybackWorker:
def __init__(self, audio_streams, on_done: Callable[[str], None]):
"""
`audio_streams.write(pcm16_bytes)` is the blocking write into the
persistent OutputStream. `on_done(label)` is invoked when a tagged
cue (e.g. "wake", "sleep") finishes playing.
"""
self._streams = audio_streams
self._on_done = on_done
self._q: "queue.Queue[Tuple[str, object]]" = queue.Queue()
self._volume = 1.0
self._thread = threading.Thread(target=self._run, name="playback", daemon=True)
self._running = False
def start(self) -> None:
self._running = True
self._thread.start()
def stop(self) -> None:
self._running = False
self._q.put(("stop", None))
self._thread.join(timeout=2.0)
def set_volume(self, level: int) -> None:
"""Set software gain. `level` is 0..100."""
if not 0 <= level <= 100:
raise ValueError("level must be 0..100")
self._volume = level / 100.0
_log.info("volume set to %d%%", level)
def enqueue_audio(self, pcm16_bytes: bytes) -> None:
self._q.put(("audio", pcm16_bytes))
def enqueue_wav(self, path: Path, label: str) -> None:
"""Queue a WAV cue. When fully drained, on_done(label) fires."""
self._q.put(("wav", (path, label)))
def enqueue_done_marker(self, label: str) -> None:
"""Fire on_done(label) once this marker is processed (after all prior audio)."""
self._q.put(("done_marker", label))
def _apply_gain(self, pcm16: bytes) -> bytes:
if self._volume == 1.0:
return pcm16
arr = np.frombuffer(pcm16, dtype=np.int16).astype(np.int32)
arr = (arr * int(self._volume * 1024)) >> 10
arr = np.clip(arr, -32768, 32767).astype(np.int16)
return arr.tobytes()
def _play_wav(self, path: Path, label: str) -> None:
if not path.exists():
_log.info("wav %s not present at %s; skipping", label, path)
self._on_done(label)
return
try:
with wave.open(str(path), "rb") as w:
if w.getframerate() != SAMPLE_RATE or w.getnchannels() != 1 or w.getsampwidth() != 2:
_log.warning(
"wav %s is %dHz/%dch/%dB; expected %dHz/mono/16-bit",
path, w.getframerate(), w.getnchannels(), w.getsampwidth() * 8, SAMPLE_RATE,
)
self._on_done(label)
return
data = w.readframes(w.getnframes())
except wave.Error as exc:
_log.warning("could not read wav %s: %s", path, exc)
self._on_done(label)
return
chunk = 3840
for i in range(0, len(data), chunk):
self._streams.write(self._apply_gain(data[i:i + chunk]))
self._on_done(label)
def _run(self) -> None:
while self._running:
try:
kind, payload = self._q.get(timeout=0.5)
except queue.Empty:
continue
try:
if kind == "stop":
return
if kind == "audio":
self._streams.write(self._apply_gain(payload))
elif kind == "wav":
path, label = payload
self._play_wav(path, label)
elif kind == "done_marker":
self._on_done(payload)
except Exception as exc: # pragma: no cover - hardware path
_log.exception("playback worker error: %s", exc)
+235
View File
@@ -0,0 +1,235 @@
"""Backend WebSocket client (asyncio)."""
import asyncio
import json
import random
from typing import Callable, Protocol
from client.log import get_logger
_log = get_logger("session")
CLIENT_VERSION = "0.1.0"
PING_INTERVAL_S = 15.0
RECONNECT_MIN_S = 1.0
RECONNECT_MAX_S = 30.0
class Callbacks(Protocol):
def on_session_started(self, conversation_id: str) -> None: ...
def on_assistant_audio(self, pcm16: bytes) -> None: ...
def on_assistant_done(self) -> None: ...
def on_session_ended(self, reason: str) -> None: ...
# Called once per backend WS lifecycle when the connection drops, so the
# owner can reset state (the conversation is dead).
def on_disconnected(self) -> None: ...
class Session:
def __init__(
self,
backend_ws: str,
device_token: str,
device_id: str,
callbacks: Callbacks,
) -> None:
self._backend_ws = backend_ws
self._device_token = device_token
self._device_id = device_id
self._cb = callbacks
self._tools: dict[str, Callable[[dict], dict]] = {}
self._ws = None
self._loop: asyncio.AbstractEventLoop | None = None
def register_tool(self, name: str, handler: Callable[[dict], dict]) -> None:
self._tools[name] = handler
async def run_forever(self) -> None:
"""Connect, pump, reconnect with exponential backoff. Never returns
cleanly except on cancellation."""
# Lazy import so the module is importable without `websockets`.
import websockets
self._loop = asyncio.get_running_loop()
backoff = RECONNECT_MIN_S
while True:
try:
_log.info("connecting to %s", self._backend_ws)
async with websockets.connect(
self._backend_ws,
additional_headers={"Authorization": f"Bearer {self._device_token}"},
max_size=4 * 1024 * 1024,
) as ws:
self._ws = ws
backoff = RECONNECT_MIN_S
ping_task = asyncio.create_task(self._ping_loop(ws))
try:
await self._pump(ws)
finally:
ping_task.cancel()
self._ws = None
self._notify_disconnected()
except asyncio.CancelledError:
raise
except Exception as exc: # noqa: BLE001
_log.warning("session error: %s; reconnecting in %.1fs", exc, backoff)
self._notify_disconnected()
jitter = random.uniform(0, backoff * 0.1)
await asyncio.sleep(backoff + jitter)
backoff = min(backoff * 2.0, RECONNECT_MAX_S)
async def _pump(self, ws) -> None:
# 1. Send hello.
await ws.send(json.dumps({
"type": "hello",
"device_id": self._device_id,
"client_version": CLIENT_VERSION,
}))
# Track ws so the threadsafe shims work when _pump is exercised directly
# (i.e. without going through run_forever, as in unit tests).
self._ws = ws
try:
self._loop = asyncio.get_running_loop()
except RuntimeError:
pass
# 2. Pump.
while True:
frame = await ws.recv()
if isinstance(frame, (bytes, bytearray, memoryview)):
self._cb.on_assistant_audio(bytes(frame))
continue
try:
env = json.loads(frame)
except json.JSONDecodeError:
_log.warning("non-JSON text frame; dropping")
continue
await self._dispatch(env, ws)
async def _dispatch(self, env: dict, ws) -> None:
t = env.get("type")
if t == "hello_ack":
_log.info("hello_ack config=%s", env.get("config"))
elif t == "pong":
pass
elif t == "session_started":
self._cb.on_session_started(str(env.get("conversation_id", "")))
elif t == "assistant_done":
self._cb.on_assistant_done()
elif t == "session_ended":
self._cb.on_session_ended(str(env.get("reason", "unknown")))
elif t == "tool_call":
await self._handle_tool_call(env, ws)
elif t == "error":
_log.warning(
"backend error code=%s message=%s fatal=%s",
env.get("code"), env.get("message"), env.get("fatal"),
)
elif t == "config_updated":
_log.info("config_updated: %s", env.get("config"))
elif t == "debug":
_log.info("backend %s", env.get("message"))
else:
_log.info("unhandled envelope type=%s", t)
async def _handle_tool_call(self, env: dict, ws) -> None:
call_id = env.get("call_id", "")
name = env.get("name", "")
args = env.get("arguments") or {}
if isinstance(args, str):
try:
args = json.loads(args)
except json.JSONDecodeError:
args = {}
handler = self._tools.get(name)
if handler is None:
await ws.send(json.dumps({
"type": "tool_result",
"call_id": call_id,
"ok": False,
"error": f"unknown tool: {name}",
}))
return
try:
result = handler(args)
except Exception as exc: # noqa: BLE001
await ws.send(json.dumps({
"type": "tool_result",
"call_id": call_id,
"ok": False,
"error": str(exc),
}))
return
await ws.send(json.dumps({
"type": "tool_result",
"call_id": call_id,
"ok": True,
"result": result,
}))
def _notify_disconnected(self) -> None:
# Tolerant: Callbacks Protocol marks on_disconnected as part of the
# surface, but real callers (and tests) may omit it.
fn = getattr(self._cb, "on_disconnected", None)
if fn is None:
return
try:
fn()
except Exception as exc: # noqa: BLE001
_log.warning("on_disconnected raised: %s", exc)
async def _ping_loop(self, ws) -> None:
try:
while True:
await asyncio.sleep(PING_INTERVAL_S)
await ws.send(json.dumps({"type": "ping"}))
except asyncio.CancelledError:
pass
# ---- Public helpers used by main.py from the asyncio loop ----
#
# All of these are called fire-and-forget via call_soon_threadsafe, so they
# must NEVER raise — the asyncio loop has nowhere to surface the exception
# except as an unhandled-task-exception traceback in the journal.
async def _safe_send(self, payload) -> None:
if self._ws is None:
return
try:
await self._ws.send(payload)
except Exception as exc: # noqa: BLE001
# ConnectionClosed / ConnectionClosedOK / ConnectionClosedError all
# land here. The reconnect loop already handles WS lifecycle.
_log.debug("send dropped: %s", exc)
async def send_wake(self) -> None:
await self._safe_send(json.dumps({"type": "wake"}))
async def send_binary(self, pcm16: bytes) -> None:
await self._safe_send(pcm16)
async def send_playback_done(self, label: str) -> None:
await self._safe_send(json.dumps({"type": "playback_done", "for": label}))
async def send_cancel(self) -> None:
await self._safe_send(json.dumps({"type": "cancel"}))
def call_soon_threadsafe_send_binary(self, pcm16: bytes) -> None:
"""Thread-safe shim used by the bridge thread to queue an uplink send."""
if self._loop is None or self._ws is None:
return
self._loop.call_soon_threadsafe(
lambda: asyncio.create_task(self.send_binary(pcm16))
)
def call_soon_threadsafe_send_wake(self) -> None:
if self._loop is None or self._ws is None:
return
self._loop.call_soon_threadsafe(
lambda: asyncio.create_task(self.send_wake())
)
def call_soon_threadsafe_send_playback_done(self, label: str) -> None:
if self._loop is None or self._ws is None:
return
self._loop.call_soon_threadsafe(
lambda: asyncio.create_task(self.send_playback_done(label))
)
+77
View File
@@ -0,0 +1,77 @@
"""Assistant client state machine.
States flow: IDLE -> WAKE_PENDING -> LISTENING <-> ASSISTANT_SPEAKING
\\ /
session_ended
\\
IDLE_PENDING -> IDLE
"""
from enum import Enum, auto
from client.log import get_logger
_log = get_logger("state")
class State(Enum):
IDLE = auto()
WAKE_PENDING = auto()
LISTENING = auto()
ASSISTANT_SPEAKING = auto()
IDLE_PENDING = auto()
class StateMachine:
def __init__(self) -> None:
self._state = State.IDLE
@property
def state(self) -> State:
return self._state
@property
def wakeword_enabled(self) -> bool:
return self._state is State.IDLE
@property
def uplink_enabled(self) -> bool:
return self._state is State.LISTENING
def _transition(self, target: State) -> None:
if target is self._state:
return
_log.info("state %s -> %s", self._state.name, target.name)
self._state = target
def wake_fired(self) -> bool:
if self._state is not State.IDLE:
return False
self._transition(State.WAKE_PENDING)
return True
def session_started(self) -> None:
if self._state in (State.WAKE_PENDING, State.IDLE):
self._transition(State.LISTENING)
def assistant_audio_arrived(self) -> None:
if self._state is State.LISTENING:
self._transition(State.ASSISTANT_SPEAKING)
def assistant_done(self) -> None:
if self._state is State.ASSISTANT_SPEAKING:
self._transition(State.LISTENING)
def session_ended(self) -> None:
if self._state in (State.LISTENING, State.ASSISTANT_SPEAKING, State.WAKE_PENDING):
self._transition(State.IDLE_PENDING)
def sleep_played(self) -> None:
if self._state is State.IDLE_PENDING:
self._transition(State.IDLE)
def force_idle(self) -> None:
"""Unconditional reset to IDLE — used when the backend WS drops and
the conversation is dead regardless of what state we thought we were in.
Re-enables the wakeword detector."""
if self._state is not State.IDLE:
self._transition(State.IDLE)
View File
+8
View File
@@ -0,0 +1,8 @@
"""Shared pytest fixtures for the client test suite."""
import sys
from pathlib import Path
# Make the repo root importable so `import client.<module>` works without install.
REPO_ROOT = Path(__file__).resolve().parents[2]
if str(REPO_ROOT) not in sys.path:
sys.path.insert(0, str(REPO_ROOT))
+2
View File
@@ -0,0 +1,2 @@
[pytest]
asyncio_mode = auto

Some files were not shown because too many files have changed in this diff Show More