Compare commits
44 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 8380e3149e | |||
| 7ba9f0cf5e | |||
| ba709d15d5 | |||
| 9023509b01 | |||
| 4dc48bd9fd | |||
| 78373b0912 | |||
| 8f53f9daa9 | |||
| a45b25038a | |||
| 7dada5fdb9 | |||
| b078f9aa78 | |||
| 24b8acdf60 | |||
| a202725c39 | |||
| 8f197d1203 | |||
| 35a6e8febb | |||
| d3e101eb29 | |||
| 212ce06f07 | |||
| ebbe23c639 | |||
| 9325d21f9e | |||
| 13d8773704 | |||
| 99cfe60a2c | |||
| 58b8a446b1 | |||
| a83e9f93de | |||
| 68738e486d | |||
| 2701b8a483 | |||
| 645a7e3722 | |||
| 88339c51eb | |||
| a7e5e13ac8 | |||
| 751030dcef | |||
| e5ae228738 | |||
| b5be4c6914 | |||
| db5c4e59a4 | |||
| 720583de0a | |||
| b862bf6f0f | |||
| 3e9c245bb3 | |||
| 68e7e81312 | |||
| a1aa5023a2 | |||
| 2a208e81b2 | |||
| 8a7d01ac9e | |||
| 824a27520d | |||
| f2bb2d984b | |||
| 2c16a9485b | |||
| 1eb0383834 | |||
| cf055c8928 | |||
| 99a9950cee |
@@ -0,0 +1,13 @@
|
||||
**/bin
|
||||
**/obj
|
||||
**/.vs
|
||||
.git
|
||||
.gitignore
|
||||
.vscode
|
||||
.idea
|
||||
docs
|
||||
tests
|
||||
backend.tests
|
||||
*.md
|
||||
.dockerignore
|
||||
Dockerfile
|
||||
+12
@@ -3,3 +3,15 @@ __pycache__/
|
||||
.venv/
|
||||
*.wav
|
||||
.openwakeword-cache/
|
||||
|
||||
# .NET build outputs (don't catch root-level bin/ scripts)
|
||||
backend/bin/
|
||||
backend/obj/
|
||||
backend.tests/bin/
|
||||
backend.tests/obj/
|
||||
|
||||
# SQLite dev database (created on first run)
|
||||
backend/*.db
|
||||
backend/*.db-shm
|
||||
backend/*.db-wal
|
||||
deploy.json
|
||||
|
||||
@@ -0,0 +1,48 @@
|
||||
|
||||
Microsoft Visual Studio Solution File, Format Version 12.00
|
||||
# Visual Studio Version 17
|
||||
VisualStudioVersion = 17.0.31903.59
|
||||
MinimumVisualStudioVersion = 10.0.40219.1
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "backend", "backend\backend.csproj", "{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}"
|
||||
EndProject
|
||||
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "backend.tests", "backend.tests\backend.tests.csproj", "{63423BEA-4ECF-45CF-B0E8-611AC65AE463}"
|
||||
EndProject
|
||||
Global
|
||||
GlobalSection(SolutionConfigurationPlatforms) = preSolution
|
||||
Debug|Any CPU = Debug|Any CPU
|
||||
Debug|x64 = Debug|x64
|
||||
Debug|x86 = Debug|x86
|
||||
Release|Any CPU = Release|Any CPU
|
||||
Release|x64 = Release|x64
|
||||
Release|x86 = Release|x86
|
||||
EndGlobalSection
|
||||
GlobalSection(ProjectConfigurationPlatforms) = postSolution
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x64.ActiveCfg = Release|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x64.Build.0 = Release|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x86.ActiveCfg = Release|Any CPU
|
||||
{98CFCD4D-61C0-4FCA-BB6A-C6F68D5F4E40}.Release|x86.Build.0 = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|Any CPU.Build.0 = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x64.ActiveCfg = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x64.Build.0 = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x86.ActiveCfg = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Debug|x86.Build.0 = Debug|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|Any CPU.ActiveCfg = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|Any CPU.Build.0 = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x64.ActiveCfg = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x64.Build.0 = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x86.ActiveCfg = Release|Any CPU
|
||||
{63423BEA-4ECF-45CF-B0E8-611AC65AE463}.Release|x86.Build.0 = Release|Any CPU
|
||||
EndGlobalSection
|
||||
GlobalSection(SolutionProperties) = preSolution
|
||||
HideSolutionNode = FALSE
|
||||
EndGlobalSection
|
||||
EndGlobal
|
||||
@@ -24,7 +24,3 @@ Copy a file to the Pi:
|
||||
```sh
|
||||
sshpass -p 'assistant' scp <local> pi@192.168.50.115:<remote>
|
||||
```
|
||||
|
||||
## Prototype + probe findings
|
||||
|
||||
`findings.md` is the canonical record of what we proved during the prototype + C# probe phase. Read it before starting any audio, wakeword, or Pi-client work — it captures hardware constraints (USB Speaker Phone is 48 kHz-native, mic and speaker are the same device), the .NET-on-Linux env-var gotcha (`Environment.SetEnvironmentVariable` does NOT reach libc `getenv()`; you must P/Invoke `setenv` for libportaudio and onnxruntime to see env knobs), and the punch list of items to redesign before the main assistant. The Python prototypes live at `tests/01-record-play/` and `tests/02-wakeword/`; the C# Test 1 port at `tests/01-record-play-cs/` (deployed via `bin/probe-cs`).
|
||||
|
||||
+30
@@ -0,0 +1,30 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
|
||||
# ---- Backend build ----
|
||||
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
|
||||
WORKDIR /src
|
||||
COPY backend/backend.csproj backend/
|
||||
RUN dotnet restore backend/backend.csproj
|
||||
COPY backend/ backend/
|
||||
RUN dotnet publish backend/backend.csproj -c Release -o /publish
|
||||
|
||||
# ---- Client bundle ----
|
||||
FROM alpine:3.20 AS bundle
|
||||
WORKDIR /src
|
||||
RUN apk add --no-cache tar gzip
|
||||
COPY client/ client/
|
||||
COPY requirements.txt requirements-nodeps.txt ./
|
||||
RUN tar czf /client.tar.gz client/ requirements.txt requirements-nodeps.txt
|
||||
|
||||
# ---- Final ----
|
||||
FROM mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
WORKDIR /app
|
||||
COPY --from=build /publish/ ./
|
||||
RUN mkdir -p ./wwwroot && mkdir -p /data /keys
|
||||
COPY --from=bundle /client.tar.gz ./wwwroot/client.tar.gz
|
||||
|
||||
ENV ASPNETCORE_URLS=http://+:8080
|
||||
ENV ConnectionStrings__Default="Data Source=/data/assistant.db"
|
||||
ENV Install__ClientTarballPath="/app/wwwroot/client.tar.gz"
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["dotnet", "backend.dll"]
|
||||
@@ -0,0 +1,146 @@
|
||||
# Smart Assistant
|
||||
|
||||
Voice assistant on a Raspberry Pi talking to the OpenAI Realtime API via an
|
||||
ASP.NET backend deployed on Coolify.
|
||||
|
||||
See `docs/superpowers/specs/2026-06-11-smart-assistant-design.md` for the full
|
||||
design. Implementation plans:
|
||||
|
||||
- Plan 1 — `docs/superpowers/plans/2026-06-11-smart-assistant-backend-foundation.md`
|
||||
(Identity, devices, pairing, install endpoints).
|
||||
- Plan 2 — `docs/superpowers/plans/2026-06-11-smart-assistant-device-hub-realtime-tools.md`
|
||||
(device WebSocket hub, OpenAI Realtime relay, tools registry).
|
||||
|
||||
## Repo layout
|
||||
|
||||
- `backend/` — ASP.NET 9 backend (Identity, Devices, Pairing, Install).
|
||||
- `backend.tests/` — xUnit integration + unit tests via `WebApplicationFactory`.
|
||||
- `client/` — Python client. **Plan 1 ships placeholders only**; the full
|
||||
wakeword + Realtime client lands in Plan 3.
|
||||
- `Dockerfile` + `docker-compose.yml` — multi-stage build; produces one image
|
||||
with the published ASP.NET app + a bundled `client.tar.gz` in
|
||||
`wwwroot/client.tar.gz`.
|
||||
- `deploy.json` — Coolify config (gitignored; contains the OpenAI key).
|
||||
- `docs/superpowers/` — specs and plans.
|
||||
|
||||
## Local development
|
||||
|
||||
```sh
|
||||
dotnet test # all backend tests
|
||||
dotnet run --project backend # starts on http://localhost:5252
|
||||
```
|
||||
|
||||
Hit `http://localhost:5252/health` to verify it's up.
|
||||
|
||||
## Smoke test against a live deploy
|
||||
|
||||
The full pairing flow can be exercised with `curl` — no UI needed yet.
|
||||
|
||||
```sh
|
||||
DOMAIN="https://assistant.volcanic.tes.gd" # or your domain
|
||||
COOKIE=$(mktemp)
|
||||
|
||||
# Register the first user (becomes Admin automatically) and sign in
|
||||
curl -sf -c "$COOKIE" -X POST "$DOMAIN/api/auth/register" \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"email":"you@example.com","password":"Passw0rd!"}'
|
||||
curl -sf -c "$COOKIE" -b "$COOKIE" -X POST "$DOMAIN/api/auth/login" \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"email":"you@example.com","password":"Passw0rd!"}'
|
||||
|
||||
# Generate a pairing code (auth required)
|
||||
CODE=$(curl -sf -c "$COOKIE" -b "$COOKIE" -X POST "$DOMAIN/api/pair-code" \
|
||||
-H 'content-type: application/json' \
|
||||
-d '{"name":"kitchen"}' \
|
||||
| python3 -c 'import sys,json; print(json.load(sys.stdin)["code"])')
|
||||
echo "code=$CODE"
|
||||
|
||||
# Pair as a device (public endpoint) and capture the token
|
||||
curl -sf -X POST "$DOMAIN/api/pair" \
|
||||
-H 'content-type: application/json' \
|
||||
-d "{\"code\":\"$CODE\",\"hostname\":\"smoke\",\"client_version\":\"0.1\"}" \
|
||||
| python3 -m json.tool
|
||||
|
||||
# Verify the install script and client bundle are served
|
||||
curl -sf "$DOMAIN/install.sh" | head -5
|
||||
curl -sIf "$DOMAIN/client.tar.gz"
|
||||
```
|
||||
|
||||
Expected: `{"ok":true}` on `/health`, JSON with `device_id`/`device_token`/`backend_ws`
|
||||
from `/api/pair`, the bash shebang from `/install.sh`, and a `200 OK` with
|
||||
`Content-Type: application/gzip` for the bundle.
|
||||
|
||||
> **Plan 1 ships a placeholder Python bundle** — running the bundled
|
||||
> `client.main` will exit with the placeholder message. Plan 3 replaces it
|
||||
> with the real wakeword + Realtime client.
|
||||
|
||||
## Deploying
|
||||
|
||||
Use the Coolify v4 API. The `deploy.json` file (gitignored) carries the
|
||||
project + server UUIDs and the OpenAI key.
|
||||
|
||||
```sh
|
||||
APP=$(python3 -c "import json; print(json.load(open('deploy.json'))['app']['uuid'])")
|
||||
curl -sf -X POST "$COOLIFY_URL/api/v1/deploy?uuid=$APP" \
|
||||
-H "Authorization: Bearer $COOLIFY_KEY"
|
||||
```
|
||||
|
||||
Fire-and-forget; the API returns a `deployment_uuid` immediately. Don't poll
|
||||
(per the global Coolify rule in `CLAUDE.md`).
|
||||
|
||||
## Plan 2: device hub + Realtime relay
|
||||
|
||||
- `wss://<domain>/device` — bearer-token WebSocket. Pair a device first to get a token.
|
||||
- Handshake: client sends `{type:"hello", device_id, client_version}`, backend
|
||||
replies `hello_ack` with per-device config (voice, model, system prompt, idle
|
||||
timeout, enabled tools).
|
||||
- Heartbeat: client sends `{type:"ping"}` every 15 s, backend replies `pong`
|
||||
and updates `Devices.LastSeenAt`.
|
||||
- Wake: `{type:"wake"}` opens a Realtime session against OpenAI; backend emits
|
||||
`session_started` (with `conversation_id`), forwards `response.audio.delta`
|
||||
as binary frames to the device, persists `response.audio_transcript.done` as
|
||||
an assistant turn, persists `conversation.item.input_audio_transcription.completed`
|
||||
as a user turn, and emits `assistant_done` after each `response.done`.
|
||||
- Tools shipped (per-device toggle via `DeviceConfig.EnabledToolsJson`):
|
||||
- `get_current_time` (server-side, returns `{"now":"<ISO-UTC>"}`),
|
||||
- `end_session` (closes the session after the current `response.done`
|
||||
completes; emits `session_ended(reason="tool")`),
|
||||
- `set_volume` (round-trips a `tool_call`/`tool_result` envelope to the Pi;
|
||||
the Pi side will run `amixer` in Plan 3).
|
||||
- Idle timeout: server-side. With no `input_audio_buffer.speech_started` upstream
|
||||
event for `IdleTimeoutSeconds`, the relay emits `session_ended(reason="idle")`.
|
||||
|
||||
The Python Pi client (Plan 3) is the production consumer of this surface. A
|
||||
quick handshake-only smoke against the live deploy:
|
||||
|
||||
```sh
|
||||
DOMAIN="https://assistant.volcanic.tes.gd"
|
||||
# Reuse the curl recipe above to log in, generate a pair-code, and pair
|
||||
# a "smoke" device — capture $TOKEN from the /api/pair response.
|
||||
python3 - <<PY
|
||||
import asyncio, json, websockets
|
||||
URL, TOKEN = "wss://assistant.volcanic.tes.gd/device", "$TOKEN"
|
||||
async def main():
|
||||
async with websockets.connect(
|
||||
URL, additional_headers={"Authorization": f"Bearer {TOKEN}"}
|
||||
) as ws:
|
||||
await ws.send(json.dumps({
|
||||
"type":"hello","device_id":"00000000-0000-0000-0000-000000000000",
|
||||
"client_version":"0.1"}))
|
||||
print("ack:", json.loads(await ws.recv())["type"])
|
||||
await ws.send(json.dumps({"type":"ping"}))
|
||||
print("pong:", json.loads(await ws.recv())["type"])
|
||||
asyncio.run(main())
|
||||
PY
|
||||
```
|
||||
|
||||
Don't send `wake` from the smoke — it would open a real OpenAI session and
|
||||
burn budget. Plan 3 exercises that end-to-end.
|
||||
|
||||
## What's next
|
||||
|
||||
- **Plan 3** — full Python client on the Pi (wakeword, VAD, audio plumbing,
|
||||
state machine, install + pair CLIs). This is the production caller of
|
||||
`/device` and the integration test for everything above.
|
||||
- **Plan 4** — React management UI for the admin dashboard, per-device
|
||||
config editor, and conversation history viewer.
|
||||
@@ -0,0 +1,65 @@
|
||||
using backend.Realtime;
|
||||
using Microsoft.AspNetCore.Hosting;
|
||||
using Microsoft.AspNetCore.Mvc.Testing;
|
||||
using Microsoft.AspNetCore.TestHost;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class ApiFactory : WebApplicationFactory<Program>
|
||||
{
|
||||
private readonly string _dbPath = Path.Combine(Path.GetTempPath(), $"assistant-test-{Guid.NewGuid():N}.db");
|
||||
|
||||
protected override void ConfigureWebHost(IWebHostBuilder builder)
|
||||
{
|
||||
builder.UseEnvironment("Test");
|
||||
builder.ConfigureAppConfiguration((_, cfg) =>
|
||||
{
|
||||
cfg.AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
["ConnectionStrings:Default"] = $"Data Source={_dbPath}",
|
||||
["OPENAI_API_KEY"] = "test-key",
|
||||
});
|
||||
});
|
||||
|
||||
builder.ConfigureTestServices(services =>
|
||||
{
|
||||
services.AddScoped<RealtimeSessionFactory, FakeRealtimeSessionFactory>();
|
||||
});
|
||||
}
|
||||
|
||||
protected override void Dispose(bool disposing)
|
||||
{
|
||||
base.Dispose(disposing);
|
||||
SqliteConnection.ClearAllPools();
|
||||
foreach (var suffix in new[] { "", "-shm", "-wal", "-journal" })
|
||||
{
|
||||
var p = _dbPath + suffix;
|
||||
try { if (File.Exists(p)) File.Delete(p); } catch { /* best effort */ }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public class FakeRealtimeSessionFactory(
|
||||
OpenAIKeyProvider keys,
|
||||
backend.Tools.ToolRegistry registry,
|
||||
backend.Conversations.ConversationLog log,
|
||||
ILoggerFactory loggerFactory)
|
||||
: RealtimeSessionFactory(keys, registry, log, loggerFactory)
|
||||
{
|
||||
public static ScriptedRealtimeUpstream Upstream { get; set; } = new();
|
||||
|
||||
public override Task<RealtimeSession> CreateAsync(
|
||||
Guid deviceId, RealtimeSettings cfg, ISessionOutput output,
|
||||
backend.Tools.IDeviceChannel channel, CancellationToken ct)
|
||||
{
|
||||
Upstream.Push(new System.Text.Json.Nodes.JsonObject { ["type"] = "session.updated" });
|
||||
var s = new RealtimeSession(
|
||||
deviceId, Upstream, output, log, registry, cfg, channel,
|
||||
loggerFactory.CreateLogger<RealtimeSession>());
|
||||
return Task.FromResult(s);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class AuthTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Register_then_login_then_me_returns_user()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.CreateClient();
|
||||
|
||||
var reg = await client.PostAsJsonAsync("/api/auth/register",
|
||||
new { email = "first@example.com", password = "Passw0rd!" });
|
||||
reg.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var login = await client.PostAsJsonAsync("/api/auth/login",
|
||||
new { email = "first@example.com", password = "Passw0rd!" });
|
||||
login.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var me = await client.GetAsync("/api/auth/me");
|
||||
me.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
(await me.Content.ReadAsStringAsync()).Should().Contain("first@example.com");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Me_without_login_returns_401()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.CreateClient();
|
||||
var res = await client.GetAsync("/api/auth/me");
|
||||
res.StatusCode.Should().Be(HttpStatusCode.Unauthorized);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task First_user_is_admin()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.CreateClient();
|
||||
await client.PostAsJsonAsync("/api/auth/register",
|
||||
new { email = "admin@example.com", password = "Passw0rd!" });
|
||||
await client.PostAsJsonAsync("/api/auth/login",
|
||||
new { email = "admin@example.com", password = "Passw0rd!" });
|
||||
|
||||
var me = await client.GetAsync("/api/auth/me");
|
||||
(await me.Content.ReadAsStringAsync()).Should().Contain("\"isAdmin\":true");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
using backend.Conversations;
|
||||
using backend.Data;
|
||||
using FluentAssertions;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class ConversationLogTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
private async Task<(AppDbContext db, ConversationLog log, Guid deviceId)> ArrangeAsync()
|
||||
{
|
||||
var scope = _factory.Services.CreateScope();
|
||||
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var deviceId = Guid.NewGuid();
|
||||
return (db, log, deviceId);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Start_creates_conversation_row()
|
||||
{
|
||||
var (db, log, deviceId) = await ArrangeAsync();
|
||||
|
||||
var convo = await log.StartAsync(deviceId, default);
|
||||
|
||||
var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
|
||||
row.DeviceId.Should().Be(deviceId);
|
||||
row.EndedAt.Should().BeNull();
|
||||
row.EndReason.Should().Be(EndReason.Unknown);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Turns_are_persisted_with_monotonic_index_per_conversation()
|
||||
{
|
||||
var (db, log, deviceId) = await ArrangeAsync();
|
||||
var convo = await log.StartAsync(deviceId, default);
|
||||
|
||||
await log.AppendUserAsync(convo.Id, "hi", default);
|
||||
await log.AppendAssistantAsync(convo.Id, "hello there", default);
|
||||
await log.AppendToolAsync(convo.Id, "get_current_time", "{}", "{\"now\":\"2026-06-11T00:00:00Z\"}", default);
|
||||
|
||||
var turns = await db.Turns
|
||||
.Where(t => t.ConversationId == convo.Id)
|
||||
.OrderBy(t => t.Index)
|
||||
.ToListAsync();
|
||||
|
||||
turns.Should().HaveCount(3);
|
||||
turns.Select(t => t.Index).Should().Equal(0, 1, 2);
|
||||
turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant, TurnRole.Tool);
|
||||
turns[0].Text.Should().Be("hi");
|
||||
turns[1].Text.Should().Be("hello there");
|
||||
turns[2].ToolName.Should().Be("get_current_time");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task End_sets_ended_at_and_reason()
|
||||
{
|
||||
var (db, log, deviceId) = await ArrangeAsync();
|
||||
var convo = await log.StartAsync(deviceId, default);
|
||||
|
||||
await log.EndAsync(convo.Id, EndReason.Idle, default);
|
||||
|
||||
var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
|
||||
row.EndedAt.Should().NotBeNull();
|
||||
row.EndReason.Should().Be(EndReason.Idle);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
using System.Net.Http.Json;
|
||||
using System.Net.WebSockets;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class DeviceHubHandshakeTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
private async Task<string> PairAndGetTokenAsync(string email)
|
||||
{
|
||||
var http = _factory.CreateClient();
|
||||
await http.PostAsJsonAsync("/api/auth/register",
|
||||
new { email, password = "Passw0rd!" });
|
||||
await http.PostAsJsonAsync("/api/auth/login",
|
||||
new { email, password = "Passw0rd!" });
|
||||
var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "ws-test" });
|
||||
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
|
||||
|
||||
var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "smoke", client_version = "0.1" });
|
||||
var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
|
||||
return pairBody!["device_token"].ToString()!;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Connect_without_token_fails_upgrade()
|
||||
{
|
||||
var client = _factory.Server.CreateWebSocketClient();
|
||||
var uri = new Uri(_factory.Server.BaseAddress, "device");
|
||||
var act = async () => await client.ConnectAsync(uri, CancellationToken.None);
|
||||
await act.Should().ThrowAsync<Exception>();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Hello_ack_includes_config()
|
||||
{
|
||||
var token = await PairAndGetTokenAsync("ws1@example.com");
|
||||
var client = _factory.Server.CreateWebSocketClient();
|
||||
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
|
||||
|
||||
var uri = new Uri(_factory.Server.BaseAddress, "device");
|
||||
var ws = await client.ConnectAsync(uri, CancellationToken.None);
|
||||
|
||||
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
|
||||
var ack = await ReceiveJson(ws);
|
||||
ack.GetProperty("type").GetString().Should().Be("hello_ack");
|
||||
ack.GetProperty("config").GetProperty("voice").GetString().Should().NotBeNullOrEmpty();
|
||||
ack.GetProperty("config").GetProperty("model").GetString().Should().NotBeNullOrEmpty();
|
||||
ack.GetProperty("config").GetProperty("enabled_tools").GetArrayLength().Should().BeGreaterThan(0);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Ping_replies_with_pong()
|
||||
{
|
||||
var token = await PairAndGetTokenAsync("ws2@example.com");
|
||||
var client = _factory.Server.CreateWebSocketClient();
|
||||
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
|
||||
var uri = new Uri(_factory.Server.BaseAddress, "device");
|
||||
var ws = await client.ConnectAsync(uri, CancellationToken.None);
|
||||
|
||||
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
|
||||
await ReceiveJson(ws);
|
||||
await SendJson(ws, new { type = "ping" });
|
||||
var pong = await ReceiveJson(ws);
|
||||
pong.GetProperty("type").GetString().Should().Be("pong");
|
||||
}
|
||||
|
||||
private static async Task SendJson(WebSocket ws, object payload)
|
||||
{
|
||||
var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
|
||||
await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
|
||||
}
|
||||
|
||||
private static async Task<JsonElement> ReceiveJson(WebSocket ws)
|
||||
{
|
||||
var buf = new byte[8192];
|
||||
using var stream = new MemoryStream();
|
||||
while (true)
|
||||
{
|
||||
var r = await ws.ReceiveAsync(buf, CancellationToken.None);
|
||||
stream.Write(buf, 0, r.Count);
|
||||
if (r.EndOfMessage) break;
|
||||
}
|
||||
return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
using System.Net.Http.Json;
|
||||
using System.Net.WebSockets;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Nodes;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
[Collection("WakeFlow")] // tests touch FakeRealtimeSessionFactory.Upstream static — serialize
|
||||
public class DeviceHubWakeFlowTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
private async Task<string> PairAndGetTokenAsync(string email)
|
||||
{
|
||||
var http = _factory.CreateClient();
|
||||
await http.PostAsJsonAsync("/api/auth/register",
|
||||
new { email, password = "Passw0rd!" });
|
||||
await http.PostAsJsonAsync("/api/auth/login",
|
||||
new { email, password = "Passw0rd!" });
|
||||
var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "wake-test" });
|
||||
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
|
||||
var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "smoke", client_version = "0.1" });
|
||||
var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
|
||||
return pairBody!["device_token"].ToString()!;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Wake_opens_session_started_then_assistant_done_then_session_ended_on_cancel()
|
||||
{
|
||||
FakeRealtimeSessionFactory.Upstream = new ScriptedRealtimeUpstream();
|
||||
var token = await PairAndGetTokenAsync("wake1@example.com");
|
||||
var client = _factory.Server.CreateWebSocketClient();
|
||||
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
|
||||
var uri = new Uri(_factory.Server.BaseAddress, "device");
|
||||
var ws = await client.ConnectAsync(uri, CancellationToken.None);
|
||||
|
||||
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
|
||||
await ReceiveJson(ws); // ack
|
||||
|
||||
await SendJson(ws, new { type = "wake", at = 0L });
|
||||
var started = await ReceiveJson(ws);
|
||||
started.GetProperty("type").GetString().Should().Be("session_started");
|
||||
|
||||
FakeRealtimeSessionFactory.Upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.audio_transcript.done",
|
||||
["transcript"] = "hello!",
|
||||
});
|
||||
FakeRealtimeSessionFactory.Upstream.Push(new JsonObject { ["type"] = "response.done" });
|
||||
|
||||
var done = await ReceiveJson(ws);
|
||||
done.GetProperty("type").GetString().Should().Be("assistant_done");
|
||||
|
||||
await SendJson(ws, new { type = "cancel" });
|
||||
var ended = await ReceiveJson(ws);
|
||||
ended.GetProperty("type").GetString().Should().Be("session_ended");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Binary_uplink_after_wake_reaches_upstream_as_input_audio_buffer_append()
|
||||
{
|
||||
FakeRealtimeSessionFactory.Upstream = new ScriptedRealtimeUpstream();
|
||||
var token = await PairAndGetTokenAsync("wake2@example.com");
|
||||
var client = _factory.Server.CreateWebSocketClient();
|
||||
client.ConfigureRequest = req => req.Headers["Authorization"] = $"Bearer {token}";
|
||||
var uri = new Uri(_factory.Server.BaseAddress, "device");
|
||||
var ws = await client.ConnectAsync(uri, CancellationToken.None);
|
||||
|
||||
await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
|
||||
await ReceiveJson(ws);
|
||||
await SendJson(ws, new { type = "wake", at = 0L });
|
||||
await ReceiveJson(ws); // session_started
|
||||
|
||||
var frame = new byte[3840];
|
||||
for (int i = 0; i < frame.Length; i++) frame[i] = 0x55;
|
||||
await ws.SendAsync(frame, WebSocketMessageType.Binary, true, CancellationToken.None);
|
||||
|
||||
// Allow time for the pump's 250 ms poll + send.
|
||||
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(2);
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
if (FakeRealtimeSessionFactory.Upstream.Sent.Any(e =>
|
||||
(string?)e["type"] == "input_audio_buffer.append")) break;
|
||||
await Task.Delay(50);
|
||||
}
|
||||
FakeRealtimeSessionFactory.Upstream.Sent
|
||||
.Any(e => (string?)e["type"] == "input_audio_buffer.append")
|
||||
.Should().BeTrue();
|
||||
|
||||
await SendJson(ws, new { type = "cancel" });
|
||||
await ReceiveJson(ws);
|
||||
}
|
||||
|
||||
private static async Task SendJson(WebSocket ws, object payload)
|
||||
{
|
||||
var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
|
||||
await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
|
||||
}
|
||||
|
||||
private static async Task<JsonElement> ReceiveJson(WebSocket ws)
|
||||
{
|
||||
var buf = new byte[16384];
|
||||
using var stream = new MemoryStream();
|
||||
while (true)
|
||||
{
|
||||
var r = await ws.ReceiveAsync(buf, CancellationToken.None);
|
||||
if (r.MessageType == WebSocketMessageType.Binary) { stream.Write(buf, 0, r.Count); continue; }
|
||||
stream.Write(buf, 0, r.Count);
|
||||
if (r.EndOfMessage) break;
|
||||
}
|
||||
return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
using System.Net.WebSockets;
|
||||
using backend.DeviceHub;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class DeviceRegistryTests
|
||||
{
|
||||
[Fact]
|
||||
public void Register_unregister_round_trip()
|
||||
{
|
||||
var reg = new DeviceRegistry();
|
||||
var id = Guid.NewGuid();
|
||||
var dev = new ActiveDevice(id, new ClientWebSocket());
|
||||
reg.TryRegister(dev).Should().BeTrue();
|
||||
reg.TryRegister(dev).Should().BeFalse();
|
||||
reg.IsOnline(id).Should().BeTrue();
|
||||
reg.OnlineCount.Should().Be(1);
|
||||
|
||||
reg.Unregister(id).Should().BeTrue();
|
||||
reg.IsOnline(id).Should().BeFalse();
|
||||
reg.OnlineCount.Should().Be(0);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
using backend.Devices;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class DeviceTokenServiceTests
|
||||
{
|
||||
private readonly DeviceTokenService _svc = new();
|
||||
|
||||
[Fact]
|
||||
public void Generate_returns_url_safe_token_of_expected_length()
|
||||
{
|
||||
var (plaintext, hash) = _svc.Generate();
|
||||
plaintext.Should().MatchRegex("^[A-Za-z0-9_-]+$");
|
||||
plaintext.Length.Should().BeGreaterThanOrEqualTo(40);
|
||||
hash.Should().NotBeNullOrEmpty();
|
||||
hash.Should().NotBe(plaintext);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Hash_is_deterministic_for_same_input()
|
||||
{
|
||||
var (plaintext, hash) = _svc.Generate();
|
||||
_svc.Hash(plaintext).Should().Be(hash);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Verify_accepts_matching_plaintext_and_rejects_others()
|
||||
{
|
||||
var (plaintext, hash) = _svc.Generate();
|
||||
_svc.Verify(plaintext, hash).Should().BeTrue();
|
||||
_svc.Verify(plaintext + "x", hash).Should().BeFalse();
|
||||
_svc.Verify("totally-different", hash).Should().BeFalse();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class DevicesEndpointsTests
|
||||
{
|
||||
private static async Task<(HttpClient signedIn, string code)> SetupWithPairCode(ApiFactory factory, string email = "owner@example.com")
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
await client.PostAsJsonAsync("/api/auth/register",
|
||||
new { email, password = "Passw0rd!" });
|
||||
await client.PostAsJsonAsync("/api/auth/login",
|
||||
new { email, password = "Passw0rd!" });
|
||||
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
|
||||
var body = await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
|
||||
return (client, body!["code"].ToString()!);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task User_sees_only_their_own_devices()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var (alice, codeA) = await SetupWithPairCode(factory, "alice@example.com");
|
||||
await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code = codeA, hostname = "rpi", client_version = "0.1" });
|
||||
|
||||
var (bob, codeB) = await SetupWithPairCode(factory, "bob@example.com");
|
||||
await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code = codeB, hostname = "rpi", client_version = "0.1" });
|
||||
|
||||
var aliceList = await alice.GetStringAsync("/api/devices");
|
||||
var bobList = await bob.GetStringAsync("/api/devices");
|
||||
|
||||
aliceList.Should().Contain("kitchen");
|
||||
bobList.Should().Contain("kitchen");
|
||||
aliceList.Should().NotBe(bobList);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Rename_device_updates_name()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var (client, code) = await SetupWithPairCode(factory);
|
||||
await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "rpi", client_version = "0.1" });
|
||||
|
||||
var listJson = await client.GetFromJsonAsync<List<Dictionary<string, object>>>("/api/devices");
|
||||
var id = listJson![0]["id"].ToString();
|
||||
|
||||
var patchRes = await client.PatchAsJsonAsync($"/api/devices/{id}", new { name = "living-room" });
|
||||
patchRes.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var after = await client.GetStringAsync($"/api/devices/{id}");
|
||||
after.Should().Contain("living-room");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Revoke_device_sets_is_revoked_true()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var (client, code) = await SetupWithPairCode(factory);
|
||||
await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "rpi", client_version = "0.1" });
|
||||
|
||||
var listJson = await client.GetFromJsonAsync<List<Dictionary<string, object>>>("/api/devices");
|
||||
var id = listJson![0]["id"].ToString();
|
||||
|
||||
var del = await client.DeleteAsync($"/api/devices/{id}");
|
||||
del.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var after = await client.GetStringAsync($"/api/devices/{id}");
|
||||
after.Should().Contain("\"isRevoked\":true");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
using System.Text.Json;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class EndSessionToolTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Returns_ok_true()
|
||||
{
|
||||
var tool = new EndSessionTool();
|
||||
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());
|
||||
|
||||
var res = await tool.ExecuteAsync(
|
||||
new ToolInvocation("call_1", JsonDocument.Parse("{}").RootElement), ctx, default);
|
||||
|
||||
res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Name_and_runs_during_response_match_spec()
|
||||
{
|
||||
var tool = new EndSessionTool();
|
||||
tool.Name.Should().Be("end_session");
|
||||
tool.RunsDuringResponse.Should().BeFalse();
|
||||
}
|
||||
|
||||
private class NoopChannel : IDeviceChannel
|
||||
{
|
||||
public Task<JsonElement> CallPiToolAsync(string n, JsonElement a, CancellationToken ct)
|
||||
=> throw new NotSupportedException();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Text.Json;
|
||||
using backend.DeviceHub;
|
||||
using backend.Tools;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class FakeDeviceChannel : IDeviceChannel, IDeviceClient
|
||||
{
|
||||
public List<HubEnvelope> SentEnvelopes { get; } = new();
|
||||
public List<byte[]> SentBinary { get; } = new();
|
||||
public ConcurrentQueue<JsonElement> PiToolReplies { get; } = new();
|
||||
|
||||
public Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
|
||||
{
|
||||
SentEnvelopes.Add(envelope);
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
|
||||
{
|
||||
SentBinary.Add(bytes.ToArray());
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
|
||||
{
|
||||
if (PiToolReplies.TryDequeue(out var reply))
|
||||
return Task.FromResult(reply);
|
||||
return Task.FromResult(JsonDocument.Parse("""{"ok":true}""").RootElement);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
using System.Text.Json;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class GetCurrentTimeToolTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Returns_now_field_with_iso_utc_timestamp()
|
||||
{
|
||||
var tool = new GetCurrentTimeTool();
|
||||
var args = JsonDocument.Parse("{}").RootElement;
|
||||
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());
|
||||
|
||||
var res = await tool.ExecuteAsync(new ToolInvocation("call_1", args), ctx, default);
|
||||
|
||||
res.Output.TryGetProperty("now", out var now).Should().BeTrue();
|
||||
now.GetString().Should().MatchRegex(@"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}");
|
||||
now.GetString()!.Should().EndWith("Z");
|
||||
}
|
||||
|
||||
private class NoopChannel : IDeviceChannel
|
||||
{
|
||||
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
|
||||
=> throw new NotSupportedException();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
using System.Net;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class HealthTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Health_returns_ok()
|
||||
{
|
||||
var client = _factory.CreateClient();
|
||||
var res = await client.GetAsync("/health");
|
||||
res.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
(await res.Content.ReadAsStringAsync()).Should().Contain("\"ok\":true");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
using System.Net;
|
||||
using FluentAssertions;
|
||||
using Microsoft.AspNetCore.Hosting;
|
||||
using Microsoft.Extensions.Configuration;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class InstallEndpointsTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Install_sh_returns_shell_script_with_request_host()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.CreateClient();
|
||||
var res = await client.GetAsync("/install.sh");
|
||||
res.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
res.Content.Headers.ContentType!.MediaType.Should().Be("text/x-shellscript");
|
||||
|
||||
var body = await res.Content.ReadAsStringAsync();
|
||||
body.Should().StartWith("#!/usr/bin/env bash");
|
||||
body.Should().Contain("BACKEND_URL=\"http://localhost\"");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Client_tarball_returns_octet_stream()
|
||||
{
|
||||
var tmp = Path.Combine(Path.GetTempPath(), $"client-test-{Guid.NewGuid():N}.tar.gz");
|
||||
File.WriteAllBytes(tmp, new byte[] { 0x1f, 0x8b }); // tiny gzip-ish marker
|
||||
|
||||
try
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.WithWebHostBuilder(b =>
|
||||
{
|
||||
b.ConfigureAppConfiguration((_, cfg) =>
|
||||
{
|
||||
cfg.AddInMemoryCollection(new Dictionary<string, string?>
|
||||
{
|
||||
["Install:ClientTarballPath"] = tmp,
|
||||
});
|
||||
});
|
||||
}).CreateClient();
|
||||
|
||||
var res = await client.GetAsync("/client.tar.gz");
|
||||
res.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
res.Content.Headers.ContentType!.MediaType.Should().Be("application/gzip");
|
||||
(await res.Content.ReadAsByteArrayAsync()).Length.Should().Be(2);
|
||||
}
|
||||
finally
|
||||
{
|
||||
if (File.Exists(tmp)) File.Delete(tmp);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
using backend.Install;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class InstallScriptBuilderTests
|
||||
{
|
||||
[Fact]
|
||||
public void Build_inlines_the_backend_url_into_the_script()
|
||||
{
|
||||
var script = new InstallScriptBuilder().Build("https://example.test");
|
||||
script.Should().Contain("BACKEND_URL=\"https://example.test\"");
|
||||
script.Should().Contain("#!/usr/bin/env bash");
|
||||
script.Should().Contain("$BACKEND_URL/client.tar.gz");
|
||||
script.Should().Contain("client.pair");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Json;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class PairingEndpointsTests
|
||||
{
|
||||
private static async Task<HttpClient> SignedInClient(ApiFactory factory, string email = "owner@example.com")
|
||||
{
|
||||
var client = factory.CreateClient();
|
||||
await client.PostAsJsonAsync("/api/auth/register",
|
||||
new { email, password = "Passw0rd!" });
|
||||
await client.PostAsJsonAsync("/api/auth/login",
|
||||
new { email, password = "Passw0rd!" });
|
||||
return client;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Anonymous_user_cannot_request_a_pairing_code()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = factory.CreateClient();
|
||||
var res = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
|
||||
res.StatusCode.Should().Be(HttpStatusCode.Unauthorized);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Pair_code_then_pair_returns_token_and_device_id()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = await SignedInClient(factory);
|
||||
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
|
||||
codeRes.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
var codeBody = await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
|
||||
var code = codeBody!["code"].ToString()!;
|
||||
|
||||
var pairRes = await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "rpi", client_version = "0.1" });
|
||||
pairRes.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
var body = await pairRes.Content.ReadAsStringAsync();
|
||||
body.Should().Contain("device_id");
|
||||
body.Should().Contain("device_token");
|
||||
body.Should().Contain("backend_ws");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Pair_with_invalid_code_returns_404()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var res = await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code = "ZZZZZZZZ", hostname = "rpi", client_version = "0.1" });
|
||||
res.StatusCode.Should().Be(HttpStatusCode.NotFound);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Pair_with_consumed_code_returns_404()
|
||||
{
|
||||
using var factory = new ApiFactory();
|
||||
var client = await SignedInClient(factory);
|
||||
var codeRes = await client.PostAsJsonAsync("/api/pair-code", new { name = "kitchen" });
|
||||
var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
|
||||
|
||||
var first = await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "rpi", client_version = "0.1" });
|
||||
first.StatusCode.Should().Be(HttpStatusCode.OK);
|
||||
|
||||
var second = await factory.CreateClient().PostAsJsonAsync("/api/pair",
|
||||
new { code, hostname = "rpi", client_version = "0.1" });
|
||||
second.StatusCode.Should().Be(HttpStatusCode.NotFound);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
using backend.Pairing;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class PairingServiceTests
|
||||
{
|
||||
[Fact]
|
||||
public void Generated_codes_use_only_safe_alphabet_and_are_8_chars()
|
||||
{
|
||||
var svc = new PairingService();
|
||||
for (var i = 0; i < 50; i++)
|
||||
{
|
||||
var code = svc.GenerateCode();
|
||||
code.Length.Should().Be(8);
|
||||
code.Should().MatchRegex("^[A-HJ-KM-NP-Z2-9]+$");
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Generated_codes_are_not_all_identical()
|
||||
{
|
||||
var svc = new PairingService();
|
||||
var codes = Enumerable.Range(0, 20).Select(_ => svc.GenerateCode()).ToHashSet();
|
||||
codes.Count.Should().BeGreaterThan(15);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionAudioTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Uplink_bytes_become_input_audio_buffer_append()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(Array.Empty<ITool>());
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var cts = new CancellationTokenSource();
|
||||
var run = session.RunAsync(cts.Token);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
var frame = new byte[3840];
|
||||
for (int i = 0; i < frame.Length; i++) frame[i] = (byte)(i % 256);
|
||||
await session.WriteUplinkAsync(frame, default);
|
||||
|
||||
await Task.Delay(100);
|
||||
|
||||
var appended = upstream.Sent
|
||||
.FirstOrDefault(e => (string?)e["type"] == "input_audio_buffer.append");
|
||||
appended.Should().NotBeNull();
|
||||
var b64 = (string?)appended!["audio"];
|
||||
Convert.FromBase64String(b64!).Length.Should().Be(3840);
|
||||
|
||||
cts.Cancel();
|
||||
await run;
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Audio_delta_from_upstream_becomes_binary_to_device()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(Array.Empty<ITool>());
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var cts = new CancellationTokenSource();
|
||||
var run = session.RunAsync(cts.Token);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
var payload = new byte[3840];
|
||||
for (int i = 0; i < payload.Length; i++) payload[i] = (byte)((i + 1) % 256);
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.audio.delta",
|
||||
["delta"] = Convert.ToBase64String(payload),
|
||||
});
|
||||
|
||||
await Task.Delay(100);
|
||||
sink.Binary.Should().NotBeEmpty();
|
||||
sink.Binary[0].Length.Should().Be(3840);
|
||||
sink.Binary[0][0].Should().Be(1);
|
||||
|
||||
cts.Cancel();
|
||||
await run;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionEndSessionTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task End_session_call_closes_session_with_reason_tool_and_skips_response_create()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(new ITool[] { new EndSessionTool() });
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "end_session" });
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var run = session.RunAsync(default);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.audio.delta",
|
||||
["delta"] = Convert.ToBase64String(new byte[3840]),
|
||||
});
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.function_call_arguments.done",
|
||||
["call_id"] = "c1",
|
||||
["name"] = "end_session",
|
||||
["arguments"] = "{}",
|
||||
});
|
||||
upstream.Push(new JsonObject { ["type"] = "response.done" });
|
||||
|
||||
await run;
|
||||
|
||||
sink.Binary.Should().NotBeEmpty();
|
||||
|
||||
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
|
||||
((string?)ended["reason"]).Should().Be("tool");
|
||||
|
||||
upstream.Sent.Should().NotContain(e => (string?)e["type"] == "response.create");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionIdleTimeoutTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Session_ends_with_reason_idle_when_no_speech_for_timeout()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(Array.Empty<ITool>());
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 1, new HashSet<string>());
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var run = session.RunAsync(default);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
await run.WaitAsync(TimeSpan.FromSeconds(4));
|
||||
|
||||
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
|
||||
((string?)ended["reason"]).Should().Be("idle");
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Speech_started_resets_the_idle_timer()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(Array.Empty<ITool>());
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 2, new HashSet<string>());
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var run = session.RunAsync(default);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
await Task.Delay(1000);
|
||||
upstream.Push(new JsonObject { ["type"] = "input_audio_buffer.speech_started" });
|
||||
await Task.Delay(1500);
|
||||
|
||||
sink.Envelopes.Any(e => (string?)e["type"] == "session_ended").Should().BeFalse();
|
||||
|
||||
await run.WaitAsync(TimeSpan.FromSeconds(4));
|
||||
var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
|
||||
((string?)ended["reason"]).Should().Be("idle");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Data;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionLifecycleTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Open_emits_session_update_then_session_started_then_session_ended_on_cancel()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var registry = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });
|
||||
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings(
|
||||
Model: "gpt-4o-realtime-preview",
|
||||
Voice: "alloy",
|
||||
SystemPrompt: "test",
|
||||
IdleTimeoutSeconds: 30,
|
||||
EnabledTools: new HashSet<string> { "get_current_time" });
|
||||
|
||||
var session = new RealtimeSession(
|
||||
deviceId: Guid.NewGuid(),
|
||||
upstream, sink, log, registry, cfg,
|
||||
channel: new FakeDeviceChannel(),
|
||||
logger: NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
|
||||
var cts = new CancellationTokenSource();
|
||||
var runTask = session.RunAsync(cts.Token);
|
||||
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
cts.Cancel();
|
||||
await runTask;
|
||||
|
||||
upstream.Sent.Select(e => (string?)e["type"]).Should().StartWith("session.update");
|
||||
sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_started");
|
||||
sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_ended");
|
||||
}
|
||||
}
|
||||
|
||||
internal class NullLogger : Microsoft.Extensions.Logging.ILogger
|
||||
{
|
||||
public static readonly NullLogger Instance = new();
|
||||
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
|
||||
public bool IsEnabled(Microsoft.Extensions.Logging.LogLevel logLevel) => false;
|
||||
public void Log<TState>(Microsoft.Extensions.Logging.LogLevel l, Microsoft.Extensions.Logging.EventId e,
|
||||
TState s, Exception? ex, Func<TState, Exception?, string> f) { }
|
||||
}
|
||||
|
||||
internal class RecordingSink : ISessionOutput
|
||||
{
|
||||
public List<JsonObject> Envelopes { get; } = new();
|
||||
public List<byte[]> Binary { get; } = new();
|
||||
public Task WriteEnvelopeAsync(JsonObject e, CancellationToken ct)
|
||||
{
|
||||
lock (Envelopes) Envelopes.Add(e);
|
||||
_gate.Release();
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
public Task WriteBinaryAsync(ReadOnlyMemory<byte> b, CancellationToken ct)
|
||||
{
|
||||
Binary.Add(b.ToArray());
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
private readonly SemaphoreSlim _gate = new(0, int.MaxValue);
|
||||
public async Task WaitForAsync(string type, TimeSpan timeout)
|
||||
{
|
||||
var deadline = DateTime.UtcNow + timeout;
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
lock (Envelopes)
|
||||
{
|
||||
if (Envelopes.Any(e => (string?)e["type"] == type)) return;
|
||||
}
|
||||
await _gate.WaitAsync(timeout);
|
||||
}
|
||||
throw new TimeoutException($"never saw {type}");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Data;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionToolsTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task Get_current_time_call_returns_function_call_output_and_persists_tool_turn()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "get_current_time" });
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var cts = new CancellationTokenSource();
|
||||
var run = session.RunAsync(cts.Token);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.function_call_arguments.done",
|
||||
["call_id"] = "call_abc",
|
||||
["name"] = "get_current_time",
|
||||
["arguments"] = "{}",
|
||||
});
|
||||
upstream.Push(new JsonObject { ["type"] = "response.done" });
|
||||
|
||||
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(2);
|
||||
while (DateTime.UtcNow < deadline && !upstream.Sent.Any(e =>
|
||||
(string?)e["type"] == "conversation.item.create"))
|
||||
{
|
||||
await Task.Delay(20);
|
||||
}
|
||||
|
||||
var item = upstream.Sent.First(e => (string?)e["type"] == "conversation.item.create");
|
||||
((string?)item["item"]!["type"]).Should().Be("function_call_output");
|
||||
((string?)item["item"]!["call_id"]).Should().Be("call_abc");
|
||||
((string?)item["item"]!["output"]).Should().Contain("\"now\"");
|
||||
|
||||
upstream.Sent.Should().Contain(e => (string?)e["type"] == "response.create");
|
||||
|
||||
var convoId = session.ConversationId!.Value;
|
||||
var toolTurn = await db.Turns.FirstAsync(t =>
|
||||
t.ConversationId == convoId && t.Role == TurnRole.Tool);
|
||||
toolTurn.ToolName.Should().Be("get_current_time");
|
||||
toolTurn.ToolResultJson.Should().Contain("\"now\"");
|
||||
|
||||
cts.Cancel();
|
||||
await run;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Data;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class RealtimeSessionTranscriptTests(ApiFactory factory) : IClassFixture<ApiFactory>
|
||||
{
|
||||
private readonly ApiFactory _factory = factory;
|
||||
|
||||
[Fact]
|
||||
public async Task User_and_assistant_transcripts_are_persisted_and_response_done_emits_assistant_done()
|
||||
{
|
||||
using var scope = _factory.Services.CreateScope();
|
||||
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
|
||||
var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
|
||||
var reg = new ToolRegistry(Array.Empty<ITool>());
|
||||
var upstream = new ScriptedRealtimeUpstream();
|
||||
var sink = new RecordingSink();
|
||||
var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
|
||||
var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
|
||||
new FakeDeviceChannel(), NullLogger.Instance);
|
||||
|
||||
upstream.Push(new JsonObject { ["type"] = "session.updated" });
|
||||
var cts = new CancellationTokenSource();
|
||||
var run = session.RunAsync(cts.Token);
|
||||
await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));
|
||||
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "conversation.item.input_audio_transcription.completed",
|
||||
["transcript"] = "hello",
|
||||
});
|
||||
upstream.Push(new JsonObject
|
||||
{
|
||||
["type"] = "response.audio_transcript.done",
|
||||
["transcript"] = "hi there",
|
||||
});
|
||||
upstream.Push(new JsonObject { ["type"] = "response.done" });
|
||||
|
||||
await sink.WaitForAsync("assistant_done", TimeSpan.FromSeconds(2));
|
||||
|
||||
var convoId = session.ConversationId!.Value;
|
||||
var turns = await db.Turns
|
||||
.Where(t => t.ConversationId == convoId)
|
||||
.OrderBy(t => t.Index).ToListAsync();
|
||||
turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant);
|
||||
turns[0].Text.Should().Be("hello");
|
||||
turns[1].Text.Should().Be("hi there");
|
||||
|
||||
cts.Cancel();
|
||||
await run;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using System.Threading.Channels;
|
||||
using backend.Realtime;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class ScriptedRealtimeUpstream : IRealtimeUpstream
|
||||
{
|
||||
private readonly Channel<JsonObject> _toRelay =
|
||||
Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleReader = true });
|
||||
|
||||
private readonly Channel<JsonObject> _fromRelay =
|
||||
Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleWriter = true });
|
||||
|
||||
public IList<JsonObject> Sent { get; } = new List<JsonObject>();
|
||||
|
||||
public Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
|
||||
{
|
||||
Sent.Add(envelope);
|
||||
return _fromRelay.Writer.WriteAsync(envelope, ct).AsTask();
|
||||
}
|
||||
|
||||
public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
return await _toRelay.Reader.ReadAsync(ct);
|
||||
}
|
||||
catch (ChannelClosedException)
|
||||
{
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
public void Push(JsonObject evt) => _toRelay.Writer.TryWrite(evt);
|
||||
public void CloseUpstream() => _toRelay.Writer.TryComplete();
|
||||
|
||||
public async Task<JsonObject> WaitForSentAsync(string type, CancellationToken ct)
|
||||
{
|
||||
await foreach (var item in _fromRelay.Reader.ReadAllAsync(ct))
|
||||
{
|
||||
if ((string?)item["type"] == type) return item;
|
||||
}
|
||||
throw new TimeoutException($"never saw outbound {type}");
|
||||
}
|
||||
|
||||
public ValueTask DisposeAsync()
|
||||
{
|
||||
_toRelay.Writer.TryComplete();
|
||||
_fromRelay.Writer.TryComplete();
|
||||
return ValueTask.CompletedTask;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
using System.Text.Json;
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class SetVolumeToolTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Forwards_call_to_pi_and_returns_its_result()
|
||||
{
|
||||
var fake = new RecordingChannel(
|
||||
JsonDocument.Parse("""{"ok":true}""").RootElement);
|
||||
var tool = new SetVolumeTool();
|
||||
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);
|
||||
|
||||
var args = JsonDocument.Parse("""{"level":42}""").RootElement;
|
||||
var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);
|
||||
|
||||
fake.LastName.Should().Be("set_volume");
|
||||
fake.LastArgs.GetProperty("level").GetInt32().Should().Be(42);
|
||||
res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Rejects_out_of_range_level_without_calling_pi()
|
||||
{
|
||||
var fake = new RecordingChannel(JsonDocument.Parse("""{"ok":true}""").RootElement);
|
||||
var tool = new SetVolumeTool();
|
||||
var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);
|
||||
|
||||
var args = JsonDocument.Parse("""{"level":150}""").RootElement;
|
||||
var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);
|
||||
|
||||
fake.LastName.Should().BeNull();
|
||||
res.Output.GetProperty("ok").GetBoolean().Should().BeFalse();
|
||||
res.Output.GetProperty("error").GetString().Should().Contain("0..100");
|
||||
}
|
||||
|
||||
private class RecordingChannel(JsonElement reply) : IDeviceChannel
|
||||
{
|
||||
public string? LastName { get; private set; }
|
||||
public JsonElement LastArgs { get; private set; }
|
||||
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
|
||||
{
|
||||
LastName = name;
|
||||
LastArgs = args;
|
||||
return Task.FromResult(reply);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
using backend.Tools;
|
||||
using FluentAssertions;
|
||||
using Xunit;
|
||||
|
||||
namespace backend.tests;
|
||||
|
||||
public class ToolRegistryTests
|
||||
{
|
||||
private readonly ITool[] _all = new ITool[]
|
||||
{
|
||||
new GetCurrentTimeTool(),
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void Get_returns_tool_by_exact_name()
|
||||
{
|
||||
var reg = new ToolRegistry(_all);
|
||||
reg.Get("get_current_time").Should().NotBeNull();
|
||||
reg.Get("does_not_exist").Should().BeNull();
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Filter_returns_only_enabled_tools_preserving_order()
|
||||
{
|
||||
var reg = new ToolRegistry(_all);
|
||||
var enabled = new HashSet<string> { "get_current_time", "ghost" };
|
||||
reg.EnabledFor(enabled).Select(t => t.Name).Should().Equal("get_current_time");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<IsPackable>false</IsPackable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="coverlet.collector" Version="6.0.2" />
|
||||
<PackageReference Include="FluentAssertions" Version="8.10.0" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="9.0.8" />
|
||||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
|
||||
<PackageReference Include="xunit" Version="2.9.2" />
|
||||
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<Using Include="Xunit" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\backend\backend.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -0,0 +1,7 @@
|
||||
using Microsoft.AspNetCore.Identity;
|
||||
|
||||
namespace backend.Auth;
|
||||
|
||||
public class ApplicationUser : IdentityUser<Guid>
|
||||
{
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
using Microsoft.AspNetCore.Identity;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
|
||||
namespace backend.Auth;
|
||||
|
||||
public record RegisterRequest(string Email, string Password);
|
||||
public record LoginRequest(string Email, string Password);
|
||||
|
||||
public static class AuthEndpoints
|
||||
{
|
||||
public static IEndpointRouteBuilder MapAuthEndpoints(this IEndpointRouteBuilder app)
|
||||
{
|
||||
var grp = app.MapGroup("/api/auth");
|
||||
|
||||
grp.MapPost("/register", async (
|
||||
[FromBody] RegisterRequest req,
|
||||
UserManager<ApplicationUser> users) =>
|
||||
{
|
||||
var user = new ApplicationUser { UserName = req.Email, Email = req.Email };
|
||||
var create = await users.CreateAsync(user, req.Password);
|
||||
if (!create.Succeeded)
|
||||
return Results.BadRequest(new { errors = create.Errors.Select(e => e.Description) });
|
||||
|
||||
var anyAdmin = await users.GetUsersInRoleAsync(Roles.Admin);
|
||||
var role = anyAdmin.Count == 0 ? Roles.Admin : Roles.User;
|
||||
await users.AddToRoleAsync(user, role);
|
||||
return Results.Ok(new { id = user.Id, email = user.Email, role });
|
||||
});
|
||||
|
||||
grp.MapPost("/login", async (
|
||||
[FromBody] LoginRequest req,
|
||||
SignInManager<ApplicationUser> signIn) =>
|
||||
{
|
||||
var res = await signIn.PasswordSignInAsync(req.Email, req.Password,
|
||||
isPersistent: true, lockoutOnFailure: false);
|
||||
return res.Succeeded ? Results.Ok() : Results.Unauthorized();
|
||||
});
|
||||
|
||||
grp.MapPost("/logout", async (SignInManager<ApplicationUser> signIn) =>
|
||||
{
|
||||
await signIn.SignOutAsync();
|
||||
return Results.Ok();
|
||||
}).RequireAuthorization();
|
||||
|
||||
grp.MapGet("/me", async (
|
||||
UserManager<ApplicationUser> users,
|
||||
HttpContext http) =>
|
||||
{
|
||||
var user = await users.GetUserAsync(http.User);
|
||||
if (user is null) return Results.Unauthorized();
|
||||
var rolesList = await users.GetRolesAsync(user);
|
||||
return Results.Ok(new
|
||||
{
|
||||
id = user.Id,
|
||||
email = user.Email,
|
||||
isAdmin = rolesList.Contains(Roles.Admin),
|
||||
});
|
||||
}).RequireAuthorization();
|
||||
|
||||
return app;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
using backend.Data;
|
||||
using Microsoft.AspNetCore.Identity;
|
||||
|
||||
namespace backend.Auth;
|
||||
|
||||
public static class IdentitySetup
|
||||
{
|
||||
public static IServiceCollection AddAppIdentity(this IServiceCollection services)
|
||||
{
|
||||
services.AddIdentity<ApplicationUser, IdentityRole<Guid>>(opt =>
|
||||
{
|
||||
opt.Password.RequiredLength = 8;
|
||||
opt.Password.RequireNonAlphanumeric = false;
|
||||
opt.User.RequireUniqueEmail = true;
|
||||
opt.SignIn.RequireConfirmedAccount = false;
|
||||
})
|
||||
.AddEntityFrameworkStores<AppDbContext>()
|
||||
.AddDefaultTokenProviders();
|
||||
|
||||
services.ConfigureApplicationCookie(opt =>
|
||||
{
|
||||
opt.Cookie.HttpOnly = true;
|
||||
opt.Cookie.SameSite = SameSiteMode.Lax;
|
||||
opt.Cookie.SecurePolicy = CookieSecurePolicy.SameAsRequest;
|
||||
opt.ExpireTimeSpan = TimeSpan.FromDays(30);
|
||||
opt.SlidingExpiration = true;
|
||||
opt.Events.OnRedirectToLogin = ctx =>
|
||||
{
|
||||
ctx.Response.StatusCode = 401;
|
||||
return Task.CompletedTask;
|
||||
};
|
||||
opt.Events.OnRedirectToAccessDenied = ctx =>
|
||||
{
|
||||
ctx.Response.StatusCode = 403;
|
||||
return Task.CompletedTask;
|
||||
};
|
||||
});
|
||||
|
||||
return services;
|
||||
}
|
||||
|
||||
public static async Task SeedRolesAsync(IServiceProvider sp)
|
||||
{
|
||||
var roleMgr = sp.GetRequiredService<RoleManager<IdentityRole<Guid>>>();
|
||||
foreach (var name in Roles.All)
|
||||
{
|
||||
if (!await roleMgr.RoleExistsAsync(name))
|
||||
await roleMgr.CreateAsync(new IdentityRole<Guid>(name));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
namespace backend.Auth;
|
||||
|
||||
public static class Roles
|
||||
{
|
||||
public const string Admin = "Admin";
|
||||
public const string User = "User";
|
||||
|
||||
public static readonly string[] All = { Admin, User };
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
namespace backend.Conversations;
|
||||
|
||||
public enum EndReason
|
||||
{
|
||||
Unknown = 0,
|
||||
Tool = 1,
|
||||
Idle = 2,
|
||||
Error = 3,
|
||||
}
|
||||
|
||||
public class Conversation
|
||||
{
|
||||
public Guid Id { get; set; } = Guid.NewGuid();
|
||||
public Guid DeviceId { get; set; }
|
||||
public DateTimeOffset StartedAt { get; set; }
|
||||
public DateTimeOffset? EndedAt { get; set; }
|
||||
public EndReason EndReason { get; set; } = EndReason.Unknown;
|
||||
public List<Turn> Turns { get; set; } = new();
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
using backend.Data;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace backend.Conversations;
|
||||
|
||||
public class ConversationLog(AppDbContext db)
|
||||
{
|
||||
public async Task<Conversation> StartAsync(Guid deviceId, CancellationToken ct)
|
||||
{
|
||||
var c = new Conversation
|
||||
{
|
||||
DeviceId = deviceId,
|
||||
StartedAt = DateTimeOffset.UtcNow,
|
||||
};
|
||||
db.Conversations.Add(c);
|
||||
await db.SaveChangesAsync(ct);
|
||||
return c;
|
||||
}
|
||||
|
||||
public Task AppendUserAsync(Guid conversationId, string text, CancellationToken ct)
|
||||
=> AppendAsync(conversationId, TurnRole.User, text: text, ct: ct);
|
||||
|
||||
public Task AppendAssistantAsync(Guid conversationId, string text, CancellationToken ct)
|
||||
=> AppendAsync(conversationId, TurnRole.Assistant, text: text, ct: ct);
|
||||
|
||||
public Task AppendToolAsync(
|
||||
Guid conversationId, string toolName, string argsJson, string resultJson, CancellationToken ct)
|
||||
=> AppendAsync(conversationId, TurnRole.Tool,
|
||||
toolName: toolName, argsJson: argsJson, resultJson: resultJson, ct: ct);
|
||||
|
||||
public async Task EndAsync(Guid conversationId, EndReason reason, CancellationToken ct)
|
||||
{
|
||||
var row = await db.Conversations.FirstAsync(c => c.Id == conversationId, ct);
|
||||
row.EndedAt = DateTimeOffset.UtcNow;
|
||||
row.EndReason = reason;
|
||||
await db.SaveChangesAsync(ct);
|
||||
}
|
||||
|
||||
private async Task AppendAsync(
|
||||
Guid conversationId, TurnRole role,
|
||||
string? text = null, string? toolName = null,
|
||||
string? argsJson = null, string? resultJson = null,
|
||||
CancellationToken ct = default)
|
||||
{
|
||||
var next = await db.Turns
|
||||
.Where(t => t.ConversationId == conversationId)
|
||||
.Select(t => (int?)t.Index)
|
||||
.MaxAsync(ct) ?? -1;
|
||||
|
||||
db.Turns.Add(new Turn
|
||||
{
|
||||
ConversationId = conversationId,
|
||||
Index = next + 1,
|
||||
Role = role,
|
||||
Text = text,
|
||||
ToolName = toolName,
|
||||
ToolArgsJson = argsJson,
|
||||
ToolResultJson = resultJson,
|
||||
CreatedAt = DateTimeOffset.UtcNow,
|
||||
});
|
||||
await db.SaveChangesAsync(ct);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
namespace backend.Conversations;
|
||||
|
||||
public enum TurnRole
|
||||
{
|
||||
User = 0,
|
||||
Assistant = 1,
|
||||
Tool = 2,
|
||||
}
|
||||
|
||||
public class Turn
|
||||
{
|
||||
public Guid Id { get; set; } = Guid.NewGuid();
|
||||
public Guid ConversationId { get; set; }
|
||||
public int Index { get; set; }
|
||||
public TurnRole Role { get; set; }
|
||||
public string? Text { get; set; }
|
||||
public string? ToolName { get; set; }
|
||||
public string? ToolArgsJson { get; set; }
|
||||
public string? ToolResultJson { get; set; }
|
||||
public DateTimeOffset CreatedAt { get; set; }
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
using backend.Auth;
|
||||
using backend.Conversations;
|
||||
using backend.Devices;
|
||||
using backend.Pairing;
|
||||
using backend.Settings;
|
||||
using Microsoft.AspNetCore.Identity;
|
||||
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace backend.Data;
|
||||
|
||||
public class AppDbContext(DbContextOptions<AppDbContext> options)
|
||||
: IdentityDbContext<ApplicationUser, IdentityRole<Guid>, Guid>(options)
|
||||
{
|
||||
public DbSet<Device> Devices => Set<Device>();
|
||||
public DbSet<DeviceConfig> DeviceConfigs => Set<DeviceConfig>();
|
||||
public DbSet<SystemSettings> SystemSettings => Set<SystemSettings>();
|
||||
public DbSet<PairingCode> PairingCodes => Set<PairingCode>();
|
||||
public DbSet<Conversation> Conversations => Set<Conversation>();
|
||||
public DbSet<Turn> Turns => Set<Turn>();
|
||||
|
||||
protected override void OnModelCreating(ModelBuilder b)
|
||||
{
|
||||
base.OnModelCreating(b);
|
||||
|
||||
b.Entity<Device>()
|
||||
.HasOne<ApplicationUser>()
|
||||
.WithMany()
|
||||
.HasForeignKey(d => d.OwnerUserId)
|
||||
.OnDelete(DeleteBehavior.Cascade);
|
||||
b.Entity<Device>().HasIndex(d => d.TokenHash).IsUnique();
|
||||
b.Entity<Device>()
|
||||
.HasOne(d => d.Config)
|
||||
.WithOne()
|
||||
.HasForeignKey<DeviceConfig>(c => c.DeviceId)
|
||||
.OnDelete(DeleteBehavior.Cascade);
|
||||
b.Entity<DeviceConfig>().HasKey(c => c.DeviceId);
|
||||
b.Entity<SystemSettings>().HasKey(s => s.Id);
|
||||
b.Entity<SystemSettings>().Property(s => s.Id).ValueGeneratedNever();
|
||||
b.Entity<PairingCode>().HasKey(p => p.Code);
|
||||
b.Entity<PairingCode>().HasIndex(p => p.ExpiresAt);
|
||||
|
||||
b.Entity<Conversation>().HasIndex(c => c.DeviceId);
|
||||
b.Entity<Conversation>()
|
||||
.HasMany(c => c.Turns)
|
||||
.WithOne()
|
||||
.HasForeignKey(t => t.ConversationId)
|
||||
.OnDelete(DeleteBehavior.Cascade);
|
||||
b.Entity<Turn>().HasIndex(t => new { t.ConversationId, t.Index }).IsUnique();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Net.WebSockets;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
|
||||
namespace backend.DeviceHub;
|
||||
|
||||
public interface IDeviceClient
|
||||
{
|
||||
Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope;
|
||||
Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
|
||||
}
|
||||
|
||||
public class ActiveDevice : IDeviceChannel, IDeviceClient, ISessionOutput, IAsyncDisposable
|
||||
{
|
||||
public Guid DeviceId { get; }
|
||||
public WebSocket Ws => _ws;
|
||||
public RealtimeSession? CurrentSession { get; set; }
|
||||
public CancellationTokenSource? SessionCts { get; set; }
|
||||
|
||||
private readonly WebSocket _ws;
|
||||
private readonly SemaphoreSlim _sendLock = new(1, 1);
|
||||
private readonly ConcurrentDictionary<string, TaskCompletionSource<JsonElement>> _pendingTools = new();
|
||||
|
||||
public ActiveDevice(Guid deviceId, WebSocket ws)
|
||||
{
|
||||
DeviceId = deviceId;
|
||||
_ws = ws;
|
||||
}
|
||||
|
||||
public async Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
|
||||
{
|
||||
var json = JsonSerializer.Serialize(envelope, envelope.GetType());
|
||||
await _sendLock.WaitAsync(ct);
|
||||
try
|
||||
{
|
||||
await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
|
||||
WebSocketMessageType.Text, endOfMessage: true, ct);
|
||||
}
|
||||
finally { _sendLock.Release(); }
|
||||
}
|
||||
|
||||
public async Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
|
||||
{
|
||||
await _sendLock.WaitAsync(ct);
|
||||
try
|
||||
{
|
||||
await _ws.SendAsync(bytes, WebSocketMessageType.Binary, endOfMessage: true, ct);
|
||||
}
|
||||
finally { _sendLock.Release(); }
|
||||
}
|
||||
|
||||
public Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct)
|
||||
{
|
||||
var json = envelope.ToJsonString();
|
||||
return WriteRawTextAsync(json, ct);
|
||||
}
|
||||
|
||||
private async Task WriteRawTextAsync(string json, CancellationToken ct)
|
||||
{
|
||||
await _sendLock.WaitAsync(ct);
|
||||
try
|
||||
{
|
||||
await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
|
||||
WebSocketMessageType.Text, endOfMessage: true, ct);
|
||||
}
|
||||
finally { _sendLock.Release(); }
|
||||
}
|
||||
|
||||
public Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
|
||||
=> SendBinaryAsync(bytes, ct);
|
||||
|
||||
public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
|
||||
{
|
||||
var callId = "srv_" + Guid.NewGuid().ToString("N")[..12];
|
||||
var tcs = new TaskCompletionSource<JsonElement>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||
_pendingTools[callId] = tcs;
|
||||
|
||||
var envelope = new ToolCallEnvelope("tool_call", callId, name, args);
|
||||
_ = SendEnvelopeAsync(envelope, ct);
|
||||
|
||||
ct.Register(() => tcs.TrySetCanceled());
|
||||
return tcs.Task;
|
||||
}
|
||||
|
||||
public void CompleteToolResult(ToolResultEnvelope env)
|
||||
{
|
||||
if (!_pendingTools.TryRemove(env.CallId, out var tcs)) return;
|
||||
if (env.Ok && env.Result is JsonElement r) tcs.TrySetResult(r);
|
||||
else
|
||||
{
|
||||
var msg = (env.Error ?? "unknown").Replace("\\", "\\\\").Replace("\"", "\\\"");
|
||||
var doc = JsonDocument.Parse($$"""{"ok":false,"error":"{{msg}}"}""");
|
||||
tcs.TrySetResult(doc.RootElement.Clone());
|
||||
}
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
try
|
||||
{
|
||||
if (_ws.State == WebSocketState.Open)
|
||||
await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None);
|
||||
}
|
||||
catch { }
|
||||
_ws.Dispose();
|
||||
_sendLock.Dispose();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
using backend.Data;
|
||||
using backend.Devices;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace backend.DeviceHub;
|
||||
|
||||
public class DeviceAuth(AppDbContext db, DeviceTokenService tokens)
|
||||
{
|
||||
public async Task<Device?> AuthenticateAsync(string? authHeader, CancellationToken ct)
|
||||
{
|
||||
if (string.IsNullOrEmpty(authHeader)) return null;
|
||||
const string prefix = "Bearer ";
|
||||
if (!authHeader.StartsWith(prefix, StringComparison.Ordinal)) return null;
|
||||
var token = authHeader[prefix.Length..].Trim();
|
||||
if (string.IsNullOrEmpty(token)) return null;
|
||||
var hash = tokens.Hash(token);
|
||||
var device = await db.Devices.Include(d => d.Config)
|
||||
.FirstOrDefaultAsync(d => d.TokenHash == hash && !d.IsRevoked, ct);
|
||||
return device;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,196 @@
|
||||
using System.Net.WebSockets;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Data;
|
||||
using backend.Devices;
|
||||
using backend.Realtime;
|
||||
using backend.Tools;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
|
||||
namespace backend.DeviceHub;
|
||||
|
||||
public static class DeviceHubEndpoint
|
||||
{
|
||||
public static IEndpointRouteBuilder MapDeviceHub(this IEndpointRouteBuilder app)
|
||||
{
|
||||
app.Map("/device", async (
|
||||
HttpContext ctx,
|
||||
DeviceAuth auth,
|
||||
DeviceRegistry reg,
|
||||
AppDbContext db,
|
||||
RealtimeSessionFactory sessions,
|
||||
IServiceScopeFactory scopes) =>
|
||||
{
|
||||
if (!ctx.WebSockets.IsWebSocketRequest)
|
||||
{
|
||||
ctx.Response.StatusCode = StatusCodes.Status400BadRequest;
|
||||
return;
|
||||
}
|
||||
|
||||
var device = await auth.AuthenticateAsync(
|
||||
ctx.Request.Headers.Authorization.ToString(), ctx.RequestAborted);
|
||||
if (device is null)
|
||||
{
|
||||
ctx.Response.StatusCode = StatusCodes.Status401Unauthorized;
|
||||
return;
|
||||
}
|
||||
|
||||
using var ws = await ctx.WebSockets.AcceptWebSocketAsync();
|
||||
await using var active = new ActiveDevice(device.Id, ws);
|
||||
if (!reg.TryRegister(active))
|
||||
{
|
||||
await ws.CloseAsync(WebSocketCloseStatus.PolicyViolation,
|
||||
"device already connected", ctx.RequestAborted);
|
||||
return;
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
await HandleAsync(active, device, db, sessions, scopes, ctx.RequestAborted);
|
||||
}
|
||||
finally
|
||||
{
|
||||
reg.Unregister(device.Id);
|
||||
}
|
||||
});
|
||||
return app;
|
||||
}
|
||||
|
||||
private static async Task HandleAsync(
|
||||
ActiveDevice active, Device device, AppDbContext db,
|
||||
RealtimeSessionFactory sessions, IServiceScopeFactory scopes,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var buffer = new byte[8192];
|
||||
var ws = active.Ws;
|
||||
|
||||
while (ws.State == WebSocketState.Open && !ct.IsCancellationRequested)
|
||||
{
|
||||
using var msg = new MemoryStream();
|
||||
WebSocketReceiveResult res;
|
||||
while (true)
|
||||
{
|
||||
res = await ws.ReceiveAsync(buffer, ct);
|
||||
if (res.MessageType == WebSocketMessageType.Close) return;
|
||||
msg.Write(buffer, 0, res.Count);
|
||||
if (res.EndOfMessage) break;
|
||||
}
|
||||
|
||||
if (res.MessageType == WebSocketMessageType.Binary)
|
||||
{
|
||||
if (msg.Length == 3840 && active.CurrentSession is not null)
|
||||
{
|
||||
await active.CurrentSession.WriteUplinkAsync(msg.ToArray(), ct);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
var text = Encoding.UTF8.GetString(msg.ToArray());
|
||||
using var doc = JsonDocument.Parse(text);
|
||||
var type = doc.RootElement.GetProperty("type").GetString();
|
||||
switch (type)
|
||||
{
|
||||
case "hello":
|
||||
await SendHelloAckAsync(active, device, ct);
|
||||
break;
|
||||
|
||||
case "ping":
|
||||
device.LastSeenAt = DateTimeOffset.UtcNow;
|
||||
await db.SaveChangesAsync(ct);
|
||||
await active.SendEnvelopeAsync(new PongEnvelope("pong"), ct);
|
||||
break;
|
||||
|
||||
case "wake":
|
||||
if (active.CurrentSession is null)
|
||||
await StartSessionAsync(active, device, sessions, scopes, ct);
|
||||
break;
|
||||
|
||||
case "cancel":
|
||||
active.SessionCts?.Cancel();
|
||||
break;
|
||||
|
||||
case "playback_done":
|
||||
break; // v1: client coordination only
|
||||
|
||||
case "tool_result":
|
||||
var env = JsonSerializer.Deserialize<ToolResultEnvelope>(text)!;
|
||||
active.CompleteToolResult(env);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task StartSessionAsync(
|
||||
ActiveDevice active, Device device,
|
||||
RealtimeSessionFactory sessions, IServiceScopeFactory scopes,
|
||||
CancellationToken parentCt)
|
||||
{
|
||||
var enabledTools = JsonSerializer.Deserialize<string[]>(device.Config?.EnabledToolsJson ?? "[]")
|
||||
?? Array.Empty<string>();
|
||||
var cfg = new RealtimeSettings(
|
||||
Model: device.Config?.Model ?? "gpt-4o-realtime-preview",
|
||||
Voice: device.Config?.Voice ?? "alloy",
|
||||
SystemPrompt: device.Config?.SystemPrompt ?? "",
|
||||
IdleTimeoutSeconds: device.Config?.IdleTimeoutSeconds ?? 30,
|
||||
EnabledTools: new HashSet<string>(enabledTools));
|
||||
|
||||
active.SessionCts = CancellationTokenSource.CreateLinkedTokenSource(parentCt);
|
||||
var ct = active.SessionCts.Token;
|
||||
|
||||
var sessionScope = scopes.CreateAsyncScope();
|
||||
var scopedSessions = sessionScope.ServiceProvider.GetRequiredService<RealtimeSessionFactory>();
|
||||
|
||||
RealtimeSession session;
|
||||
try
|
||||
{
|
||||
session = await scopedSessions.CreateAsync(
|
||||
device.Id, cfg, active, active, ct);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await active.SendEnvelopeAsync(
|
||||
new ErrorEnvelope("error", "upstream", ex.Message, true), parentCt);
|
||||
await sessionScope.DisposeAsync();
|
||||
return;
|
||||
}
|
||||
|
||||
active.CurrentSession = session;
|
||||
_ = Task.Run(async () =>
|
||||
{
|
||||
try { await session.RunAsync(ct); }
|
||||
finally
|
||||
{
|
||||
active.CurrentSession = null;
|
||||
active.SessionCts?.Dispose();
|
||||
active.SessionCts = null;
|
||||
await sessionScope.DisposeAsync();
|
||||
}
|
||||
}, ct);
|
||||
}
|
||||
|
||||
private static async Task SendHelloAckAsync(ActiveDevice active, Device device, CancellationToken ct)
|
||||
{
|
||||
var enabled = device.Config?.EnabledToolsJson ?? "[]";
|
||||
var config = JsonNode.Parse($$"""
|
||||
{
|
||||
"voice": "{{Escape(device.Config?.Voice)}}",
|
||||
"model": "{{Escape(device.Config?.Model)}}",
|
||||
"system_prompt": "{{Escape(device.Config?.SystemPrompt)}}",
|
||||
"idle_timeout_seconds": {{device.Config?.IdleTimeoutSeconds ?? 30}},
|
||||
"enabled_tools": {{enabled}}
|
||||
}
|
||||
""")!.AsObject();
|
||||
|
||||
await active.WriteEnvelopeAsync(new JsonObject
|
||||
{
|
||||
["type"] = "hello_ack",
|
||||
["config"] = config,
|
||||
}, ct);
|
||||
}
|
||||
|
||||
private static string Escape(string? s) =>
|
||||
(s ?? "").Replace("\\", "\\\\").Replace("\"", "\\\"");
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
using System.Collections.Concurrent;
|
||||
|
||||
namespace backend.DeviceHub;
|
||||
|
||||
public class DeviceRegistry
|
||||
{
|
||||
private readonly ConcurrentDictionary<Guid, ActiveDevice> _online = new();
|
||||
|
||||
public bool TryRegister(ActiveDevice dev)
|
||||
=> _online.TryAdd(dev.DeviceId, dev);
|
||||
|
||||
public bool Unregister(Guid deviceId)
|
||||
=> _online.TryRemove(deviceId, out _);
|
||||
|
||||
public ActiveDevice? Get(Guid deviceId)
|
||||
=> _online.TryGetValue(deviceId, out var d) ? d : null;
|
||||
|
||||
public bool IsOnline(Guid deviceId) => _online.ContainsKey(deviceId);
|
||||
|
||||
public int OnlineCount => _online.Count;
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Serialization;
|
||||
|
||||
namespace backend.DeviceHub;
|
||||
|
||||
public record HubEnvelope([property: JsonPropertyName("type")] string Type);
|
||||
|
||||
public record HelloEnvelope(string Type,
|
||||
[property: JsonPropertyName("device_id")] Guid DeviceId,
|
||||
[property: JsonPropertyName("client_version")] string ClientVersion) : HubEnvelope(Type);
|
||||
|
||||
public record HelloAckEnvelope(string Type,
|
||||
[property: JsonPropertyName("config")] JsonElement Config) : HubEnvelope(Type);
|
||||
|
||||
public record PongEnvelope(string Type) : HubEnvelope(Type);
|
||||
|
||||
public record WakeEnvelope(string Type,
|
||||
[property: JsonPropertyName("at")] long? At) : HubEnvelope(Type);
|
||||
|
||||
public record SessionStartedEnvelope(string Type,
|
||||
[property: JsonPropertyName("conversation_id")] Guid ConversationId) : HubEnvelope(Type);
|
||||
|
||||
public record SessionEndedEnvelope(string Type,
|
||||
[property: JsonPropertyName("reason")] string Reason) : HubEnvelope(Type);
|
||||
|
||||
public record AssistantDoneEnvelope(string Type) : HubEnvelope(Type);
|
||||
|
||||
public record ToolCallEnvelope(string Type,
|
||||
[property: JsonPropertyName("call_id")] string CallId,
|
||||
[property: JsonPropertyName("name")] string Name,
|
||||
[property: JsonPropertyName("arguments")] JsonElement Arguments) : HubEnvelope(Type);
|
||||
|
||||
public record ToolResultEnvelope(string Type,
|
||||
[property: JsonPropertyName("call_id")] string CallId,
|
||||
[property: JsonPropertyName("ok")] bool Ok,
|
||||
[property: JsonPropertyName("result")] JsonElement? Result,
|
||||
[property: JsonPropertyName("error")] string? Error) : HubEnvelope(Type);
|
||||
|
||||
public record ErrorEnvelope(string Type,
|
||||
[property: JsonPropertyName("code")] string Code,
|
||||
[property: JsonPropertyName("message")] string Message,
|
||||
[property: JsonPropertyName("fatal")] bool? Fatal) : HubEnvelope(Type);
|
||||
@@ -0,0 +1,13 @@
|
||||
namespace backend.Devices;
|
||||
|
||||
public class Device
|
||||
{
|
||||
public Guid Id { get; set; } = Guid.NewGuid();
|
||||
public Guid OwnerUserId { get; set; }
|
||||
public string Name { get; set; } = "";
|
||||
public DateTimeOffset PairedAt { get; set; }
|
||||
public DateTimeOffset? LastSeenAt { get; set; }
|
||||
public string TokenHash { get; set; } = "";
|
||||
public bool IsRevoked { get; set; }
|
||||
public DeviceConfig? Config { get; set; }
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
namespace backend.Devices;
|
||||
|
||||
public class DeviceConfig
|
||||
{
|
||||
public Guid DeviceId { get; set; }
|
||||
public string SystemPrompt { get; set; } = "";
|
||||
public string Voice { get; set; } = "";
|
||||
public string Model { get; set; } = "";
|
||||
public int IdleTimeoutSeconds { get; set; } = 30;
|
||||
public string EnabledToolsJson { get; set; } = "[\"end_session\",\"get_current_time\",\"set_volume\"]";
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
using System.Security.Cryptography;
|
||||
using System.Text;
|
||||
|
||||
namespace backend.Devices;
|
||||
|
||||
public class DeviceTokenService
|
||||
{
|
||||
public (string Plaintext, string Hash) Generate()
|
||||
{
|
||||
Span<byte> bytes = stackalloc byte[32];
|
||||
RandomNumberGenerator.Fill(bytes);
|
||||
var plaintext = Base64UrlEncode(bytes);
|
||||
return (plaintext, Hash(plaintext));
|
||||
}
|
||||
|
||||
public string Hash(string plaintext)
|
||||
{
|
||||
Span<byte> hash = stackalloc byte[32];
|
||||
SHA256.HashData(Encoding.UTF8.GetBytes(plaintext), hash);
|
||||
return Convert.ToHexString(hash);
|
||||
}
|
||||
|
||||
public bool Verify(string plaintext, string expectedHash)
|
||||
{
|
||||
var actual = Hash(plaintext);
|
||||
return CryptographicOperations.FixedTimeEquals(
|
||||
Encoding.UTF8.GetBytes(actual),
|
||||
Encoding.UTF8.GetBytes(expectedHash));
|
||||
}
|
||||
|
||||
private static string Base64UrlEncode(ReadOnlySpan<byte> bytes)
|
||||
=> Convert.ToBase64String(bytes)
|
||||
.TrimEnd('=')
|
||||
.Replace('+', '-')
|
||||
.Replace('/', '_');
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
using System.Security.Claims;
|
||||
using backend.Data;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace backend.Devices;
|
||||
|
||||
public record RenameRequest(string Name);
|
||||
|
||||
public static class DevicesEndpoints
|
||||
{
|
||||
public static IEndpointRouteBuilder MapDevicesEndpoints(this IEndpointRouteBuilder app)
|
||||
{
|
||||
var grp = app.MapGroup("/api/devices").RequireAuthorization();
|
||||
|
||||
grp.MapGet("/", async (AppDbContext db, HttpContext http, CancellationToken ct) =>
|
||||
{
|
||||
var uid = GetUserId(http);
|
||||
var devs = await db.Devices
|
||||
.Where(d => d.OwnerUserId == uid)
|
||||
.Select(d => new
|
||||
{
|
||||
id = d.Id,
|
||||
name = d.Name,
|
||||
pairedAt = d.PairedAt,
|
||||
lastSeenAt = d.LastSeenAt,
|
||||
isRevoked = d.IsRevoked,
|
||||
})
|
||||
.ToListAsync(ct);
|
||||
return Results.Ok(devs);
|
||||
});
|
||||
|
||||
grp.MapGet("/{id:guid}", async (Guid id, AppDbContext db, HttpContext http, CancellationToken ct) =>
|
||||
{
|
||||
var uid = GetUserId(http);
|
||||
var d = await db.Devices.Include(x => x.Config)
|
||||
.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
|
||||
return d is null
|
||||
? Results.NotFound()
|
||||
: Results.Ok(new
|
||||
{
|
||||
id = d.Id,
|
||||
name = d.Name,
|
||||
pairedAt = d.PairedAt,
|
||||
lastSeenAt = d.LastSeenAt,
|
||||
isRevoked = d.IsRevoked,
|
||||
config = d.Config,
|
||||
});
|
||||
});
|
||||
|
||||
grp.MapPatch("/{id:guid}", async (
|
||||
Guid id, [FromBody] RenameRequest req,
|
||||
AppDbContext db, HttpContext http, CancellationToken ct) =>
|
||||
{
|
||||
var uid = GetUserId(http);
|
||||
var d = await db.Devices.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
|
||||
if (d is null) return Results.NotFound();
|
||||
if (string.IsNullOrWhiteSpace(req.Name))
|
||||
return Results.BadRequest(new { error = "name is required" });
|
||||
d.Name = req.Name.Trim();
|
||||
await db.SaveChangesAsync(ct);
|
||||
return Results.Ok();
|
||||
});
|
||||
|
||||
grp.MapDelete("/{id:guid}", async (Guid id, AppDbContext db, HttpContext http, CancellationToken ct) =>
|
||||
{
|
||||
var uid = GetUserId(http);
|
||||
var d = await db.Devices.FirstOrDefaultAsync(x => x.Id == id && x.OwnerUserId == uid, ct);
|
||||
if (d is null) return Results.NotFound();
|
||||
d.IsRevoked = true;
|
||||
await db.SaveChangesAsync(ct);
|
||||
return Results.Ok();
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
|
||||
private static Guid GetUserId(HttpContext http)
|
||||
{
|
||||
var s = http.User.FindFirstValue(ClaimTypes.NameIdentifier);
|
||||
return Guid.TryParse(s, out var g) ? g : Guid.Empty;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
using Microsoft.Extensions.Configuration;
|
||||
|
||||
namespace backend.Install;
|
||||
|
||||
public static class InstallEndpoints
|
||||
{
|
||||
public static IEndpointRouteBuilder MapInstallEndpoints(this IEndpointRouteBuilder app)
|
||||
{
|
||||
app.MapGet("/install.sh", (HttpContext http, InstallScriptBuilder builder) =>
|
||||
{
|
||||
var scheme = http.Request.Scheme;
|
||||
var host = http.Request.Host.ToString();
|
||||
var script = builder.Build($"{scheme}://{host}");
|
||||
return Results.Text(script, "text/x-shellscript");
|
||||
});
|
||||
|
||||
app.MapGet("/client.tar.gz", (IConfiguration cfg, IWebHostEnvironment env) =>
|
||||
{
|
||||
var path = cfg["Install:ClientTarballPath"]
|
||||
?? Path.Combine(env.WebRootPath ?? "wwwroot", "client.tar.gz");
|
||||
if (!File.Exists(path))
|
||||
return Results.NotFound(new { error = "client.tar.gz not found at " + path });
|
||||
return Results.File(path, "application/gzip", "client.tar.gz");
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
namespace backend.Install;
|
||||
|
||||
public class InstallScriptBuilder
|
||||
{
|
||||
public string Build(string backendUrl)
|
||||
{
|
||||
return $$"""
|
||||
#!/usr/bin/env bash
|
||||
# Smart Assistant installer (idempotent). Re-run to upgrade.
|
||||
set -euo pipefail
|
||||
|
||||
BACKEND_URL="{{backendUrl}}"
|
||||
ASSISTANT_HOME="$HOME/assistant"
|
||||
RESET=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--reset) RESET=1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo "Installing system deps (sudo password may be prompted)..."
|
||||
sudo apt-get update -qq
|
||||
sudo apt-get install -y --no-install-recommends python3-venv libportaudio2 alsa-utils
|
||||
|
||||
mkdir -p "$ASSISTANT_HOME/code" "$ASSISTANT_HOME/state"
|
||||
|
||||
if [ ! -d "$ASSISTANT_HOME/.venv" ]; then
|
||||
python3 -m venv "$ASSISTANT_HOME/.venv"
|
||||
fi
|
||||
|
||||
echo "Downloading client bundle from $BACKEND_URL ..."
|
||||
curl -fsSL "$BACKEND_URL/client.tar.gz" | tar -xz -C "$ASSISTANT_HOME/code"
|
||||
|
||||
"$ASSISTANT_HOME/.venv/bin/pip" install --quiet -r "$ASSISTANT_HOME/code/requirements.txt"
|
||||
"$ASSISTANT_HOME/.venv/bin/pip" install --quiet --no-deps -r "$ASSISTANT_HOME/code/requirements-nodeps.txt"
|
||||
|
||||
if [ "$RESET" -eq 1 ]; then
|
||||
rm -f "$ASSISTANT_HOME/state/config.json"
|
||||
fi
|
||||
|
||||
if [ ! -f "$ASSISTANT_HOME/state/config.json" ]; then
|
||||
echo
|
||||
read -r -p "Pairing code: " CODE
|
||||
"$ASSISTANT_HOME/.venv/bin/python" -m client.pair --backend "$BACKEND_URL" --code "$CODE"
|
||||
fi
|
||||
|
||||
mkdir -p "$HOME/.config/systemd/user"
|
||||
cat > "$HOME/.config/systemd/user/assistant.service" <<UNIT
|
||||
[Unit]
|
||||
Description=Smart Assistant Pi client
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Environment=PA_ALSA_PLUGHW=1
|
||||
ExecStart=$ASSISTANT_HOME/.venv/bin/python -m client.main
|
||||
WorkingDirectory=$ASSISTANT_HOME/code
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=default.target
|
||||
UNIT
|
||||
|
||||
loginctl enable-linger "$USER" 2>/dev/null || true
|
||||
systemctl --user daemon-reload
|
||||
systemctl --user enable --now assistant
|
||||
echo "Done. systemctl --user status assistant"
|
||||
""";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
// <auto-generated />
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611181007_Initial")]
|
||||
partial class Initial
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class Initial : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,267 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611182529_AddIdentity")]
|
||||
partial class AddIdentity
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class AddIdentity : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetRoles",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
Name = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
NormalizedName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
ConcurrencyStamp = table.Column<string>(type: "TEXT", nullable: true)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetRoles", x => x.Id);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetUsers",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
UserName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
NormalizedUserName = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
Email = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
NormalizedEmail = table.Column<string>(type: "TEXT", maxLength: 256, nullable: true),
|
||||
EmailConfirmed = table.Column<bool>(type: "INTEGER", nullable: false),
|
||||
PasswordHash = table.Column<string>(type: "TEXT", nullable: true),
|
||||
SecurityStamp = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ConcurrencyStamp = table.Column<string>(type: "TEXT", nullable: true),
|
||||
PhoneNumber = table.Column<string>(type: "TEXT", nullable: true),
|
||||
PhoneNumberConfirmed = table.Column<bool>(type: "INTEGER", nullable: false),
|
||||
TwoFactorEnabled = table.Column<bool>(type: "INTEGER", nullable: false),
|
||||
LockoutEnd = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
|
||||
LockoutEnabled = table.Column<bool>(type: "INTEGER", nullable: false),
|
||||
AccessFailedCount = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetUsers", x => x.Id);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetRoleClaims",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
.Annotation("Sqlite:Autoincrement", true),
|
||||
RoleId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
ClaimType = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ClaimValue = table.Column<string>(type: "TEXT", nullable: true)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetRoleClaims", x => x.Id);
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetRoleClaims_AspNetRoles_RoleId",
|
||||
column: x => x.RoleId,
|
||||
principalTable: "AspNetRoles",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetUserClaims",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
.Annotation("Sqlite:Autoincrement", true),
|
||||
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
ClaimType = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ClaimValue = table.Column<string>(type: "TEXT", nullable: true)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetUserClaims", x => x.Id);
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetUserClaims_AspNetUsers_UserId",
|
||||
column: x => x.UserId,
|
||||
principalTable: "AspNetUsers",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetUserLogins",
|
||||
columns: table => new
|
||||
{
|
||||
LoginProvider = table.Column<string>(type: "TEXT", nullable: false),
|
||||
ProviderKey = table.Column<string>(type: "TEXT", nullable: false),
|
||||
ProviderDisplayName = table.Column<string>(type: "TEXT", nullable: true),
|
||||
UserId = table.Column<Guid>(type: "TEXT", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetUserLogins", x => new { x.LoginProvider, x.ProviderKey });
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetUserLogins_AspNetUsers_UserId",
|
||||
column: x => x.UserId,
|
||||
principalTable: "AspNetUsers",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetUserRoles",
|
||||
columns: table => new
|
||||
{
|
||||
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
RoleId = table.Column<Guid>(type: "TEXT", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetUserRoles", x => new { x.UserId, x.RoleId });
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetUserRoles_AspNetRoles_RoleId",
|
||||
column: x => x.RoleId,
|
||||
principalTable: "AspNetRoles",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetUserRoles_AspNetUsers_UserId",
|
||||
column: x => x.UserId,
|
||||
principalTable: "AspNetUsers",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "AspNetUserTokens",
|
||||
columns: table => new
|
||||
{
|
||||
UserId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
LoginProvider = table.Column<string>(type: "TEXT", nullable: false),
|
||||
Name = table.Column<string>(type: "TEXT", nullable: false),
|
||||
Value = table.Column<string>(type: "TEXT", nullable: true)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_AspNetUserTokens", x => new { x.UserId, x.LoginProvider, x.Name });
|
||||
table.ForeignKey(
|
||||
name: "FK_AspNetUserTokens_AspNetUsers_UserId",
|
||||
column: x => x.UserId,
|
||||
principalTable: "AspNetUsers",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_AspNetRoleClaims_RoleId",
|
||||
table: "AspNetRoleClaims",
|
||||
column: "RoleId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "RoleNameIndex",
|
||||
table: "AspNetRoles",
|
||||
column: "NormalizedName",
|
||||
unique: true);
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_AspNetUserClaims_UserId",
|
||||
table: "AspNetUserClaims",
|
||||
column: "UserId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_AspNetUserLogins_UserId",
|
||||
table: "AspNetUserLogins",
|
||||
column: "UserId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_AspNetUserRoles_RoleId",
|
||||
table: "AspNetUserRoles",
|
||||
column: "RoleId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "EmailIndex",
|
||||
table: "AspNetUsers",
|
||||
column: "NormalizedEmail");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "UserNameIndex",
|
||||
table: "AspNetUsers",
|
||||
column: "NormalizedUserName",
|
||||
unique: true);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetRoleClaims");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetUserClaims");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetUserLogins");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetUserRoles");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetUserTokens");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetRoles");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "AspNetUsers");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,372 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611184053_AddDevicesAndSettings")]
|
||||
partial class AddDevicesAndSettings
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("IsRevoked")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LastSeenAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("PairedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("TokenHash")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("OwnerUserId");
|
||||
|
||||
b.HasIndex("TokenHash")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Devices");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("EnabledToolsJson")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("IdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Model")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("SystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Voice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("DeviceId");
|
||||
|
||||
b.ToTable("DeviceConfigs");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("DefaultIdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("DefaultModel")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultSystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultVoice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.ToTable("SystemSettings");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.HasOne("backend.Devices.Device", null)
|
||||
.WithOne("Config")
|
||||
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Navigation("Config");
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class AddDevicesAndSettings : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateTable(
|
||||
name: "Devices",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
OwnerUserId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
Name = table.Column<string>(type: "TEXT", nullable: false),
|
||||
PairedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
|
||||
LastSeenAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
|
||||
TokenHash = table.Column<string>(type: "TEXT", nullable: false),
|
||||
IsRevoked = table.Column<bool>(type: "INTEGER", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_Devices", x => x.Id);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "SystemSettings",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
.Annotation("Sqlite:Autoincrement", true),
|
||||
DefaultSystemPrompt = table.Column<string>(type: "TEXT", nullable: false),
|
||||
DefaultVoice = table.Column<string>(type: "TEXT", nullable: false),
|
||||
DefaultModel = table.Column<string>(type: "TEXT", nullable: false),
|
||||
DefaultIdleTimeoutSeconds = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_SystemSettings", x => x.Id);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "DeviceConfigs",
|
||||
columns: table => new
|
||||
{
|
||||
DeviceId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
SystemPrompt = table.Column<string>(type: "TEXT", nullable: false),
|
||||
Voice = table.Column<string>(type: "TEXT", nullable: false),
|
||||
Model = table.Column<string>(type: "TEXT", nullable: false),
|
||||
IdleTimeoutSeconds = table.Column<int>(type: "INTEGER", nullable: false),
|
||||
EnabledToolsJson = table.Column<string>(type: "TEXT", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_DeviceConfigs", x => x.DeviceId);
|
||||
table.ForeignKey(
|
||||
name: "FK_DeviceConfigs_Devices_DeviceId",
|
||||
column: x => x.DeviceId,
|
||||
principalTable: "Devices",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Devices_OwnerUserId",
|
||||
table: "Devices",
|
||||
column: "OwnerUserId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Devices_TokenHash",
|
||||
table: "Devices",
|
||||
column: "TokenHash",
|
||||
unique: true);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropTable(
|
||||
name: "DeviceConfigs");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "SystemSettings");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "Devices");
|
||||
}
|
||||
}
|
||||
}
|
||||
+380
@@ -0,0 +1,380 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611184536_AddDeviceOwnerFkAndPinSettingsId")]
|
||||
partial class AddDeviceOwnerFkAndPinSettingsId
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("IsRevoked")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LastSeenAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("PairedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("TokenHash")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("OwnerUserId");
|
||||
|
||||
b.HasIndex("TokenHash")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Devices");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("EnabledToolsJson")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("IdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Model")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("SystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Voice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("DeviceId");
|
||||
|
||||
b.ToTable("DeviceConfigs");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("DefaultIdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("DefaultModel")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultSystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultVoice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.ToTable("SystemSettings");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("OwnerUserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.HasOne("backend.Devices.Device", null)
|
||||
.WithOne("Config")
|
||||
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Navigation("Config");
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class AddDeviceOwnerFkAndPinSettingsId : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.AlterColumn<int>(
|
||||
name: "Id",
|
||||
table: "SystemSettings",
|
||||
type: "INTEGER",
|
||||
nullable: false,
|
||||
oldClrType: typeof(int),
|
||||
oldType: "INTEGER")
|
||||
.OldAnnotation("Sqlite:Autoincrement", true);
|
||||
|
||||
migrationBuilder.AddForeignKey(
|
||||
name: "FK_Devices_AspNetUsers_OwnerUserId",
|
||||
table: "Devices",
|
||||
column: "OwnerUserId",
|
||||
principalTable: "AspNetUsers",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropForeignKey(
|
||||
name: "FK_Devices_AspNetUsers_OwnerUserId",
|
||||
table: "Devices");
|
||||
|
||||
migrationBuilder.AlterColumn<int>(
|
||||
name: "Id",
|
||||
table: "SystemSettings",
|
||||
type: "INTEGER",
|
||||
nullable: false,
|
||||
oldClrType: typeof(int),
|
||||
oldType: "INTEGER")
|
||||
.Annotation("Sqlite:Autoincrement", true);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,405 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611185048_AddPairingCodes")]
|
||||
partial class AddPairingCodes
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("IsRevoked")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LastSeenAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("PairedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("TokenHash")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("OwnerUserId");
|
||||
|
||||
b.HasIndex("TokenHash")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Devices");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("EnabledToolsJson")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("IdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Model")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("SystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Voice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("DeviceId");
|
||||
|
||||
b.ToTable("DeviceConfigs");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
|
||||
{
|
||||
b.Property<string>("Code")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset?>("ConsumedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DeviceName")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("ExpiresAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Code");
|
||||
|
||||
b.HasIndex("ExpiresAt");
|
||||
|
||||
b.ToTable("PairingCodes");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("DefaultIdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("DefaultModel")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultSystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultVoice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.ToTable("SystemSettings");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("OwnerUserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.HasOne("backend.Devices.Device", null)
|
||||
.WithOne("Config")
|
||||
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Navigation("Config");
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class AddPairingCodes : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateTable(
|
||||
name: "PairingCodes",
|
||||
columns: table => new
|
||||
{
|
||||
Code = table.Column<string>(type: "TEXT", nullable: false),
|
||||
OwnerUserId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
DeviceName = table.Column<string>(type: "TEXT", nullable: false),
|
||||
ExpiresAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
|
||||
ConsumedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_PairingCodes", x => x.Code);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_PairingCodes_ExpiresAt",
|
||||
table: "PairingCodes",
|
||||
column: "ExpiresAt");
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropTable(
|
||||
name: "PairingCodes");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,482 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
[Migration("20260611210852_AddConversationsAndTurns")]
|
||||
partial class AddConversationsAndTurns
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Conversation", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("EndReason")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("EndedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("StartedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("DeviceId");
|
||||
|
||||
b.ToTable("Conversations");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Turn", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("ConversationId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("CreatedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("Index")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("Role")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Text")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolArgsJson")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolResultJson")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("ConversationId", "Index")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Turns");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("IsRevoked")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LastSeenAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("PairedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("TokenHash")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("OwnerUserId");
|
||||
|
||||
b.HasIndex("TokenHash")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Devices");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("EnabledToolsJson")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("IdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Model")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("SystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Voice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("DeviceId");
|
||||
|
||||
b.ToTable("DeviceConfigs");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
|
||||
{
|
||||
b.Property<string>("Code")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset?>("ConsumedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DeviceName")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("ExpiresAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Code");
|
||||
|
||||
b.HasIndex("ExpiresAt");
|
||||
|
||||
b.ToTable("PairingCodes");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("DefaultIdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("DefaultModel")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultSystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultVoice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.ToTable("SystemSettings");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Turn", b =>
|
||||
{
|
||||
b.HasOne("backend.Conversations.Conversation", null)
|
||||
.WithMany("Turns")
|
||||
.HasForeignKey("ConversationId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("OwnerUserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.HasOne("backend.Devices.Device", null)
|
||||
.WithOne("Config")
|
||||
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Conversation", b =>
|
||||
{
|
||||
b.Navigation("Turns");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Navigation("Config");
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class AddConversationsAndTurns : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateTable(
|
||||
name: "Conversations",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
DeviceId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
StartedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false),
|
||||
EndedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: true),
|
||||
EndReason = table.Column<int>(type: "INTEGER", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_Conversations", x => x.Id);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateTable(
|
||||
name: "Turns",
|
||||
columns: table => new
|
||||
{
|
||||
Id = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
ConversationId = table.Column<Guid>(type: "TEXT", nullable: false),
|
||||
Index = table.Column<int>(type: "INTEGER", nullable: false),
|
||||
Role = table.Column<int>(type: "INTEGER", nullable: false),
|
||||
Text = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ToolName = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ToolArgsJson = table.Column<string>(type: "TEXT", nullable: true),
|
||||
ToolResultJson = table.Column<string>(type: "TEXT", nullable: true),
|
||||
CreatedAt = table.Column<DateTimeOffset>(type: "TEXT", nullable: false)
|
||||
},
|
||||
constraints: table =>
|
||||
{
|
||||
table.PrimaryKey("PK_Turns", x => x.Id);
|
||||
table.ForeignKey(
|
||||
name: "FK_Turns_Conversations_ConversationId",
|
||||
column: x => x.ConversationId,
|
||||
principalTable: "Conversations",
|
||||
principalColumn: "Id",
|
||||
onDelete: ReferentialAction.Cascade);
|
||||
});
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Conversations_DeviceId",
|
||||
table: "Conversations",
|
||||
column: "DeviceId");
|
||||
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_Turns_ConversationId_Index",
|
||||
table: "Turns",
|
||||
columns: new[] { "ConversationId", "Index" },
|
||||
unique: true);
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.DropTable(
|
||||
name: "Turns");
|
||||
|
||||
migrationBuilder.DropTable(
|
||||
name: "Conversations");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,479 @@
|
||||
// <auto-generated />
|
||||
using System;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||
using backend.Data;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace backend.Migrations
|
||||
{
|
||||
[DbContext(typeof(AppDbContext))]
|
||||
partial class AppDbContextModelSnapshot : ModelSnapshot
|
||||
{
|
||||
protected override void BuildModel(ModelBuilder modelBuilder)
|
||||
{
|
||||
#pragma warning disable 612, 618
|
||||
modelBuilder.HasAnnotation("ProductVersion", "9.0.17");
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("RoleNameIndex");
|
||||
|
||||
b.ToTable("AspNetRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetRoleClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ClaimType")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ClaimValue")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserClaims", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderKey")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ProviderDisplayName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("LoginProvider", "ProviderKey");
|
||||
|
||||
b.HasIndex("UserId");
|
||||
|
||||
b.ToTable("AspNetUserLogins", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("RoleId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "RoleId");
|
||||
|
||||
b.HasIndex("RoleId");
|
||||
|
||||
b.ToTable("AspNetUserRoles", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.Property<Guid>("UserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("LoginProvider")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Value")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("UserId", "LoginProvider", "Name");
|
||||
|
||||
b.ToTable("AspNetUserTokens", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Auth.ApplicationUser", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("AccessFailedCount")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("ConcurrencyStamp")
|
||||
.IsConcurrencyToken()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Email")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("EmailConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<bool>("LockoutEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LockoutEnd")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedEmail")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("NormalizedUserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PasswordHash")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PhoneNumber")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("PhoneNumberConfirmed")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("SecurityStamp")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("TwoFactorEnabled")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("UserName")
|
||||
.HasMaxLength(256)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("NormalizedEmail")
|
||||
.HasDatabaseName("EmailIndex");
|
||||
|
||||
b.HasIndex("NormalizedUserName")
|
||||
.IsUnique()
|
||||
.HasDatabaseName("UserNameIndex");
|
||||
|
||||
b.ToTable("AspNetUsers", (string)null);
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Conversation", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("EndReason")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("EndedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("StartedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("DeviceId");
|
||||
|
||||
b.ToTable("Conversations");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Turn", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("ConversationId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("CreatedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("Index")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("Role")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Text")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolArgsJson")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolName")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("ToolResultJson")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("ConversationId", "Index")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Turns");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Property<Guid>("Id")
|
||||
.ValueGeneratedOnAdd()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<bool>("IsRevoked")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<DateTimeOffset?>("LastSeenAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Name")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("PairedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("TokenHash")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("OwnerUserId");
|
||||
|
||||
b.HasIndex("TokenHash")
|
||||
.IsUnique();
|
||||
|
||||
b.ToTable("Devices");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.Property<Guid>("DeviceId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("EnabledToolsJson")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<int>("IdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("Model")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("SystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("Voice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("DeviceId");
|
||||
|
||||
b.ToTable("DeviceConfigs");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Pairing.PairingCode", b =>
|
||||
{
|
||||
b.Property<string>("Code")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset?>("ConsumedAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DeviceName")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<DateTimeOffset>("ExpiresAt")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<Guid>("OwnerUserId")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Code");
|
||||
|
||||
b.HasIndex("ExpiresAt");
|
||||
|
||||
b.ToTable("PairingCodes");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Settings.SystemSettings", b =>
|
||||
{
|
||||
b.Property<int>("Id")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<int>("DefaultIdleTimeoutSeconds")
|
||||
.HasColumnType("INTEGER");
|
||||
|
||||
b.Property<string>("DefaultModel")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultSystemPrompt")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("DefaultVoice")
|
||||
.IsRequired()
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.ToTable("SystemSettings");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityRoleClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserClaim<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserLogin<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserRole<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("Microsoft.AspNetCore.Identity.IdentityRole<System.Guid>", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("RoleId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("Microsoft.AspNetCore.Identity.IdentityUserToken<System.Guid>", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("UserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Turn", b =>
|
||||
{
|
||||
b.HasOne("backend.Conversations.Conversation", null)
|
||||
.WithMany("Turns")
|
||||
.HasForeignKey("ConversationId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.HasOne("backend.Auth.ApplicationUser", null)
|
||||
.WithMany()
|
||||
.HasForeignKey("OwnerUserId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.DeviceConfig", b =>
|
||||
{
|
||||
b.HasOne("backend.Devices.Device", null)
|
||||
.WithOne("Config")
|
||||
.HasForeignKey("backend.Devices.DeviceConfig", "DeviceId")
|
||||
.OnDelete(DeleteBehavior.Cascade)
|
||||
.IsRequired();
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Conversations.Conversation", b =>
|
||||
{
|
||||
b.Navigation("Turns");
|
||||
});
|
||||
|
||||
modelBuilder.Entity("backend.Devices.Device", b =>
|
||||
{
|
||||
b.Navigation("Config");
|
||||
});
|
||||
#pragma warning restore 612, 618
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
namespace backend.Pairing;
|
||||
|
||||
public class PairingCode
|
||||
{
|
||||
public string Code { get; set; } = ""; // 8 chars, primary key
|
||||
public Guid OwnerUserId { get; set; }
|
||||
public string DeviceName { get; set; } = "";
|
||||
public DateTimeOffset ExpiresAt { get; set; }
|
||||
public DateTimeOffset? ConsumedAt { get; set; }
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
using System.Security.Claims;
|
||||
using Microsoft.AspNetCore.Mvc;
|
||||
|
||||
namespace backend.Pairing;
|
||||
|
||||
public record IssueCodeRequest(string Name);
|
||||
public record PairRequest(string Code, string Hostname, string Client_version);
|
||||
|
||||
public static class PairingEndpoints
|
||||
{
|
||||
public static IEndpointRouteBuilder MapPairingEndpoints(this IEndpointRouteBuilder app)
|
||||
{
|
||||
app.MapPost("/api/pair-code", async (
|
||||
[FromBody] IssueCodeRequest req,
|
||||
PairingService svc,
|
||||
HttpContext http,
|
||||
CancellationToken ct) =>
|
||||
{
|
||||
var uidStr = http.User.FindFirstValue(ClaimTypes.NameIdentifier);
|
||||
if (!Guid.TryParse(uidStr, out var uid)) return Results.Unauthorized();
|
||||
if (string.IsNullOrWhiteSpace(req.Name))
|
||||
return Results.BadRequest(new { error = "name is required" });
|
||||
var pc = await svc.IssueAsync(uid, req.Name.Trim(), ct);
|
||||
return Results.Ok(new { code = pc.Code, expires_at = pc.ExpiresAt });
|
||||
}).RequireAuthorization();
|
||||
|
||||
app.MapPost("/api/pair", async (
|
||||
[FromBody] PairRequest req,
|
||||
PairingService svc,
|
||||
HttpContext http,
|
||||
CancellationToken ct) =>
|
||||
{
|
||||
var result = await svc.ConsumeAsync(req.Code, ct);
|
||||
if (result is null) return Results.NotFound(new { error = "code invalid or expired" });
|
||||
var scheme = http.Request.Scheme == "https" ? "wss" : "ws";
|
||||
var host = http.Request.Host.ToString();
|
||||
return Results.Ok(new
|
||||
{
|
||||
device_id = result.DeviceId,
|
||||
device_token = result.DeviceToken,
|
||||
backend_ws = $"{scheme}://{host}/device",
|
||||
});
|
||||
});
|
||||
|
||||
return app;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
using System.Security.Cryptography;
|
||||
using backend.Data;
|
||||
using backend.Devices;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace backend.Pairing;
|
||||
|
||||
public record PairResult(Guid DeviceId, string DeviceToken);
|
||||
|
||||
public class PairingService
|
||||
{
|
||||
private const string Alphabet = "ABCDEFGHJKMNPQRSTUVWXYZ23456789"; // no O, 0, I, 1, L
|
||||
private const int CodeLength = 8;
|
||||
public static readonly TimeSpan CodeTtl = TimeSpan.FromMinutes(10);
|
||||
|
||||
private readonly AppDbContext? _db;
|
||||
private readonly DeviceTokenService? _tokens;
|
||||
|
||||
public PairingService() { }
|
||||
|
||||
public PairingService(AppDbContext db, DeviceTokenService tokens)
|
||||
{
|
||||
_db = db;
|
||||
_tokens = tokens;
|
||||
}
|
||||
|
||||
public string GenerateCode()
|
||||
{
|
||||
Span<byte> bytes = stackalloc byte[CodeLength];
|
||||
RandomNumberGenerator.Fill(bytes);
|
||||
Span<char> chars = stackalloc char[CodeLength];
|
||||
for (var i = 0; i < CodeLength; i++)
|
||||
chars[i] = Alphabet[bytes[i] % Alphabet.Length];
|
||||
return new string(chars);
|
||||
}
|
||||
|
||||
public async Task<PairingCode> IssueAsync(Guid ownerUserId, string deviceName, CancellationToken ct)
|
||||
{
|
||||
if (_db is null) throw new InvalidOperationException("PairingService not constructed with DB context.");
|
||||
var entity = new PairingCode
|
||||
{
|
||||
Code = GenerateCode(),
|
||||
OwnerUserId = ownerUserId,
|
||||
DeviceName = deviceName,
|
||||
ExpiresAt = DateTimeOffset.UtcNow.Add(CodeTtl),
|
||||
};
|
||||
_db.PairingCodes.Add(entity);
|
||||
await _db.SaveChangesAsync(ct);
|
||||
return entity;
|
||||
}
|
||||
|
||||
public async Task<PairResult?> ConsumeAsync(string code, CancellationToken ct)
|
||||
{
|
||||
if (_db is null || _tokens is null)
|
||||
throw new InvalidOperationException("PairingService not constructed with DB context + token service.");
|
||||
var now = DateTimeOffset.UtcNow;
|
||||
var pc = await _db.PairingCodes.FirstOrDefaultAsync(p => p.Code == code, ct);
|
||||
if (pc is null || pc.ConsumedAt is not null || pc.ExpiresAt < now)
|
||||
return null;
|
||||
|
||||
var (plaintext, hash) = _tokens.Generate();
|
||||
var defaults = await _db.SystemSettings.FirstAsync(ct);
|
||||
|
||||
var device = new Device
|
||||
{
|
||||
OwnerUserId = pc.OwnerUserId,
|
||||
Name = pc.DeviceName,
|
||||
PairedAt = now,
|
||||
TokenHash = hash,
|
||||
};
|
||||
device.Config = new DeviceConfig
|
||||
{
|
||||
DeviceId = device.Id,
|
||||
SystemPrompt = defaults.DefaultSystemPrompt,
|
||||
Voice = defaults.DefaultVoice,
|
||||
Model = defaults.DefaultModel,
|
||||
IdleTimeoutSeconds = defaults.DefaultIdleTimeoutSeconds,
|
||||
};
|
||||
_db.Devices.Add(device);
|
||||
pc.ConsumedAt = now;
|
||||
await _db.SaveChangesAsync(ct);
|
||||
return new PairResult(device.Id, plaintext);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
using backend.Auth;
|
||||
using backend.Data;
|
||||
using backend.DeviceHub;
|
||||
using backend.Devices;
|
||||
using backend.Install;
|
||||
using backend.Pairing;
|
||||
using Microsoft.AspNetCore.DataProtection;
|
||||
using Microsoft.AspNetCore.HttpOverrides;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
var builder = WebApplication.CreateBuilder(args);
|
||||
|
||||
var keysDir = Environment.GetEnvironmentVariable("DataProtection__KeysDir") ?? "/keys";
|
||||
if (Directory.Exists(keysDir))
|
||||
{
|
||||
builder.Services.AddDataProtection()
|
||||
.PersistKeysToFileSystem(new DirectoryInfo(keysDir))
|
||||
.SetApplicationName("smart-assistant");
|
||||
}
|
||||
|
||||
builder.Services.Configure<ForwardedHeadersOptions>(opt =>
|
||||
{
|
||||
opt.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
|
||||
opt.KnownNetworks.Clear();
|
||||
opt.KnownProxies.Clear();
|
||||
});
|
||||
|
||||
builder.Services.AddDbContext<AppDbContext>(opt =>
|
||||
opt.UseSqlite(builder.Configuration.GetConnectionString("Default")));
|
||||
|
||||
builder.Services.AddAppIdentity();
|
||||
builder.Services.AddAuthorization();
|
||||
|
||||
builder.Services.AddScoped<backend.Devices.DeviceTokenService>();
|
||||
builder.Services.AddScoped<backend.Pairing.PairingService>();
|
||||
builder.Services.AddSingleton<backend.Install.InstallScriptBuilder>();
|
||||
builder.Services.AddScoped<backend.Conversations.ConversationLog>();
|
||||
builder.Services.AddSingleton<backend.Realtime.OpenAIKeyProvider>();
|
||||
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.GetCurrentTimeTool>();
|
||||
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.EndSessionTool>();
|
||||
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.SetVolumeTool>();
|
||||
builder.Services.AddSingleton<backend.Tools.ToolRegistry>();
|
||||
builder.Services.AddScoped<backend.Realtime.RealtimeSessionFactory>();
|
||||
builder.Services.AddSingleton<backend.DeviceHub.DeviceRegistry>();
|
||||
builder.Services.AddScoped<backend.DeviceHub.DeviceAuth>();
|
||||
|
||||
var app = builder.Build();
|
||||
|
||||
using (var scope = app.Services.CreateScope())
|
||||
{
|
||||
var sp = scope.ServiceProvider;
|
||||
var db = sp.GetRequiredService<AppDbContext>();
|
||||
db.Database.Migrate();
|
||||
await IdentitySetup.SeedRolesAsync(sp);
|
||||
|
||||
if (db.SystemSettings.Find(1) is null)
|
||||
{
|
||||
db.SystemSettings.Add(new backend.Settings.SystemSettings { Id = 1 });
|
||||
await db.SaveChangesAsync();
|
||||
}
|
||||
}
|
||||
|
||||
app.UseForwardedHeaders();
|
||||
app.UseAuthentication();
|
||||
app.UseAuthorization();
|
||||
|
||||
app.MapAuthEndpoints();
|
||||
app.MapPairingEndpoints();
|
||||
app.MapDevicesEndpoints();
|
||||
app.MapInstallEndpoints();
|
||||
|
||||
app.MapGet("/health", () => Results.Ok(new { ok = true }));
|
||||
|
||||
app.UseWebSockets();
|
||||
app.MapDeviceHub();
|
||||
|
||||
app.Run();
|
||||
|
||||
public partial class Program { }
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"$schema": "https://json.schemastore.org/launchsettings.json",
|
||||
"profiles": {
|
||||
"http": {
|
||||
"commandName": "Project",
|
||||
"dotnetRunMessages": true,
|
||||
"launchBrowser": true,
|
||||
"applicationUrl": "http://localhost:5252",
|
||||
"environmentVariables": {
|
||||
"ASPNETCORE_ENVIRONMENT": "Development"
|
||||
}
|
||||
},
|
||||
"https": {
|
||||
"commandName": "Project",
|
||||
"dotnetRunMessages": true,
|
||||
"launchBrowser": true,
|
||||
"applicationUrl": "https://localhost:7061;http://localhost:5252",
|
||||
"environmentVariables": {
|
||||
"ASPNETCORE_ENVIRONMENT": "Development"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
namespace backend.Realtime;
|
||||
|
||||
public interface IRealtimeUpstream : IAsyncDisposable
|
||||
{
|
||||
Task SendJsonAsync(JsonObject envelope, CancellationToken ct);
|
||||
Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct);
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
namespace backend.Realtime;
|
||||
|
||||
public class OpenAIKeyProvider
|
||||
{
|
||||
private readonly string? _key;
|
||||
|
||||
public OpenAIKeyProvider(IConfiguration cfg)
|
||||
{
|
||||
_key = cfg["OPENAI_API_KEY"] ?? cfg["OpenAI:ApiKey"];
|
||||
}
|
||||
|
||||
public bool HasKey => !string.IsNullOrWhiteSpace(_key);
|
||||
|
||||
public string GetRequired()
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(_key))
|
||||
throw new InvalidOperationException(
|
||||
"OPENAI_API_KEY is not configured. Set the env var on the Coolify app.");
|
||||
return _key!;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
using System.Net.WebSockets;
|
||||
using System.Text;
|
||||
using System.Text.Json.Nodes;
|
||||
|
||||
namespace backend.Realtime;
|
||||
|
||||
public class OpenAIRealtimeUpstream : IRealtimeUpstream
|
||||
{
|
||||
private readonly ClientWebSocket _ws = new();
|
||||
|
||||
public async Task ConnectAsync(string apiKey, string model, CancellationToken ct)
|
||||
{
|
||||
_ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
|
||||
_ws.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
|
||||
var uri = new Uri($"wss://api.openai.com/v1/realtime?model={Uri.EscapeDataString(model)}");
|
||||
await _ws.ConnectAsync(uri, ct);
|
||||
}
|
||||
|
||||
public async Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
|
||||
{
|
||||
var json = envelope.ToJsonString();
|
||||
var bytes = Encoding.UTF8.GetBytes(json);
|
||||
await _ws.SendAsync(bytes, WebSocketMessageType.Text, endOfMessage: true, ct);
|
||||
}
|
||||
|
||||
public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
|
||||
{
|
||||
var buffer = new byte[8192];
|
||||
using var stream = new MemoryStream();
|
||||
while (true)
|
||||
{
|
||||
WebSocketReceiveResult result;
|
||||
try { result = await _ws.ReceiveAsync(buffer, ct); }
|
||||
catch (WebSocketException) { return null; }
|
||||
catch (OperationCanceledException) { return null; }
|
||||
|
||||
if (result.MessageType == WebSocketMessageType.Close) return null;
|
||||
stream.Write(buffer, 0, result.Count);
|
||||
if (result.EndOfMessage)
|
||||
{
|
||||
var json = Encoding.UTF8.GetString(stream.ToArray());
|
||||
return JsonNode.Parse(json)?.AsObject();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public async ValueTask DisposeAsync()
|
||||
{
|
||||
if (_ws.State == WebSocketState.Open)
|
||||
{
|
||||
try { await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None); }
|
||||
catch { }
|
||||
}
|
||||
_ws.Dispose();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Tools;
|
||||
|
||||
namespace backend.Realtime;
|
||||
|
||||
public static class RealtimeEvents
|
||||
{
|
||||
public static JsonObject SessionUpdate(RealtimeSettings cfg, IEnumerable<ITool> enabledTools)
|
||||
{
|
||||
var tools = new JsonArray();
|
||||
foreach (var t in enabledTools)
|
||||
{
|
||||
tools.Add(new JsonObject
|
||||
{
|
||||
["type"] = "function",
|
||||
["name"] = t.Name,
|
||||
["description"] = t.Description,
|
||||
["parameters"] = JsonNode.Parse(t.ParameterSchema.GetRawText())!,
|
||||
});
|
||||
}
|
||||
|
||||
return new JsonObject
|
||||
{
|
||||
["type"] = "session.update",
|
||||
["session"] = new JsonObject
|
||||
{
|
||||
["voice"] = cfg.Voice,
|
||||
["instructions"] = cfg.SystemPrompt,
|
||||
["modalities"] = new JsonArray("text", "audio"),
|
||||
["input_audio_format"] = "pcm16",
|
||||
["output_audio_format"] = "pcm16",
|
||||
["input_audio_transcription"] = new JsonObject { ["model"] = "whisper-1" },
|
||||
["turn_detection"] = new JsonObject
|
||||
{
|
||||
["type"] = "server_vad",
|
||||
["threshold"] = 0.5,
|
||||
["prefix_padding_ms"] = 300,
|
||||
["silence_duration_ms"] = 500,
|
||||
},
|
||||
["tools"] = tools,
|
||||
["tool_choice"] = "auto",
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
public static JsonObject InputAudioBufferAppend(ReadOnlySpan<byte> pcm16)
|
||||
{
|
||||
return new JsonObject
|
||||
{
|
||||
["type"] = "input_audio_buffer.append",
|
||||
["audio"] = Convert.ToBase64String(pcm16),
|
||||
};
|
||||
}
|
||||
|
||||
public static JsonObject FunctionCallOutput(string callId, string outputJson)
|
||||
{
|
||||
return new JsonObject
|
||||
{
|
||||
["type"] = "conversation.item.create",
|
||||
["item"] = new JsonObject
|
||||
{
|
||||
["type"] = "function_call_output",
|
||||
["call_id"] = callId,
|
||||
["output"] = outputJson,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
public static JsonObject ResponseCreate() => new() { ["type"] = "response.create" };
|
||||
}
|
||||
@@ -0,0 +1,280 @@
|
||||
using System.Text.Json.Nodes;
|
||||
using backend.Conversations;
|
||||
using backend.Tools;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace backend.Realtime;
|
||||
|
||||
public interface ISessionOutput
|
||||
{
|
||||
Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct);
|
||||
Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
|
||||
}
|
||||
|
||||
public class RealtimeSession
|
||||
{
|
||||
private readonly Guid _deviceId;
|
||||
private readonly IRealtimeUpstream _upstream;
|
||||
private readonly ISessionOutput _output;
|
||||
private readonly ConversationLog _log;
|
||||
private readonly ToolRegistry _registry;
|
||||
private readonly RealtimeSettings _cfg;
|
||||
private readonly IDeviceChannel _channel;
|
||||
private readonly ILogger _logger;
|
||||
|
||||
private Conversation? _conversation;
|
||||
private EndReason _endReason = EndReason.Unknown;
|
||||
private bool _endRequested;
|
||||
private DateTime _lastSpeechAt = DateTime.UtcNow;
|
||||
private readonly List<(string CallId, string Name, string ArgsJson)> _pendingCalls = new();
|
||||
|
||||
private readonly System.Threading.Channels.Channel<byte[]> _uplink =
|
||||
System.Threading.Channels.Channel.CreateBounded<byte[]>(
|
||||
new System.Threading.Channels.BoundedChannelOptions(64)
|
||||
{
|
||||
FullMode = System.Threading.Channels.BoundedChannelFullMode.DropOldest,
|
||||
});
|
||||
|
||||
public ValueTask WriteUplinkAsync(byte[] frame, CancellationToken ct)
|
||||
=> _uplink.Writer.WriteAsync(frame, ct);
|
||||
|
||||
public RealtimeSession(
|
||||
Guid deviceId,
|
||||
IRealtimeUpstream upstream,
|
||||
ISessionOutput output,
|
||||
ConversationLog log,
|
||||
ToolRegistry registry,
|
||||
RealtimeSettings cfg,
|
||||
IDeviceChannel channel,
|
||||
ILogger logger)
|
||||
{
|
||||
_deviceId = deviceId;
|
||||
_upstream = upstream;
|
||||
_output = output;
|
||||
_log = log;
|
||||
_registry = registry;
|
||||
_cfg = cfg;
|
||||
_channel = channel;
|
||||
_logger = logger;
|
||||
}
|
||||
|
||||
public Guid? ConversationId => _conversation?.Id;
|
||||
|
||||
public async Task RunAsync(CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
var enabled = _registry.EnabledFor(_cfg.EnabledTools).ToList();
|
||||
await _upstream.SendJsonAsync(RealtimeEvents.SessionUpdate(_cfg, enabled), ct);
|
||||
|
||||
var ack = await WaitForUpstreamTypeAsync("session.updated", ct);
|
||||
if (ack is null)
|
||||
{
|
||||
_endReason = EndReason.Error;
|
||||
return;
|
||||
}
|
||||
|
||||
_conversation = await _log.StartAsync(_deviceId, ct);
|
||||
await _output.WriteEnvelopeAsync(new JsonObject
|
||||
{
|
||||
["type"] = "session_started",
|
||||
["conversation_id"] = _conversation.Id.ToString(),
|
||||
}, ct);
|
||||
_lastSpeechAt = DateTime.UtcNow;
|
||||
|
||||
var uplinkTask = Task.Run(() => PumpUplinkAsync(ct), ct);
|
||||
var idleTask = Task.Run(() => WatchdogAsync(ct), ct);
|
||||
await PumpAsync(ct);
|
||||
_uplink.Writer.TryComplete();
|
||||
await uplinkTask;
|
||||
await idleTask;
|
||||
}
|
||||
catch (OperationCanceledException)
|
||||
{
|
||||
if (_endReason == EndReason.Unknown) _endReason = EndReason.Error;
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogError(ex, "RealtimeSession crashed");
|
||||
_endReason = EndReason.Error;
|
||||
}
|
||||
finally
|
||||
{
|
||||
await _upstream.DisposeAsync();
|
||||
if (_conversation is not null)
|
||||
{
|
||||
await _log.EndAsync(_conversation.Id, _endReason, CancellationToken.None);
|
||||
}
|
||||
var endEnv = new JsonObject
|
||||
{
|
||||
["type"] = "session_ended",
|
||||
["reason"] = _endReason.ToString().ToLowerInvariant(),
|
||||
};
|
||||
await _output.WriteEnvelopeAsync(endEnv, CancellationToken.None);
|
||||
}
|
||||
}
|
||||
|
||||
private async Task PumpAsync(CancellationToken ct)
|
||||
{
|
||||
while (!ct.IsCancellationRequested && !_endRequested)
|
||||
{
|
||||
var evt = await ReceiveOrIdleAsync(ct);
|
||||
if (evt is null && _endRequested) return;
|
||||
if (evt is null && ct.IsCancellationRequested) return;
|
||||
if (evt is null) continue;
|
||||
var type = (string?)evt["type"];
|
||||
switch (type)
|
||||
{
|
||||
case "input_audio_buffer.speech_started":
|
||||
_lastSpeechAt = DateTime.UtcNow;
|
||||
break;
|
||||
case "response.audio.delta":
|
||||
await ForwardAudioDeltaAsync(evt, ct);
|
||||
break;
|
||||
case "conversation.item.input_audio_transcription.completed":
|
||||
{
|
||||
var text = (string?)evt["transcript"];
|
||||
if (_conversation is not null && !string.IsNullOrEmpty(text))
|
||||
await _log.AppendUserAsync(_conversation.Id, text!, ct);
|
||||
break;
|
||||
}
|
||||
case "response.audio_transcript.done":
|
||||
{
|
||||
var text = (string?)evt["transcript"];
|
||||
if (_conversation is not null && !string.IsNullOrEmpty(text))
|
||||
await _log.AppendAssistantAsync(_conversation.Id, text!, ct);
|
||||
break;
|
||||
}
|
||||
case "response.function_call_arguments.done":
|
||||
{
|
||||
var callId = (string?)evt["call_id"] ?? "";
|
||||
var name = (string?)evt["name"] ?? "";
|
||||
var args = (string?)evt["arguments"] ?? "{}";
|
||||
_pendingCalls.Add((callId, name, args));
|
||||
break;
|
||||
}
|
||||
case "response.done":
|
||||
{
|
||||
await _output.WriteEnvelopeAsync(
|
||||
new JsonObject { ["type"] = "assistant_done" }, ct);
|
||||
|
||||
if (_pendingCalls.Count == 0) break;
|
||||
var calls = _pendingCalls.ToList();
|
||||
_pendingCalls.Clear();
|
||||
|
||||
if (calls.Any(c => c.Name == "end_session"))
|
||||
{
|
||||
_endReason = EndReason.Tool;
|
||||
_endRequested = true;
|
||||
break;
|
||||
}
|
||||
|
||||
foreach (var c in calls)
|
||||
{
|
||||
await ExecuteToolAsync(c.CallId, c.Name, c.ArgsJson, ct);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async Task ExecuteToolAsync(string callId, string name, string argsJson, CancellationToken ct)
|
||||
{
|
||||
var tool = _registry.Get(name);
|
||||
string outputJson;
|
||||
if (tool is null)
|
||||
{
|
||||
outputJson = $$"""{"ok":false,"error":"unknown tool: {{name}}"}""";
|
||||
}
|
||||
else
|
||||
{
|
||||
try
|
||||
{
|
||||
using var argsDoc = System.Text.Json.JsonDocument.Parse(argsJson);
|
||||
var ctx = new DeviceContext(_deviceId, _conversation!.Id, _channel);
|
||||
var res = await tool.ExecuteAsync(
|
||||
new ToolInvocation(callId, argsDoc.RootElement.Clone()), ctx, ct);
|
||||
outputJson = res.Output.GetRawText();
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
outputJson = System.Text.Json.JsonSerializer.Serialize(new
|
||||
{
|
||||
ok = false,
|
||||
error = ex.Message,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (_conversation is not null)
|
||||
{
|
||||
await _log.AppendToolAsync(_conversation.Id, name, argsJson, outputJson, ct);
|
||||
}
|
||||
|
||||
await _upstream.SendJsonAsync(RealtimeEvents.FunctionCallOutput(callId, outputJson), ct);
|
||||
await _upstream.SendJsonAsync(RealtimeEvents.ResponseCreate(), ct);
|
||||
}
|
||||
|
||||
private async Task ForwardAudioDeltaAsync(JsonObject evt, CancellationToken ct)
|
||||
{
|
||||
var b64 = (string?)evt["delta"];
|
||||
if (string.IsNullOrEmpty(b64)) return;
|
||||
var bytes = Convert.FromBase64String(b64);
|
||||
await _output.WriteBinaryAsync(bytes, ct);
|
||||
}
|
||||
|
||||
private async Task WatchdogAsync(CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
var timeout = TimeSpan.FromSeconds(_cfg.IdleTimeoutSeconds);
|
||||
while (!ct.IsCancellationRequested && !_endRequested)
|
||||
{
|
||||
await Task.Delay(250, ct);
|
||||
if (DateTime.UtcNow - _lastSpeechAt > timeout)
|
||||
{
|
||||
_endReason = EndReason.Idle;
|
||||
_endRequested = true;
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { }
|
||||
}
|
||||
|
||||
private async Task<JsonObject?> ReceiveOrIdleAsync(CancellationToken ct)
|
||||
{
|
||||
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
|
||||
var receive = _upstream.ReceiveJsonAsync(timeout.Token);
|
||||
var delay = Task.Delay(250, timeout.Token);
|
||||
var done = await Task.WhenAny(receive, delay);
|
||||
if (done == receive) return await receive;
|
||||
timeout.Cancel();
|
||||
try { await receive; } catch { /* ignore */ }
|
||||
return null;
|
||||
}
|
||||
|
||||
private async Task PumpUplinkAsync(CancellationToken ct)
|
||||
{
|
||||
try
|
||||
{
|
||||
await foreach (var frame in _uplink.Reader.ReadAllAsync(ct))
|
||||
{
|
||||
await _upstream.SendJsonAsync(RealtimeEvents.InputAudioBufferAppend(frame), ct);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { }
|
||||
catch (Exception ex) { _logger.LogWarning(ex, "uplink pump errored"); }
|
||||
}
|
||||
|
||||
private async Task<JsonObject?> WaitForUpstreamTypeAsync(string type, CancellationToken ct)
|
||||
{
|
||||
while (true)
|
||||
{
|
||||
var evt = await _upstream.ReceiveJsonAsync(ct);
|
||||
if (evt is null) return null;
|
||||
if ((string?)evt["type"] == type) return evt;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
using backend.Conversations;
|
||||
using backend.Tools;
|
||||
using Microsoft.Extensions.Logging;
|
||||
|
||||
namespace backend.Realtime;
|
||||
|
||||
public class RealtimeSessionFactory
|
||||
{
|
||||
protected readonly OpenAIKeyProvider keys;
|
||||
protected readonly ToolRegistry registry;
|
||||
protected readonly ConversationLog log;
|
||||
protected readonly ILoggerFactory loggerFactory;
|
||||
|
||||
public RealtimeSessionFactory(
|
||||
OpenAIKeyProvider keys, ToolRegistry registry,
|
||||
ConversationLog log, ILoggerFactory loggerFactory)
|
||||
{
|
||||
this.keys = keys;
|
||||
this.registry = registry;
|
||||
this.log = log;
|
||||
this.loggerFactory = loggerFactory;
|
||||
}
|
||||
|
||||
public virtual async Task<RealtimeSession> CreateAsync(
|
||||
Guid deviceId,
|
||||
RealtimeSettings cfg,
|
||||
ISessionOutput output,
|
||||
IDeviceChannel channel,
|
||||
CancellationToken ct)
|
||||
{
|
||||
var upstream = new OpenAIRealtimeUpstream();
|
||||
await upstream.ConnectAsync(keys.GetRequired(), cfg.Model, ct);
|
||||
return new RealtimeSession(
|
||||
deviceId, upstream, output, log, registry, cfg, channel,
|
||||
loggerFactory.CreateLogger<RealtimeSession>());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
namespace backend.Realtime;
|
||||
|
||||
public record RealtimeSettings(
|
||||
string Model,
|
||||
string Voice,
|
||||
string SystemPrompt,
|
||||
int IdleTimeoutSeconds,
|
||||
IReadOnlySet<string> EnabledTools);
|
||||
@@ -0,0 +1,11 @@
|
||||
namespace backend.Settings;
|
||||
|
||||
public class SystemSettings
|
||||
{
|
||||
public int Id { get; set; } = 1;
|
||||
public string DefaultSystemPrompt { get; set; } =
|
||||
"You are a helpful voice assistant. Keep responses concise.";
|
||||
public string DefaultVoice { get; set; } = "alloy";
|
||||
public string DefaultModel { get; set; } = "gpt-4o-realtime-preview";
|
||||
public int DefaultIdleTimeoutSeconds { get; set; } = 30;
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
using System.Text.Json;
|
||||
|
||||
namespace backend.Tools;
|
||||
|
||||
public class EndSessionTool : ITool
|
||||
{
|
||||
public string Name => "end_session";
|
||||
public string Description =>
|
||||
"Call this when the user has said goodbye or otherwise indicates the conversation is over.";
|
||||
public bool RunsDuringResponse => false;
|
||||
|
||||
public JsonElement ParameterSchema { get; } =
|
||||
JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;
|
||||
|
||||
public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
|
||||
{
|
||||
var output = JsonDocument.Parse("""{"ok":true}""").RootElement;
|
||||
return Task.FromResult(new ToolResult(output));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
using System.Text.Json;
|
||||
|
||||
namespace backend.Tools;
|
||||
|
||||
public class GetCurrentTimeTool : ITool
|
||||
{
|
||||
public string Name => "get_current_time";
|
||||
public string Description => "Returns the current UTC time as an ISO-8601 string.";
|
||||
public bool RunsDuringResponse => false;
|
||||
|
||||
public JsonElement ParameterSchema { get; } =
|
||||
JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;
|
||||
|
||||
public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
|
||||
{
|
||||
var now = DateTimeOffset.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ");
|
||||
var output = JsonDocument.Parse($$"""{"now":"{{now}}"}""").RootElement;
|
||||
return Task.FromResult(new ToolResult(output));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
using System.Text.Json;
|
||||
|
||||
namespace backend.Tools;
|
||||
|
||||
public interface IDeviceChannel
|
||||
{
|
||||
Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct);
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
using System.Text.Json;
|
||||
|
||||
namespace backend.Tools;
|
||||
|
||||
public record DeviceContext(Guid DeviceId, Guid ConversationId, IDeviceChannel Channel);
|
||||
public record ToolInvocation(string CallId, JsonElement Arguments);
|
||||
public record ToolResult(JsonElement Output);
|
||||
|
||||
public interface ITool
|
||||
{
|
||||
string Name { get; }
|
||||
string Description { get; }
|
||||
JsonElement ParameterSchema { get; }
|
||||
bool RunsDuringResponse { get; }
|
||||
Task<ToolResult> ExecuteAsync(
|
||||
ToolInvocation invocation,
|
||||
DeviceContext device,
|
||||
CancellationToken ct);
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
using System.Text.Json;
|
||||
|
||||
namespace backend.Tools;
|
||||
|
||||
public class SetVolumeTool : ITool
|
||||
{
|
||||
public string Name => "set_volume";
|
||||
public string Description =>
|
||||
"Sets the assistant's audio output volume on the device. Level is 0..100.";
|
||||
public bool RunsDuringResponse => true;
|
||||
|
||||
public JsonElement ParameterSchema { get; } = JsonDocument.Parse(
|
||||
"""
|
||||
{
|
||||
"type":"object",
|
||||
"properties":{"level":{"type":"integer","minimum":0,"maximum":100}},
|
||||
"required":["level"]
|
||||
}
|
||||
""").RootElement;
|
||||
|
||||
public async Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
|
||||
{
|
||||
if (!invocation.Arguments.TryGetProperty("level", out var lvlEl)
|
||||
|| !lvlEl.TryGetInt32(out var level)
|
||||
|| level < 0 || level > 100)
|
||||
{
|
||||
var err = JsonDocument.Parse("""{"ok":false,"error":"level must be an integer in 0..100"}""").RootElement;
|
||||
return new ToolResult(err);
|
||||
}
|
||||
|
||||
var piReply = await device.Channel.CallPiToolAsync(Name, invocation.Arguments, ct);
|
||||
return new ToolResult(piReply);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
namespace backend.Tools;
|
||||
|
||||
public class ToolRegistry
|
||||
{
|
||||
private readonly Dictionary<string, ITool> _byName;
|
||||
private readonly List<ITool> _all;
|
||||
|
||||
public ToolRegistry(IEnumerable<ITool> tools)
|
||||
{
|
||||
_all = tools.ToList();
|
||||
_byName = _all.ToDictionary(t => t.Name);
|
||||
}
|
||||
|
||||
public IReadOnlyList<ITool> All => _all;
|
||||
|
||||
public ITool? Get(string name) => _byName.GetValueOrDefault(name);
|
||||
|
||||
public IEnumerable<ITool> EnabledFor(IReadOnlySet<string> enabled)
|
||||
=> _all.Where(t => enabled.Contains(t.Name));
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
{
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft.AspNetCore": "Warning"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"ConnectionStrings": {
|
||||
"Default": "Data Source=./assistant.db"
|
||||
},
|
||||
"Logging": {
|
||||
"LogLevel": {
|
||||
"Default": "Information",
|
||||
"Microsoft.AspNetCore": "Warning"
|
||||
}
|
||||
},
|
||||
"AllowedHosts": "*"
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk.Web">
|
||||
|
||||
<PropertyGroup>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.AspNetCore.DataProtection" Version="9.*" />
|
||||
<PackageReference Include="Microsoft.AspNetCore.Identity.EntityFrameworkCore" Version="9.*" />
|
||||
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="9.*">
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
<PrivateAssets>all</PrivateAssets>
|
||||
</PackageReference>
|
||||
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="9.*" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/01-record-play-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs && mkdir ~/probe-cs'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs/"
|
||||
|
||||
echo ">> ssh + run on Pi"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs && chmod +x Probe && ./Probe'
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-2-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/02-wakeword-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-2/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-2 && mkdir ~/probe-cs-2'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-2/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-2 && chmod +x Probe && ./Probe'
|
||||
@@ -1,25 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-3-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/03-full-cycle-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-3/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-3 && mkdir ~/probe-cs-3'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-3/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-3 && chmod +x Probe && ./Probe'
|
||||
@@ -1,36 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-4a-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> precondition: ~/.openai_key present on Pi"
|
||||
if ! sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'test -f ~/.openai_key && test -r ~/.openai_key'; then
|
||||
echo "ERROR: ~/.openai_key missing or unreadable on Pi."
|
||||
echo "Create it on the Pi with:"
|
||||
echo " sshpass -p '$PI_PASS' ssh $PI_USER@$PI_HOST \\"
|
||||
echo " 'umask 077 && printf \"%s\\n\" \"sk-PASTE-YOUR-KEY\" > ~/.openai_key'"
|
||||
exit 1
|
||||
fi
|
||||
echo " OK."
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/04a-realtime-oneshot-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-4a/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-4a && mkdir ~/probe-cs-4a'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-4a/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-4a && chmod +x Probe && ./Probe'
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Smart Assistant client entry point. Implemented in Plan 3."""
|
||||
|
||||
|
||||
def main() -> None:
|
||||
raise SystemExit("client.main: placeholder; full implementation comes in Plan 3.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,9 @@
|
||||
"""Smart Assistant pairing CLI. Implemented in Plan 3."""
|
||||
|
||||
|
||||
def main() -> None:
|
||||
raise SystemExit("client.pair: placeholder; full implementation comes in Plan 3.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,20 @@
|
||||
services:
|
||||
app:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
expose:
|
||||
- "8080"
|
||||
environment:
|
||||
- ASPNETCORE_URLS=http://+:8080
|
||||
- 'ConnectionStrings__Default=Data Source=/data/assistant.db'
|
||||
- Install__ClientTarballPath=/app/wwwroot/client.tar.gz
|
||||
- SERVICE_FQDN_APP_8080
|
||||
volumes:
|
||||
- assistant-data:/data
|
||||
- assistant-keys:/keys
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
assistant-data:
|
||||
assistant-keys:
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,570 +0,0 @@
|
||||
# Record-play C# probe — implementation plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Re-implement Test 1 (record 5 s from the USB Speaker Phone → live RMS meter → write `out.wav` → play it back) in C# on the Raspberry Pi, proving .NET-driven PortAudio works on linux-arm64 with the same pass criteria the Python version met.
|
||||
|
||||
**Architecture:** Single-folder throwaway probe under `tests/01-record-play-cs/`. One `Program.cs` driving PortAudio via the PortAudioSharp2 NuGet binding, plus a tiny hand-rolled `WavWriter.cs` for the PCM16 RIFF file. A new `bin/probe-cs` script builds it as a linux-arm64 self-contained binary, rsyncs to the Pi, and runs it over SSH. The Pi gets no .NET install. No automated tests — verification is the same listening test the Python probe uses (`findings.md`).
|
||||
|
||||
**Tech Stack:** .NET 9, C#, PortAudioSharp2 (NuGet), bash, sshpass, rsync. Target host: Pi at `192.168.50.115`, user `pi`, password `assistant` (see `CLAUDE.md`).
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
| Path | Created in task | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `tests/01-record-play-cs/Probe.csproj` | Task 1 | net9.0 console project, linux-arm64 RID, PortAudioSharp2 reference |
|
||||
| `tests/01-record-play-cs/Program.cs` | Task 1 (sanity spike), expanded in Tasks 4 & 5 | Single entry point; mirrors the Python probe's structure |
|
||||
| `tests/01-record-play-cs/WavWriter.cs` | Task 3 | Static `Write(path, samples, sampleRate)` — 44-byte RIFF + raw PCM16 LE |
|
||||
| `bin/probe-cs` | Task 2 | `dotnet publish` → `rsync` → `ssh` flow |
|
||||
|
||||
`tests/01-record-play/` (Python) and the rest of the repo are untouched.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Scaffold project + PortAudio sanity spike on the Pi
|
||||
|
||||
**Goal:** Stand up the .NET project and prove on the Pi that `PortAudio.Initialize()` + device enumeration work end-to-end from a self-contained linux-arm64 publish. No audio I/O yet. This de-risks the unknown the spec called out: does PortAudioSharp2's NuGet ship a `libportaudio` for linux-arm64, or does it dlopen the system one?
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/01-record-play-cs/Probe.csproj`
|
||||
- Create: `tests/01-record-play-cs/Program.cs`
|
||||
|
||||
- [ ] **Step 1: Find the latest stable PortAudioSharp2 version**
|
||||
|
||||
Open `https://www.nuget.org/packages/PortAudioSharp2` in a browser (or `curl -s https://api.nuget.org/v3-flatcontainer/portaudiosharp2/index.json | jq -r '.versions[-1]'`). Note the latest non-prerelease version (call it `<PASHARP_VER>` below). Pin that exact version in the csproj — do **not** use a floating `*`. If `curl/jq` are not available, the NuGet web page lists "Latest version" at the top right.
|
||||
|
||||
- [ ] **Step 2: Write `tests/01-record-play-cs/Probe.csproj`**
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<RootNamespace>RecordPlayProbe</RootNamespace>
|
||||
<AssemblyName>Probe</AssemblyName>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
|
||||
<SelfContained>true</SelfContained>
|
||||
<PublishSingleFile>false</PublishSingleFile>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="PortAudioSharp2" Version="<PASHARP_VER>" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
|
||||
Replace `<PASHARP_VER>` with the version from Step 1.
|
||||
|
||||
- [ ] **Step 3: Write `tests/01-record-play-cs/Program.cs` (spike version)**
|
||||
|
||||
```csharp
|
||||
using PortAudioSharp;
|
||||
|
||||
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
|
||||
|
||||
PortAudio.Initialize();
|
||||
try
|
||||
{
|
||||
int count = PortAudio.DeviceCount;
|
||||
Console.WriteLine($"PortAudio version: {PortAudio.VersionInfo.versionText}");
|
||||
Console.WriteLine($"Devices ({count}):");
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
Console.WriteLine(
|
||||
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels} " +
|
||||
$"defaultSr={info.defaultSampleRate}");
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
PortAudio.Terminate();
|
||||
}
|
||||
```
|
||||
|
||||
**Note for the engineer:** The above uses the API surface as documented in PortAudioSharp2's README at time of writing (`PortAudio.Initialize/Terminate`, `PortAudio.DeviceCount`, `PortAudio.GetDeviceInfo(int)`, `PortAudio.VersionInfo`). If a compile error reveals the surface has changed, read PortAudioSharp2's README at the version you pinned in Step 1 and adjust the calls — the intent is "init, dump device table, terminate". Do **not** change to a different binding.
|
||||
|
||||
- [ ] **Step 4: Build and publish locally**
|
||||
|
||||
Run from the repo root:
|
||||
|
||||
```sh
|
||||
dotnet publish tests/01-record-play-cs -c Release -r linux-arm64 --self-contained \
|
||||
-o /tmp/probe-cs-out
|
||||
```
|
||||
|
||||
Expected: build succeeds. Verify the output contains the native PortAudio binary:
|
||||
|
||||
```sh
|
||||
ls /tmp/probe-cs-out | grep -E 'libportaudio|Probe'
|
||||
```
|
||||
|
||||
Expected: `Probe` (the binary) is present. `libportaudio.so*` may or may not be present — if absent, PortAudioSharp2 will dlopen the system `libportaudio2` already installed by `bin/bootstrap` on the Pi.
|
||||
|
||||
- [ ] **Step 5: Push the spike to the Pi and run it**
|
||||
|
||||
```sh
|
||||
sshpass -p 'assistant' ssh pi@192.168.50.115 'rm -rf ~/probe-cs && mkdir ~/probe-cs'
|
||||
sshpass -p 'assistant' scp -r /tmp/probe-cs-out/. pi@192.168.50.115:~/probe-cs/
|
||||
sshpass -p 'assistant' ssh pi@192.168.50.115 'cd ~/probe-cs && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
(We use `scp -r` instead of `rsync` because the build sandbox does not have rsync installed. The remote `rm -rf` + `mkdir` gives us the `--delete` semantics we want.)
|
||||
|
||||
Expected output: PortAudio version string + a device table that includes at least one entry whose name contains `USB` (the Anhui LISTENAI USB Speaker Phone — exposed as ALSA card 3 per `findings.md`). If you see `dlopen` failures for `libportaudio`, install it on the Pi: `sshpass -p 'assistant' ssh pi@192.168.50.115 'sudo apt-get install -y libportaudio2'` (the prototype `bin/bootstrap` already does this, but the Pi may have been wiped).
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/01-record-play-cs/Probe.csproj tests/01-record-play-cs/Program.cs
|
||||
git commit -m "Scaffold C# record-play probe + PortAudio device-list spike"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Wrap the deploy flow as `bin/probe-cs`
|
||||
|
||||
**Goal:** Capture the Task 1 build/rsync/ssh sequence as a single script so subsequent tasks just run `bin/probe-cs`.
|
||||
|
||||
**Files:**
|
||||
- Create: `bin/probe-cs`
|
||||
|
||||
- [ ] **Step 1: Write `bin/probe-cs`**
|
||||
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/01-record-play-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs && mkdir ~/probe-cs'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs/"
|
||||
|
||||
echo ">> ssh + run on Pi"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
(Using `scp -r` rather than `rsync` because the build sandbox does not have rsync installed. The remote `rm -rf` + `mkdir` gives us the `--delete` semantics we want.)
|
||||
|
||||
- [ ] **Step 2: Mark it executable**
|
||||
|
||||
```sh
|
||||
chmod +x bin/probe-cs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Smoke-test the script against Task 1's spike**
|
||||
|
||||
```sh
|
||||
bin/probe-cs
|
||||
```
|
||||
|
||||
Expected output: same device table as in Task 1 Step 5. If it errors, fix the script — do not work around it in later tasks.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```sh
|
||||
git add bin/probe-cs
|
||||
git commit -m "Add bin/probe-cs: build + rsync + run the C# probe on the Pi"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: `WavWriter.cs` — hand-rolled PCM16 RIFF writer
|
||||
|
||||
**Goal:** A static helper that writes a mono PCM16 WAV file. No external deps.
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/01-record-play-cs/WavWriter.cs`
|
||||
|
||||
- [ ] **Step 1: Write `tests/01-record-play-cs/WavWriter.cs`**
|
||||
|
||||
```csharp
|
||||
namespace RecordPlayProbe;
|
||||
|
||||
internal static class WavWriter
|
||||
{
|
||||
public static void Write(string path, ReadOnlySpan<short> samples, int sampleRate)
|
||||
{
|
||||
const short channels = 1;
|
||||
const short bitsPerSample = 16;
|
||||
const short pcmFormat = 1;
|
||||
|
||||
int byteRate = sampleRate * channels * bitsPerSample / 8;
|
||||
short blockAlign = (short)(channels * bitsPerSample / 8);
|
||||
int dataBytes = samples.Length * sizeof(short);
|
||||
int riffChunkSize = 36 + dataBytes;
|
||||
|
||||
using var fs = new FileStream(path, FileMode.Create, FileAccess.Write);
|
||||
using var bw = new BinaryWriter(fs);
|
||||
|
||||
// RIFF header
|
||||
bw.Write(new[] { (byte)'R', (byte)'I', (byte)'F', (byte)'F' });
|
||||
bw.Write(riffChunkSize);
|
||||
bw.Write(new[] { (byte)'W', (byte)'A', (byte)'V', (byte)'E' });
|
||||
|
||||
// fmt chunk
|
||||
bw.Write(new[] { (byte)'f', (byte)'m', (byte)'t', (byte)' ' });
|
||||
bw.Write(16); // fmt chunk size for PCM
|
||||
bw.Write(pcmFormat);
|
||||
bw.Write(channels);
|
||||
bw.Write(sampleRate);
|
||||
bw.Write(byteRate);
|
||||
bw.Write(blockAlign);
|
||||
bw.Write(bitsPerSample);
|
||||
|
||||
// data chunk
|
||||
bw.Write(new[] { (byte)'d', (byte)'a', (byte)'t', (byte)'a' });
|
||||
bw.Write(dataBytes);
|
||||
foreach (short s in samples)
|
||||
bw.Write(s);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`BinaryWriter` writes integers little-endian on every supported platform — that matches the WAV spec. No byte-order conversion needed.
|
||||
|
||||
- [ ] **Step 2: Build (no test step — spec says no automated tests)**
|
||||
|
||||
```sh
|
||||
dotnet build tests/01-record-play-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/01-record-play-cs/WavWriter.cs
|
||||
git commit -m "Add WavWriter: mono PCM16 RIFF writer for the C# probe"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Recording with live RMS meter + WAV write
|
||||
|
||||
**Goal:** Replace the spike `Program.cs` with the real record + meter + write flow. No playback yet — we want to confirm recording independently of playback so a regression on either is unambiguous.
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/01-record-play-cs/Program.cs` (rewrite — the spike code is replaced)
|
||||
|
||||
- [ ] **Step 1: Rewrite `tests/01-record-play-cs/Program.cs`**
|
||||
|
||||
```csharp
|
||||
using PortAudioSharp;
|
||||
|
||||
const int SampleRate = 16_000;
|
||||
const int Channels = 1;
|
||||
const int DurationS = 5;
|
||||
const uint BlockFrames = 1024;
|
||||
const string OutWav = "out.wav";
|
||||
|
||||
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
|
||||
|
||||
PortAudio.Initialize();
|
||||
try
|
||||
{
|
||||
int device = FindUsbDevice();
|
||||
Console.WriteLine(
|
||||
$"Recording {DurationS}s from device {device} " +
|
||||
$"('{PortAudio.GetDeviceInfo(device).name}')");
|
||||
|
||||
short[] buf = new short[SampleRate * DurationS];
|
||||
int writeOffset = 0;
|
||||
object gate = new();
|
||||
|
||||
var inParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowInputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback callback = (IntPtr input, IntPtr _, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags status, IntPtr _3) =>
|
||||
{
|
||||
if (status.HasFlag(StreamCallbackFlags.InputOverflow))
|
||||
Console.Error.WriteLine("[status] input overflow");
|
||||
|
||||
lock (gate)
|
||||
{
|
||||
int remaining = buf.Length - writeOffset;
|
||||
int take = (int)Math.Min(frameCount, (uint)remaining);
|
||||
if (take > 0)
|
||||
{
|
||||
unsafe
|
||||
{
|
||||
short* src = (short*)input.ToPointer();
|
||||
fixed (short* dst = &buf[writeOffset])
|
||||
{
|
||||
Buffer.MemoryCopy(src, dst, take * sizeof(short), take * sizeof(short));
|
||||
}
|
||||
}
|
||||
writeOffset += take;
|
||||
}
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
using var stream = new Stream(
|
||||
inParams, null, SampleRate, BlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
|
||||
stream.Start();
|
||||
|
||||
var start = DateTime.UtcNow;
|
||||
while (true)
|
||||
{
|
||||
int written;
|
||||
lock (gate) { written = writeOffset; }
|
||||
if (written >= buf.Length) break;
|
||||
|
||||
double level = Rms(buf, Math.Max(0, written - 1600), written);
|
||||
int bars = (int)(level * 60);
|
||||
double elapsed = (DateTime.UtcNow - start).TotalSeconds;
|
||||
Console.Write($"\r{elapsed,4:0.0}s |{new string('#', bars).PadRight(60)}|");
|
||||
Thread.Sleep(50);
|
||||
}
|
||||
Console.WriteLine();
|
||||
stream.Stop();
|
||||
|
||||
WavWriter.Write(OutWav, buf, SampleRate);
|
||||
Console.WriteLine($"Wrote {OutWav} ({buf.Length} frames @ {SampleRate} Hz)");
|
||||
}
|
||||
finally
|
||||
{
|
||||
PortAudio.Terminate();
|
||||
}
|
||||
|
||||
static int FindUsbDevice()
|
||||
{
|
||||
int count = PortAudio.DeviceCount;
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
if (info.name.ToLowerInvariant().Contains("usb") && info.maxInputChannels >= 1)
|
||||
return i;
|
||||
}
|
||||
Console.Error.WriteLine("USB audio device not found. Devices:");
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
Console.Error.WriteLine(
|
||||
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels}");
|
||||
}
|
||||
Environment.Exit(1);
|
||||
return -1; // unreachable
|
||||
}
|
||||
|
||||
static double Rms(short[] data, int from, int to)
|
||||
{
|
||||
if (to <= from) return 0.0;
|
||||
double sumSq = 0.0;
|
||||
for (int i = from; i < to; i++)
|
||||
{
|
||||
double x = data[i] / 32768.0;
|
||||
sumSq += x * x;
|
||||
}
|
||||
return Math.Sqrt(sumSq / (to - from) + 1e-12);
|
||||
}
|
||||
```
|
||||
|
||||
**RMS formula matches the Python probe** (`np.sqrt((x*x).mean() + 1e-12)`). **Tail window matches** (last 1600 samples, ~100 ms at 16 kHz). **Meter cadence matches** (50 ms sleep → ~20 Hz refresh).
|
||||
|
||||
The csproj also needs `<AllowUnsafeBlocks>true</AllowUnsafeBlocks>` for the `unsafe` block in the callback. Add it:
|
||||
|
||||
- [ ] **Step 2: Enable unsafe blocks in `Probe.csproj`**
|
||||
|
||||
Add `<AllowUnsafeBlocks>true</AllowUnsafeBlocks>` inside the existing `<PropertyGroup>`. The full PropertyGroup should now read:
|
||||
|
||||
```xml
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<RootNamespace>RecordPlayProbe</RootNamespace>
|
||||
<AssemblyName>Probe</AssemblyName>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
|
||||
<SelfContained>true</SelfContained>
|
||||
<PublishSingleFile>false</PublishSingleFile>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
</PropertyGroup>
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build locally**
|
||||
|
||||
```sh
|
||||
dotnet build tests/01-record-play-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success. If you hit a `StreamParameters` / `Stream` / `Callback` API mismatch, see PortAudioSharp2's README at the version you pinned in Task 1 and adjust the type names — the structure (struct of device + channelCount + sampleFormat + latency; Stream constructor that takes input params, output params, sample rate, frames-per-buffer, flags, callback, user data) is the conventional PortAudio shape and should map closely.
|
||||
|
||||
- [ ] **Step 4: Run on the Pi**
|
||||
|
||||
```sh
|
||||
bin/probe-cs
|
||||
```
|
||||
|
||||
Expected on the Pi:
|
||||
- A line `Recording 5s from device <N> ('USB...')`.
|
||||
- A live meter: `\r 0.0s |######...|` updating ~20×/sec for 5 seconds. The bar visibly responds to speech vs. silence.
|
||||
- A final line `Wrote out.wav (80000 frames @ 16000 Hz)`.
|
||||
- No sustained `[status] input overflow` messages (one at startup is acceptable).
|
||||
|
||||
Confirm the WAV is well-formed:
|
||||
|
||||
```sh
|
||||
sshpass -p 'assistant' ssh pi@192.168.50.115 'soxi ~/probe-cs/out.wav'
|
||||
```
|
||||
|
||||
Expected: `soxi` reports 1 channel, 16000 Hz, 16-bit PCM, ~5 seconds duration. (If `soxi` isn't installed: `sudo apt-get install -y sox` on the Pi, or use `file out.wav` which will at least say `RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 16000 Hz`.)
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/01-record-play-cs/Probe.csproj tests/01-record-play-cs/Program.cs
|
||||
git commit -m "Record loop: callback InputStream + RMS meter + WAV write"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Playback + full pass-criteria verification
|
||||
|
||||
**Goal:** Add the playback step and verify all three pass criteria from the spec on real hardware.
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/01-record-play-cs/Program.cs` (append playback after the WAV write)
|
||||
|
||||
- [ ] **Step 1: Add playback to `Program.cs`**
|
||||
|
||||
Insert this block immediately after the `Console.WriteLine($"Wrote {OutWav}...");` line and before the `finally` block:
|
||||
|
||||
```csharp
|
||||
Console.WriteLine($"Playing back through device {device}");
|
||||
|
||||
int playOffset = 0;
|
||||
object playGate = new();
|
||||
var doneEvent = new ManualResetEventSlim(false);
|
||||
|
||||
var outParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback playCb = (IntPtr _, IntPtr output, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
|
||||
{
|
||||
lock (playGate)
|
||||
{
|
||||
int remaining = buf.Length - playOffset;
|
||||
int take = (int)Math.Min(frameCount, (uint)remaining);
|
||||
int silence = (int)frameCount - take;
|
||||
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
if (take > 0)
|
||||
{
|
||||
fixed (short* src = &buf[playOffset])
|
||||
Buffer.MemoryCopy(src, dst, take * sizeof(short), take * sizeof(short));
|
||||
playOffset += take;
|
||||
}
|
||||
if (silence > 0)
|
||||
{
|
||||
for (int i = take; i < frameCount; i++) dst[i] = 0;
|
||||
}
|
||||
}
|
||||
|
||||
if (playOffset >= buf.Length)
|
||||
{
|
||||
doneEvent.Set();
|
||||
return StreamCallbackResult.Complete;
|
||||
}
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
using var playStream = new Stream(
|
||||
null, outParams, SampleRate, BlockFrames, StreamFlags.NoFlag, playCb, IntPtr.Zero);
|
||||
playStream.Start();
|
||||
doneEvent.Wait();
|
||||
// Give PortAudio a moment to drain the device buffer before Stop().
|
||||
Thread.Sleep(200);
|
||||
playStream.Stop();
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Build locally**
|
||||
|
||||
```sh
|
||||
dotnet build tests/01-record-play-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success.
|
||||
|
||||
- [ ] **Step 3: Run on the Pi and verify all three pass criteria**
|
||||
|
||||
```sh
|
||||
bin/probe-cs
|
||||
```
|
||||
|
||||
Then **at the Pi's audio output** (the USB Speaker Phone is mic and speaker — playback comes out of the same device):
|
||||
|
||||
1. **Playback is recognisable as what was said into the mic.** Speak something distinct during the 5-second record window (e.g., "the quick brown fox jumps over the lazy dog") and confirm you can clearly understand the playback.
|
||||
2. **The meter visibly responds to speech and to silence.** During the recording, alternate between speaking and silence; the `####` bar should clearly grow and shrink.
|
||||
3. **No clipping, dropouts, or sustained PortAudio under/overrun warnings.** Scan the stderr output for `[status] input overflow` lines — a single one at start-up is acceptable, sustained ones are not. The playback should not have audible clicks, gaps, or chipmunk artefacts.
|
||||
|
||||
If any of the three fails: do not "fix" by tweaking thresholds or silencing logs — diagnose. Use `findings.md` ("Force PortAudio through ALSA's `plug` plugin") as the first reference; the `PA_ALSA_PLUGHW=1` env var must be in place before `PortAudio.Initialize()` runs.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/01-record-play-cs/Program.cs
|
||||
git commit -m "Playback: append OutputStream playback of recorded buffer"
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Note the probe outcome**
|
||||
|
||||
The probe is done. Append a short note to `findings.md` under a new section `## C# probe outcome (2026-06-12)` recording: (a) which PortAudioSharp2 version you pinned, (b) whether the NuGet bundled `libportaudio.so` or you used the Pi's system `libportaudio2`, (c) any API drift you had to adjust for vs. the plan, (d) whether all three pass criteria held. This belongs in `findings.md` because the next probe (Test 2 in C#) will benefit from it.
|
||||
|
||||
Commit:
|
||||
|
||||
```sh
|
||||
git add findings.md
|
||||
git commit -m "Findings: C# record-play probe outcome"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes on what is intentionally **not** in this plan
|
||||
|
||||
- **No automated tests.** The spec is explicit; verification is the listening test on the Pi. WavWriter is verified transitively by the playback step in Task 5.
|
||||
- **No queue / consumer thread.** The Python probe has the same shape (callback writes, main thread reads). The findings doc's "callback queue for input" rework belongs to the eventual main client, not this probe.
|
||||
- **No cleanup on Ctrl-C mid-recording.** Probe. Crash trace is fine.
|
||||
- **No retry on PortAudio init failure.** Crash and surface.
|
||||
- **No abstraction over PortAudio.** No `IAudioDevice` interface, no DI. One file, one job.
|
||||
|
||||
If, during implementation, you feel the urge to add any of the above — stop and re-read the spec. The point of this probe is to be cheap and disposable.
|
||||
@@ -1,969 +0,0 @@
|
||||
# Test 2 wakeword C# probe — implementation plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Re-implement Test 2 (listen for the "alexa" wakeword via openwakeword's stock model → beep on detection) in C# on the Raspberry Pi, by porting `openwakeword/model.py`'s streaming inference pipeline to C# / `Microsoft.ML.OnnxRuntime`. Pass the same hardware criteria as the Python version.
|
||||
|
||||
**Architecture:** Single-folder throwaway probe under `tests/02-wakeword-cs/`. `WakewordModel.cs` chains three vendored ONNX models (mel-spectrogram → Google speech embedding → alexa keyword classifier) with the same streaming buffer geometry as the Python implementation. `Program.cs` drives a PortAudio input stream into a bounded `BlockingCollection`; the main thread pulls frames, runs `WakewordModel.Predict`, and dispatches a fire-and-forget output stream for the detection beep so input ingestion never stalls. `bin/probe-cs-2` builds a self-contained linux-arm64 binary and ships it to the Pi.
|
||||
|
||||
**Tech Stack:** .NET 9, C#, PortAudioSharp2 (NuGet), Microsoft.ML.OnnxRuntime (NuGet), bash, sshpass. Target host: Pi at `192.168.50.115`, user `pi`, password `assistant` (see `CLAUDE.md`). The Pi gets no .NET install — the binary is self-contained.
|
||||
|
||||
---
|
||||
|
||||
## File structure
|
||||
|
||||
| Path | Created in task | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `tests/02-wakeword-cs/Probe.csproj` | Task 1 | net9.0 console project; linux-arm64 RID; PortAudioSharp2 + Microsoft.ML.OnnxRuntime refs; bundles `models/*.onnx` to output. |
|
||||
| `tests/02-wakeword-cs/models/melspectrogram.onnx` | Task 1 | Vendored from the Pi's openwakeword install (~14 KB). |
|
||||
| `tests/02-wakeword-cs/models/embedding_model.onnx` | Task 1 | Vendored from the Pi (~10 MB), Google's speech embedding model. |
|
||||
| `tests/02-wakeword-cs/models/alexa.onnx` | Task 1 | Vendored from the Pi (~1.5 MB), stock alexa classifier. |
|
||||
| `tests/02-wakeword-cs/Program.cs` | Task 1 (spike), rewritten in Task 4 & extended in Task 5 | Entry point: env vars → PortAudio init → device pick → input stream → main inference loop → optional beep. |
|
||||
| `tests/02-wakeword-cs/WakewordModel.cs` | Task 3 | Loads the three sessions; `Predict(short[] frame1280) → float`. |
|
||||
| `bin/probe-cs-2` | Task 2 | `dotnet publish` → `scp` → `ssh -t` flow. |
|
||||
|
||||
`tests/02-wakeword/` (Python), `tests/01-record-play-cs/` (Test 1 C#), and `bin/probe-cs` are untouched.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Scaffold project + vendor ONNX models + ORT sanity spike on the Pi
|
||||
|
||||
**Goal:** Stand up the .NET project with both NuGet refs, vendor the three openwakeword ONNX models into the repo, and prove on the Pi that `Microsoft.ML.OnnxRuntime` loads all three from a self-contained linux-arm64 publish. Capture each model's declared input / output shape (we'll use these as ground-truth assertions in Task 3).
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/02-wakeword-cs/Probe.csproj`
|
||||
- Create: `tests/02-wakeword-cs/models/melspectrogram.onnx` (vendored)
|
||||
- Create: `tests/02-wakeword-cs/models/embedding_model.onnx` (vendored)
|
||||
- Create: `tests/02-wakeword-cs/models/alexa.onnx` (vendored)
|
||||
- Create: `tests/02-wakeword-cs/Program.cs` (spike — replaced in Task 4)
|
||||
|
||||
- [ ] **Step 1: Pin the Microsoft.ML.OnnxRuntime version**
|
||||
|
||||
Use Context7 to fetch the current Microsoft.ML.OnnxRuntime NuGet docs and identify the latest stable version (or check `https://www.nuget.org/packages/Microsoft.ML.OnnxRuntime`):
|
||||
|
||||
```sh
|
||||
curl -s https://api.nuget.org/v3-flatcontainer/microsoft.ml.onnxruntime/index.json | jq -r '.versions[]' | grep -v -E '(rc|preview|alpha|beta)' | tail -1
|
||||
```
|
||||
|
||||
Note the exact version (call it `<ORT_VER>`). Pin this in the csproj — do NOT use a floating `*`.
|
||||
|
||||
Same drill for PortAudioSharp2 (already pinned at 1.0.6 in Test 1; use that exact version unless a newer non-prerelease has shipped):
|
||||
|
||||
```sh
|
||||
curl -s https://api.nuget.org/v3-flatcontainer/portaudiosharp2/index.json | jq -r '.versions[-1]'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Vendor the three ONNX files from the Pi**
|
||||
|
||||
```sh
|
||||
mkdir -p tests/02-wakeword-cs/models
|
||||
sshpass -p assistant scp \
|
||||
pi@192.168.50.115:assistant/.venv/lib/python3.13/site-packages/openwakeword/resources/models/melspectrogram.onnx \
|
||||
pi@192.168.50.115:assistant/.venv/lib/python3.13/site-packages/openwakeword/resources/models/embedding_model.onnx \
|
||||
pi@192.168.50.115:assistant/.venv/lib/python3.13/site-packages/openwakeword/resources/models/alexa.onnx \
|
||||
tests/02-wakeword-cs/models/
|
||||
ls -l tests/02-wakeword-cs/models/
|
||||
sha256sum tests/02-wakeword-cs/models/*.onnx
|
||||
```
|
||||
|
||||
Expected: three files, roughly 14 KB / 10 MB / 1.5 MB. Record the SHA-256 hashes in your commit message in Step 8 — that pins the exact model files used.
|
||||
|
||||
If `scp` fails because the Pi's openwakeword venv lives elsewhere, run:
|
||||
|
||||
```sh
|
||||
sshpass -p assistant ssh pi@192.168.50.115 'find ~ -path "*openwakeword/resources/models/*.onnx" 2>/dev/null'
|
||||
```
|
||||
|
||||
and substitute the correct directory.
|
||||
|
||||
- [ ] **Step 3: Write `tests/02-wakeword-cs/Probe.csproj`**
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<RootNamespace>WakewordProbe</RootNamespace>
|
||||
<AssemblyName>Probe</AssemblyName>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
|
||||
<SelfContained>true</SelfContained>
|
||||
<PublishSingleFile>false</PublishSingleFile>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="PortAudioSharp2" Version="1.0.6" />
|
||||
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="<ORT_VER>" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Content Include="models/*.onnx">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</Content>
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
|
||||
Replace `<ORT_VER>` with the version from Step 1.
|
||||
|
||||
- [ ] **Step 4: Write the spike `tests/02-wakeword-cs/Program.cs`**
|
||||
|
||||
This temporary Program.cs loads each model and prints its declared shapes — exactly what Task 3 needs to assert against. It is replaced in Task 4.
|
||||
|
||||
```csharp
|
||||
using Microsoft.ML.OnnxRuntime;
|
||||
|
||||
string modelsDir = Path.Combine(AppContext.BaseDirectory, "models");
|
||||
string[] modelFiles = { "melspectrogram.onnx", "embedding_model.onnx", "alexa.onnx" };
|
||||
|
||||
var opts = new SessionOptions
|
||||
{
|
||||
IntraOpNumThreads = 1,
|
||||
InterOpNumThreads = 1,
|
||||
LogSeverityLevel = OrtLoggingLevel.ORT_LOGGING_LEVEL_ERROR,
|
||||
};
|
||||
|
||||
foreach (var f in modelFiles)
|
||||
{
|
||||
string path = Path.Combine(modelsDir, f);
|
||||
Console.WriteLine($"== {f} ==");
|
||||
using var session = new InferenceSession(path, opts);
|
||||
foreach (var kv in session.InputMetadata)
|
||||
{
|
||||
var dims = string.Join(",", kv.Value.Dimensions);
|
||||
Console.WriteLine($" input '{kv.Key}': shape=[{dims}] dtype={kv.Value.ElementType.Name}");
|
||||
}
|
||||
foreach (var kv in session.OutputMetadata)
|
||||
{
|
||||
var dims = string.Join(",", kv.Value.Dimensions);
|
||||
Console.WriteLine($" output '{kv.Key}': shape=[{dims}] dtype={kv.Value.ElementType.Name}");
|
||||
}
|
||||
}
|
||||
Console.WriteLine("All three models loaded.");
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Build and publish locally**
|
||||
|
||||
```sh
|
||||
dotnet publish tests/02-wakeword-cs -c Release -r linux-arm64 --self-contained \
|
||||
-o /tmp/probe-cs-2-out
|
||||
```
|
||||
|
||||
Expected: build succeeds. Confirm the publish dir contains the binary, the three model files, and the bundled ONNX Runtime native:
|
||||
|
||||
```sh
|
||||
ls /tmp/probe-cs-2-out | grep -E 'libonnxruntime|libportaudio|Probe'
|
||||
ls /tmp/probe-cs-2-out/models/
|
||||
```
|
||||
|
||||
Expected: `Probe`, `libonnxruntime.so` (or `.so.<version>`), `libportaudio.so` (or similar), plus the three `.onnx` files under `models/`. If `libonnxruntime.so` is missing, the `Microsoft.ML.OnnxRuntime` package for your version may have split the native runtime — install the matching native package (`Microsoft.ML.OnnxRuntime.Native.<rid>` if it exists for `<ORT_VER>`) or downgrade `<ORT_VER>` to a version where the linux-arm64 native is included in the main package. Do NOT proceed to Step 6 if the native is missing.
|
||||
|
||||
- [ ] **Step 6: Push the spike to the Pi and run it**
|
||||
|
||||
```sh
|
||||
sshpass -p assistant ssh pi@192.168.50.115 'rm -rf ~/probe-cs-2 && mkdir ~/probe-cs-2'
|
||||
sshpass -p assistant scp -r /tmp/probe-cs-2-out/. pi@192.168.50.115:~/probe-cs-2/
|
||||
sshpass -p assistant ssh pi@192.168.50.115 'cd ~/probe-cs-2 && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
Expected output (numbers and names matching the openwakeword Python source from `/tmp/oww-utils.py`):
|
||||
|
||||
```
|
||||
== melspectrogram.onnx ==
|
||||
input '<name>': shape=[-1,-1] dtype=Single (1D audio, variable length)
|
||||
output '<name>': shape=[1,-1,1,32] dtype=Single (mel: 32 bins, variable n_frames)
|
||||
== embedding_model.onnx ==
|
||||
input 'input_1': shape=[-1,76,32,1] dtype=Single (76 mel frames × 32 bins × 1 channel, variable batch)
|
||||
output '<name>': shape=[-1,1,1,96] dtype=Single (96-d embedding per batch)
|
||||
== alexa.onnx ==
|
||||
input '<name>': shape=[-1,16,96] dtype=Single (16 embeddings × 96 dims, variable batch)
|
||||
output '<name>': shape=[-1,1] dtype=Single (binary classifier score)
|
||||
All three models loaded.
|
||||
```
|
||||
|
||||
Negative dims (`-1`) mean "dynamic / batch dimension" — that's fine, we'll pass `1` at inference time.
|
||||
|
||||
**Save the actual printed output to your scratchpad** — the input/output names (`'input_1'` etc.) and exact dimension lists become the assertion values in Task 3 Step 4. The dimension numbers (76, 32, 16, 96) should match the Python source; if they don't, stop and investigate before proceeding.
|
||||
|
||||
- [ ] **Step 7: Mark `bin/probe-cs-2` placeholder NOT created yet**
|
||||
|
||||
Task 2 creates the deploy script. For Task 1 we use the inline scp commands above. (Task 1 cannot depend on Task 2.)
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/02-wakeword-cs/Probe.csproj tests/02-wakeword-cs/Program.cs tests/02-wakeword-cs/models/
|
||||
git commit -m "Scaffold C# wakeword probe: vendored ONNX models + ORT sanity spike
|
||||
|
||||
Models sourced from openwakeword==0.6.0 on the Pi.
|
||||
SHA-256:
|
||||
melspectrogram.onnx <hash>
|
||||
embedding_model.onnx <hash>
|
||||
alexa.onnx <hash>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Wrap the deploy flow as `bin/probe-cs-2`
|
||||
|
||||
**Goal:** Capture the build / scp / ssh sequence as a single script so subsequent tasks just run `bin/probe-cs-2`. Mirrors `bin/probe-cs` with new paths and adds `ssh -t` so workstation Ctrl-C propagates as SIGINT to the long-running probe.
|
||||
|
||||
**Files:**
|
||||
- Create: `bin/probe-cs-2`
|
||||
|
||||
- [ ] **Step 1: Write `bin/probe-cs-2`**
|
||||
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-2-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/02-wakeword-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-2/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-2 && mkdir ~/probe-cs-2'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-2/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-2 && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
The `ssh -t` (force pseudo-tty) makes Ctrl-C from the workstation propagate as SIGINT to the remote `./Probe` process — needed because Task 4's main loop runs until Ctrl-C. Test 1's `bin/probe-cs` didn't need it because the probe self-terminated after 5 s of recording + playback.
|
||||
|
||||
- [ ] **Step 2: Mark it executable**
|
||||
|
||||
```sh
|
||||
chmod +x bin/probe-cs-2
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Smoke-test against Task 1's spike**
|
||||
|
||||
```sh
|
||||
bin/probe-cs-2
|
||||
```
|
||||
|
||||
Expected: same shape-dump output as Task 1 Step 6, ending with `All three models loaded.` Ctrl-C is not needed because the spike exits on its own. If it errors, fix the script — do not work around in later tasks.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```sh
|
||||
git add bin/probe-cs-2
|
||||
git commit -m "Add bin/probe-cs-2: build + scp + run the C# wakeword probe on the Pi"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: `WakewordModel.cs` — port openwakeword's streaming pipeline
|
||||
|
||||
**Goal:** A `WakewordModel` class that wraps the three ONNX sessions and exposes a single `Predict(short[] frame1280) → float` method. Internal state matches `openwakeword/AudioFeatures._streaming_features()` and `Model.predict()` from the openwakeword 0.6.0 source on the Pi (already fetched to `/tmp/model.py` and `/tmp/oww-utils.py` during plan-writing — re-fetch if needed: `sshpass -p assistant scp pi@192.168.50.115:assistant/.venv/lib/python3.13/site-packages/openwakeword/{model,utils}.py /tmp/`).
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/02-wakeword-cs/WakewordModel.cs`
|
||||
- Modify: `tests/02-wakeword-cs/Program.cs` (replace spike with a load-and-print test)
|
||||
|
||||
### Pipeline geometry (derived from openwakeword/utils.py `AudioFeatures`)
|
||||
|
||||
Per call to `Predict(short[] frame1280)`:
|
||||
|
||||
1. **Raw-audio buffer.** Append the 1280 new samples to a ring of recent samples. Keep at least the last `1280 + 480 = 1760` samples (the +480 = `160 * 3` is the mel model's 30 ms warm-up context that openwakeword passes in via the `-n_samples-160*3:` slice in `_streaming_melspectrogram`).
|
||||
2. **Mel stage.** Slice the last 1760 samples → run `melspectrogram.onnx` with input shape `(1, 1760)` dtype Single → output is `(1, n_frames, 1, 32)`; squeeze trailing dims to get `(n_frames, 32)` where `n_frames` ≈ 8 (`ceil(1760/160 - 3) = ceil(8) = 8`). Apply the openwakeword post-transform `x = x / 10 + 2` element-wise. Append the new mel frames to the mel-frame ring. Trim ring to the last 970 frames (= `10 * 97`, openwakeword's `melspectrogram_max_len`).
|
||||
3. **Embedding stage.** Slice the last 76 mel frames from the ring → shape `(1, 76, 32, 1)`. Run `embedding_model.onnx` (input name `input_1`) → output shape `(1, 1, 1, 96)`; squeeze to a `float[96]` embedding. Append to the embedding ring. Trim ring to the last 120 entries (openwakeword's `feature_buffer_max_len`).
|
||||
4. **Classifier stage.** Slice the last 16 embeddings from the ring → shape `(1, 16, 96)`. Run `alexa.onnx` → output shape `(1, 1)`; return `output[0, 0]` as the score.
|
||||
5. **Initialization gate.** Return `0f` for the first 16 calls — until the embedding ring has 16 real entries, classifier output is meaningless. (openwakeword pre-populates with random-audio embeddings then zeros the first 5 scores; we do the simpler equivalent: skip outright until we have 16 real embeddings.)
|
||||
|
||||
- [ ] **Step 1: Document the geometry in `WakewordModel.cs`**
|
||||
|
||||
Create `tests/02-wakeword-cs/WakewordModel.cs` and start with constants and the class skeleton:
|
||||
|
||||
```csharp
|
||||
using Microsoft.ML.OnnxRuntime;
|
||||
using Microsoft.ML.OnnxRuntime.Tensors;
|
||||
|
||||
namespace WakewordProbe;
|
||||
|
||||
// Streaming wakeword inference port of openwakeword 0.6.0's predict pipeline.
|
||||
// References (read alongside this file):
|
||||
// openwakeword/utils.py:AudioFeatures._streaming_features (mel + embedding stages)
|
||||
// openwakeword/utils.py:AudioFeatures._streaming_melspectrogram
|
||||
// openwakeword/utils.py:AudioFeatures._get_embeddings
|
||||
// openwakeword/model.py:Model.predict (classifier stage)
|
||||
internal sealed class WakewordModel : IDisposable
|
||||
{
|
||||
// Audio I/O geometry — per-call input contract.
|
||||
public const int FrameSamples = 1280; // 80 ms @ 16 kHz; each Predict() call.
|
||||
public const int SampleRate = 16_000;
|
||||
|
||||
// Mel model: input is last (FrameSamples + MelContextSamples) raw samples
|
||||
// -- the +480 is openwakeword's `-n_samples-160*3:` slice (3 hops of context).
|
||||
public const int MelContextSamples = 480; // 160 * 3
|
||||
public const int MelInputSamples = FrameSamples + MelContextSamples; // 1760
|
||||
public const int MelBins = 32; // melspectrogram model output dim
|
||||
public const int MelBufferMaxFrames = 970; // 10 * 97 (openwakeword `melspectrogram_max_len`)
|
||||
|
||||
// Embedding model: 76-mel-frame window in, 96-d embedding out.
|
||||
public const int EmbeddingWindowMelFrames = 76;
|
||||
public const int EmbeddingDim = 96;
|
||||
public const int EmbeddingBufferMax = 120; // openwakeword `feature_buffer_max_len`
|
||||
public const string EmbeddingInputName = "input_1"; // openwakeword convention; assert at startup
|
||||
|
||||
// Classifier: 16 embeddings in, scalar score out.
|
||||
public const int ClassifierEmbeddings = 16;
|
||||
|
||||
// Skip the first N Predict() calls — buffer fill-up window.
|
||||
public const int WarmupFrames = ClassifierEmbeddings; // 16 frames ≈ 1.28 s
|
||||
|
||||
private readonly InferenceSession _mel;
|
||||
private readonly InferenceSession _emb;
|
||||
private readonly InferenceSession _cls;
|
||||
|
||||
private readonly string _melInputName; // discovered at startup
|
||||
private readonly string _clsInputName; // discovered at startup
|
||||
|
||||
private readonly short[] _rawRing = new short[MelInputSamples];
|
||||
private int _rawRingFill = 0; // samples buffered (≤ MelInputSamples)
|
||||
|
||||
private readonly List<float[]> _melRing = new(MelBufferMaxFrames); // each entry is a length-32 mel frame
|
||||
private readonly List<float[]> _embRing = new(EmbeddingBufferMax); // each entry is a length-96 embedding
|
||||
|
||||
private int _framesSeen = 0;
|
||||
|
||||
public WakewordModel(string melPath, string embeddingPath, string classifierPath)
|
||||
{
|
||||
var opts = new SessionOptions
|
||||
{
|
||||
IntraOpNumThreads = 1,
|
||||
InterOpNumThreads = 1,
|
||||
LogSeverityLevel = OrtLoggingLevel.ORT_LOGGING_LEVEL_ERROR,
|
||||
};
|
||||
|
||||
_mel = new InferenceSession(melPath, opts);
|
||||
_emb = new InferenceSession(embeddingPath, opts);
|
||||
_cls = new InferenceSession(classifierPath, opts);
|
||||
|
||||
// Discover input names + assert shape geometry.
|
||||
_melInputName = _mel.InputMetadata.Keys.Single();
|
||||
|
||||
AssertEmbeddingShape(_emb); // input_1: [batch, 76, 32, 1], dtype float
|
||||
_clsInputName = _cls.InputMetadata.Keys.Single();
|
||||
AssertClassifierShape(_cls); // [batch, 16, 96], dtype float
|
||||
}
|
||||
|
||||
// ... methods follow in Step 2 ...
|
||||
|
||||
public void Dispose()
|
||||
{
|
||||
_mel.Dispose();
|
||||
_emb.Dispose();
|
||||
_cls.Dispose();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Implement the shape-assertion helpers**
|
||||
|
||||
Append inside the `WakewordModel` class:
|
||||
|
||||
```csharp
|
||||
private static void AssertEmbeddingShape(InferenceSession sess)
|
||||
{
|
||||
if (!sess.InputMetadata.TryGetValue(EmbeddingInputName, out var meta))
|
||||
throw new InvalidOperationException(
|
||||
$"embedding_model.onnx: expected input named '{EmbeddingInputName}', got [{string.Join(",", sess.InputMetadata.Keys)}]");
|
||||
var d = meta.Dimensions;
|
||||
// Expected: [batch, 76, 32, 1] — batch may be -1 (dynamic).
|
||||
if (d.Length != 4 || d[1] != EmbeddingWindowMelFrames || d[2] != MelBins || d[3] != 1)
|
||||
throw new InvalidOperationException(
|
||||
$"embedding_model.onnx: expected input shape [batch,{EmbeddingWindowMelFrames},{MelBins},1], got [{string.Join(",", d)}]");
|
||||
if (meta.ElementType != typeof(float))
|
||||
throw new InvalidOperationException($"embedding_model.onnx: expected Single input, got {meta.ElementType.Name}");
|
||||
}
|
||||
|
||||
private static void AssertClassifierShape(InferenceSession sess)
|
||||
{
|
||||
var inputName = sess.InputMetadata.Keys.Single();
|
||||
var meta = sess.InputMetadata[inputName];
|
||||
var d = meta.Dimensions;
|
||||
// Expected: [batch, 16, 96] — batch may be -1.
|
||||
if (d.Length != 3 || d[1] != ClassifierEmbeddings || d[2] != EmbeddingDim)
|
||||
throw new InvalidOperationException(
|
||||
$"alexa.onnx: expected input shape [batch,{ClassifierEmbeddings},{EmbeddingDim}], got [{string.Join(",", d)}]");
|
||||
if (meta.ElementType != typeof(float))
|
||||
throw new InvalidOperationException($"alexa.onnx: expected Single input, got {meta.ElementType.Name}");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Implement `Predict`**
|
||||
|
||||
Append inside the `WakewordModel` class:
|
||||
|
||||
```csharp
|
||||
public float Predict(short[] frame1280)
|
||||
{
|
||||
if (frame1280.Length != FrameSamples)
|
||||
throw new ArgumentException($"Expected {FrameSamples} samples, got {frame1280.Length}");
|
||||
|
||||
// 1. Append 1280 new samples to the raw ring (shift older samples down if full).
|
||||
if (_rawRingFill < MelInputSamples)
|
||||
{
|
||||
int copyToFront = Math.Min(MelInputSamples - _rawRingFill, FrameSamples);
|
||||
Array.Copy(frame1280, 0, _rawRing, _rawRingFill, copyToFront);
|
||||
_rawRingFill += copyToFront;
|
||||
if (copyToFront < FrameSamples)
|
||||
{
|
||||
// Shouldn't happen on the very first call, but defensive.
|
||||
int leftover = FrameSamples - copyToFront;
|
||||
Array.Copy(_rawRing, leftover, _rawRing, 0, MelInputSamples - leftover);
|
||||
Array.Copy(frame1280, copyToFront, _rawRing, MelInputSamples - leftover, leftover);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
// Shift older samples left by FrameSamples, then append new at the tail.
|
||||
Array.Copy(_rawRing, FrameSamples, _rawRing, 0, MelInputSamples - FrameSamples);
|
||||
Array.Copy(frame1280, 0, _rawRing, MelInputSamples - FrameSamples, FrameSamples);
|
||||
}
|
||||
|
||||
// Skip everything until we have the full mel-context window primed.
|
||||
if (_rawRingFill < MelInputSamples)
|
||||
{
|
||||
_framesSeen++;
|
||||
return 0f;
|
||||
}
|
||||
|
||||
// 2. Mel stage: feed the full _rawRing as float32 (1, MelInputSamples) into mel model.
|
||||
var melInputData = new float[MelInputSamples];
|
||||
for (int i = 0; i < MelInputSamples; i++) melInputData[i] = _rawRing[i]; // int16 → float32, NO normalisation
|
||||
var melInputTensor = new DenseTensor<float>(melInputData, new[] { 1, MelInputSamples });
|
||||
using var melResults = _mel.Run(new[] {
|
||||
NamedOnnxValue.CreateFromTensor(_melInputName, melInputTensor)
|
||||
});
|
||||
var melOut = melResults.First().AsTensor<float>(); // shape (1, n_frames, 1, 32)
|
||||
|
||||
// Apply openwakeword's `x / 10 + 2` transform and append each new frame to _melRing.
|
||||
int nFrames = melOut.Dimensions[1];
|
||||
for (int f = 0; f < nFrames; f++)
|
||||
{
|
||||
var bin = new float[MelBins];
|
||||
for (int b = 0; b < MelBins; b++)
|
||||
bin[b] = melOut[0, f, 0, b] / 10f + 2f;
|
||||
_melRing.Add(bin);
|
||||
}
|
||||
if (_melRing.Count > MelBufferMaxFrames)
|
||||
_melRing.RemoveRange(0, _melRing.Count - MelBufferMaxFrames);
|
||||
|
||||
// 3. Embedding stage: need ≥ 76 mel frames; slice the last 76 → (1, 76, 32, 1).
|
||||
if (_melRing.Count < EmbeddingWindowMelFrames)
|
||||
{
|
||||
_framesSeen++;
|
||||
return 0f;
|
||||
}
|
||||
var embInputData = new float[EmbeddingWindowMelFrames * MelBins];
|
||||
int startMel = _melRing.Count - EmbeddingWindowMelFrames;
|
||||
for (int f = 0; f < EmbeddingWindowMelFrames; f++)
|
||||
Array.Copy(_melRing[startMel + f], 0, embInputData, f * MelBins, MelBins);
|
||||
var embInputTensor = new DenseTensor<float>(embInputData, new[] { 1, EmbeddingWindowMelFrames, MelBins, 1 });
|
||||
using var embResults = _emb.Run(new[] {
|
||||
NamedOnnxValue.CreateFromTensor(EmbeddingInputName, embInputTensor)
|
||||
});
|
||||
var embOut = embResults.First().AsTensor<float>(); // shape (1, 1, 1, 96)
|
||||
var newEmb = new float[EmbeddingDim];
|
||||
for (int i = 0; i < EmbeddingDim; i++) newEmb[i] = embOut[0, 0, 0, i];
|
||||
_embRing.Add(newEmb);
|
||||
if (_embRing.Count > EmbeddingBufferMax)
|
||||
_embRing.RemoveRange(0, _embRing.Count - EmbeddingBufferMax);
|
||||
|
||||
_framesSeen++;
|
||||
|
||||
// 4. Warm-up + classifier stage.
|
||||
if (_embRing.Count < ClassifierEmbeddings || _framesSeen <= WarmupFrames)
|
||||
return 0f;
|
||||
|
||||
var clsInputData = new float[ClassifierEmbeddings * EmbeddingDim];
|
||||
int startEmb = _embRing.Count - ClassifierEmbeddings;
|
||||
for (int i = 0; i < ClassifierEmbeddings; i++)
|
||||
Array.Copy(_embRing[startEmb + i], 0, clsInputData, i * EmbeddingDim, EmbeddingDim);
|
||||
var clsInputTensor = new DenseTensor<float>(clsInputData, new[] { 1, ClassifierEmbeddings, EmbeddingDim });
|
||||
using var clsResults = _cls.Run(new[] {
|
||||
NamedOnnxValue.CreateFromTensor(_clsInputName, clsInputTensor)
|
||||
});
|
||||
var clsOut = clsResults.First().AsTensor<float>(); // shape (1, 1)
|
||||
return clsOut[0, 0];
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Replace `Program.cs` spike with a load-only smoke test**
|
||||
|
||||
Replace `tests/02-wakeword-cs/Program.cs` with:
|
||||
|
||||
```csharp
|
||||
using WakewordProbe;
|
||||
|
||||
string modelsDir = Path.Combine(AppContext.BaseDirectory, "models");
|
||||
Console.WriteLine("Loading WakewordModel...");
|
||||
var t0 = DateTime.UtcNow;
|
||||
using var model = new WakewordModel(
|
||||
Path.Combine(modelsDir, "melspectrogram.onnx"),
|
||||
Path.Combine(modelsDir, "embedding_model.onnx"),
|
||||
Path.Combine(modelsDir, "alexa.onnx"));
|
||||
Console.WriteLine($"Loaded in {(DateTime.UtcNow - t0).TotalSeconds:0.00}s.");
|
||||
|
||||
// Smoke test: feed 50 frames of silence (1280 zero samples each).
|
||||
// Expect every score to be 0f (warmup gate + no signal).
|
||||
var silentFrame = new short[WakewordModel.FrameSamples];
|
||||
int nonZero = 0;
|
||||
for (int i = 0; i < 50; i++)
|
||||
{
|
||||
float s = model.Predict(silentFrame);
|
||||
if (s != 0f) nonZero++;
|
||||
}
|
||||
Console.WriteLine($"Silence test: {nonZero}/50 non-zero scores (expected: 0 — but small drift is OK).");
|
||||
Console.WriteLine("Predict pipeline ran without throwing.");
|
||||
```
|
||||
|
||||
The "non-zero on silence" count may be small but not literally 0 — the classifier sees ~zero input but its output is rarely exactly 0f. Anything under ~5 (and well below threshold 0.5) is fine. The point of the smoke test is: did `Predict` run end-to-end without throwing on shape mismatches? If it throws, the assertion bug is in the model load or pipeline geometry.
|
||||
|
||||
- [ ] **Step 5: Build locally**
|
||||
|
||||
```sh
|
||||
dotnet build tests/02-wakeword-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success. Common compile errors:
|
||||
|
||||
- `NamedOnnxValue` / `DenseTensor` / `InferenceSession` not found → confirm `using Microsoft.ML.OnnxRuntime;` and `using Microsoft.ML.OnnxRuntime.Tensors;` are at top.
|
||||
- `'InputMetadata' has no Single()` → add `using System.Linq;` (with `ImplicitUsings` enabled this should already be there, but the spike didn't need it).
|
||||
- API surface drift in `Microsoft.ML.OnnxRuntime` `<ORT_VER>` (the package occasionally renames `NamedOnnxValue.CreateFromTensor`) → consult Context7 `/microsoft/onnxruntime` docs for the pinned version. Do NOT silently switch to a different API; document the version-specific shape if it diverges.
|
||||
|
||||
- [ ] **Step 6: Run on the Pi via `bin/probe-cs-2`**
|
||||
|
||||
```sh
|
||||
bin/probe-cs-2
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```
|
||||
Loading WakewordModel...
|
||||
Loaded in 0.XXs.
|
||||
Silence test: <small>/50 non-zero scores (expected: 0 — but small drift is OK).
|
||||
Predict pipeline ran without throwing.
|
||||
```
|
||||
|
||||
If the run aborts with an `InvalidOperationException` from one of the shape assertions, the model in the repo disagrees with what this plan expects — STOP. Either the vendored ONNX file is from a newer openwakeword version with a different shape (re-verify the SHA-256 hashes against Step 8 of Task 1), or our shape derivation from `oww-utils.py` was wrong. Investigate by re-running the Task 1 spike and comparing dimensions before adjusting `WakewordModel.cs`.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/02-wakeword-cs/WakewordModel.cs tests/02-wakeword-cs/Program.cs
|
||||
git commit -m "WakewordModel: port openwakeword streaming pipeline (mel→emb→cls)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: `Program.cs` — live input stream + main inference loop (detection-only, no beep)
|
||||
|
||||
**Goal:** Replace the load-only smoke test with the full input stream + inference loop. Detection lines print to stdout but no beep yet — we want detection working in isolation before adding the second PortAudio stream.
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/02-wakeword-cs/Program.cs` (full rewrite)
|
||||
|
||||
- [ ] **Step 1: Rewrite `tests/02-wakeword-cs/Program.cs`**
|
||||
|
||||
```csharp
|
||||
using System.Collections.Concurrent;
|
||||
using System.Diagnostics;
|
||||
using System.Runtime.InteropServices;
|
||||
using PortAudioSharp;
|
||||
using WakewordProbe;
|
||||
using Stream = PortAudioSharp.Stream;
|
||||
|
||||
const int SampleRate = WakewordModel.SampleRate;
|
||||
const int Channels = 1;
|
||||
const uint BlockFrames = WakewordModel.FrameSamples; // 1280
|
||||
const float Threshold = 0.5f;
|
||||
const double CooldownSeconds = 1.0;
|
||||
|
||||
// .NET-side mirror for any readers that go through Environment.GetEnvironmentVariable,
|
||||
// AND libc setenv so PortAudio / onnxruntime (which use getenv()) see the values.
|
||||
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
|
||||
Libc.setenv("PA_ALSA_PLUGHW", "1", 1);
|
||||
Environment.SetEnvironmentVariable("ORT_LOGGING_LEVEL", "3");
|
||||
Libc.setenv("ORT_LOGGING_LEVEL", "3", 1);
|
||||
|
||||
PortAudio.Initialize();
|
||||
WakewordModel? model = null;
|
||||
try
|
||||
{
|
||||
int device = FindUsbDevice();
|
||||
Console.WriteLine($"Using device {device} ('{PortAudio.GetDeviceInfo(device).name}')");
|
||||
|
||||
Console.WriteLine("Loading WakewordModel...");
|
||||
var t0 = DateTime.UtcNow;
|
||||
string modelsDir = Path.Combine(AppContext.BaseDirectory, "models");
|
||||
model = new WakewordModel(
|
||||
Path.Combine(modelsDir, "melspectrogram.onnx"),
|
||||
Path.Combine(modelsDir, "embedding_model.onnx"),
|
||||
Path.Combine(modelsDir, "alexa.onnx"));
|
||||
Console.WriteLine($"Loaded in {(DateTime.UtcNow - t0).TotalSeconds:0.00}s.");
|
||||
|
||||
var queue = new BlockingCollection<short[]>(boundedCapacity: 16);
|
||||
using var cts = new CancellationTokenSource();
|
||||
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
|
||||
|
||||
var inParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowInputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback callback = (IntPtr input, IntPtr _, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags status, IntPtr _3) =>
|
||||
{
|
||||
if (status.HasFlag(StreamCallbackFlags.InputOverflow))
|
||||
Console.Error.WriteLine("[status] input overflow");
|
||||
|
||||
if (frameCount != BlockFrames)
|
||||
{
|
||||
Console.Error.WriteLine($"[status] unexpected callback frameCount={frameCount} (want {BlockFrames})");
|
||||
return StreamCallbackResult.Continue;
|
||||
}
|
||||
|
||||
var frame = new short[BlockFrames];
|
||||
unsafe
|
||||
{
|
||||
short* src = (short*)input.ToPointer();
|
||||
fixed (short* dst = &frame[0])
|
||||
Buffer.MemoryCopy(src, dst, BlockFrames * sizeof(short), BlockFrames * sizeof(short));
|
||||
}
|
||||
if (!queue.TryAdd(frame, 0))
|
||||
Console.Error.WriteLine("[status] consumer behind, dropping frame");
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
using var stream = new Stream(
|
||||
inParams, null, SampleRate, BlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
|
||||
stream.Start();
|
||||
Console.WriteLine("Listening. Say 'alexa'. Ctrl-C to exit.");
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
TimeSpan lastTrigger = TimeSpan.FromSeconds(-CooldownSeconds);
|
||||
|
||||
try
|
||||
{
|
||||
while (!cts.IsCancellationRequested)
|
||||
{
|
||||
short[] frame = queue.Take(cts.Token);
|
||||
float score = model.Predict(frame);
|
||||
var now = sw.Elapsed;
|
||||
if (score >= Threshold && (now - lastTrigger).TotalSeconds >= CooldownSeconds)
|
||||
{
|
||||
Console.WriteLine($"DETECTED alexa score={score:0.000} t={now.TotalSeconds:0.0}s");
|
||||
lastTrigger = now;
|
||||
// Beep is added in Task 5.
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { /* Ctrl-C path */ }
|
||||
|
||||
stream.Stop();
|
||||
Console.WriteLine("\nbye");
|
||||
}
|
||||
finally
|
||||
{
|
||||
model?.Dispose();
|
||||
PortAudio.Terminate();
|
||||
}
|
||||
|
||||
static int FindUsbDevice()
|
||||
{
|
||||
int count = PortAudio.DeviceCount;
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
if (info.name.ToLowerInvariant().Contains("usb") && info.maxInputChannels >= 1)
|
||||
return i;
|
||||
}
|
||||
Console.Error.WriteLine("USB audio device not found. Devices:");
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
Console.Error.WriteLine(
|
||||
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels}");
|
||||
}
|
||||
Environment.Exit(1);
|
||||
return -1; // unreachable
|
||||
}
|
||||
|
||||
static class Libc
|
||||
{
|
||||
[DllImport("libc", EntryPoint = "setenv")]
|
||||
public static extern int setenv(string name, string value, int overwrite);
|
||||
}
|
||||
```
|
||||
|
||||
Key reuses from Test 1's `Program.cs` (`tests/01-record-play-cs/Program.cs`): the `using Stream = PortAudioSharp.Stream;` disambiguation, the `Libc.setenv` P/Invoke, the `FindUsbDevice` body, the `unsafe` callback `Buffer.MemoryCopy` pattern, the dual `Environment.SetEnvironmentVariable` + `Libc.setenv` calls.
|
||||
|
||||
- [ ] **Step 2: Build locally**
|
||||
|
||||
```sh
|
||||
dotnet build tests/02-wakeword-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success.
|
||||
|
||||
- [ ] **Step 3: Deploy + run on the Pi**
|
||||
|
||||
```sh
|
||||
bin/probe-cs-2
|
||||
```
|
||||
|
||||
Expected on the Pi:
|
||||
|
||||
```
|
||||
Using device <N> ('USB ...')
|
||||
Loading WakewordModel...
|
||||
Loaded in 0.XXs.
|
||||
Listening. Say 'alexa'. Ctrl-C to exit.
|
||||
```
|
||||
|
||||
Then say "alexa" 3-4 times at normal volume from ~1 m. Expected: one `DETECTED alexa score=0.XXX t=YYs` line per spoken wakeword, fired within ~1 s of saying it. **No beep yet** — that's Task 5.
|
||||
|
||||
What to check before moving on:
|
||||
|
||||
- Detection actually fires. If 0/4 attempts trigger, the port has a bug — re-verify shapes and the `x/10 + 2` mel transform before continuing.
|
||||
- No sustained `[status] input overflow` or `[status] consumer behind` lines. Occasional ones at startup are OK.
|
||||
- No ORT GPU warnings (`/sys/class/drm/card0` etc.) — `ORT_LOGGING_LEVEL=3` should silence them. If they appear, the `Libc.setenv` for ORT_LOGGING_LEVEL isn't being honoured by the onnxruntime version pinned; consult Context7 for the correct env var name for that version.
|
||||
- Ctrl-C from the workstation cleanly exits the probe (prints `bye`). If Ctrl-C just hangs the SSH session, check that `bin/probe-cs-2` uses `ssh -t`.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```sh
|
||||
git add tests/02-wakeword-cs/Program.cs
|
||||
git commit -m "Live wakeword loop: PortAudio input + queue + threshold detection"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Fire-and-forget beep + full hardware verification + `findings.md` update
|
||||
|
||||
**Goal:** Add the detection beep without blocking input ingestion, then run the full hardware test against all three pass criteria from the spec, and capture findings.
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/02-wakeword-cs/Program.cs` (add beep helper + wire it into the detection branch)
|
||||
- Modify: `findings.md` (append Test 2 outcome section)
|
||||
|
||||
- [ ] **Step 1: Add the beep buffer + dispatch helper to `Program.cs`**
|
||||
|
||||
At the top of `Program.cs`, after the `const double CooldownSeconds = 1.0;` line, add:
|
||||
|
||||
```csharp
|
||||
const double BeepHz = 880.0;
|
||||
const double BeepSeconds = 0.2;
|
||||
```
|
||||
|
||||
Before the line `Console.WriteLine("Listening. Say 'alexa'. Ctrl-C to exit.");`, add the beep buffer precomputation:
|
||||
|
||||
```csharp
|
||||
short[] beepBuffer = MakeBeep(BeepHz, BeepSeconds, SampleRate);
|
||||
```
|
||||
|
||||
In the detection branch (inside the main loop, after `lastTrigger = now;`), add:
|
||||
|
||||
```csharp
|
||||
FireAndForgetBeep(device, beepBuffer);
|
||||
```
|
||||
|
||||
At the bottom of the file (after `FindUsbDevice` but before `class Libc`), add:
|
||||
|
||||
```csharp
|
||||
static short[] MakeBeep(double freqHz, double durationS, int sampleRate)
|
||||
{
|
||||
int n = (int)(sampleRate * durationS);
|
||||
var buf = new short[n];
|
||||
for (int i = 0; i < n; i++)
|
||||
{
|
||||
double t = i / (double)sampleRate;
|
||||
double v = 0.3 * Math.Sin(2.0 * Math.PI * freqHz * t);
|
||||
buf[i] = (short)(v * short.MaxValue);
|
||||
}
|
||||
return buf;
|
||||
}
|
||||
|
||||
static void FireAndForgetBeep(int device, short[] beepBuffer)
|
||||
{
|
||||
int offset = 0;
|
||||
var done = new ManualResetEventSlim(false);
|
||||
|
||||
var outParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = 1,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback playCb = (IntPtr _, IntPtr output, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
|
||||
{
|
||||
int remaining = beepBuffer.Length - offset;
|
||||
int take = (int)Math.Min(frameCount, (uint)remaining);
|
||||
int silence = (int)frameCount - take;
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
if (take > 0)
|
||||
{
|
||||
fixed (short* src = &beepBuffer[offset])
|
||||
Buffer.MemoryCopy(src, dst, take * sizeof(short), take * sizeof(short));
|
||||
offset += take;
|
||||
}
|
||||
for (int i = take; i < frameCount; i++) dst[i] = 0;
|
||||
}
|
||||
if (offset >= beepBuffer.Length)
|
||||
{
|
||||
done.Set();
|
||||
return StreamCallbackResult.Complete;
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
var stream = new Stream(
|
||||
null, outParams, WakewordModel.SampleRate, 1024, StreamFlags.NoFlag, playCb, IntPtr.Zero);
|
||||
Task.Run(() =>
|
||||
{
|
||||
done.Wait();
|
||||
// Brief drain pause matches Test 1's behaviour; the device buffer needs a moment after Complete.
|
||||
Thread.Sleep(200);
|
||||
stream.Stop();
|
||||
stream.Dispose();
|
||||
done.Dispose();
|
||||
});
|
||||
stream.Start();
|
||||
}
|
||||
```
|
||||
|
||||
Note: `MakeBeep` and `FireAndForgetBeep` are `static` local-method-style helpers; with top-level statements they live at file scope alongside `FindUsbDevice` and `Libc`. They reference `Stream` (the `using` alias) and `PortAudio.GetDeviceInfo` — both already in scope.
|
||||
|
||||
The `Task.Run` is captured by the closure but not awaited; the main loop returns immediately after `stream.Start()`. The stream + event handle are kept alive by the closure until the task disposes them.
|
||||
|
||||
- [ ] **Step 2: Build locally**
|
||||
|
||||
```sh
|
||||
dotnet build tests/02-wakeword-cs -c Release
|
||||
```
|
||||
|
||||
Expected: success.
|
||||
|
||||
- [ ] **Step 3: Run on the Pi and verify pass criterion 1 (true-positive rate)**
|
||||
|
||||
```sh
|
||||
bin/probe-cs-2
|
||||
```
|
||||
|
||||
Wait for `Listening...`. From ~1 m, say "alexa" at normal volume, **10 times**, pausing ≥ 2 seconds between attempts. Count printed `DETECTED ...` lines AND audible beeps.
|
||||
|
||||
**Pass criterion 1:** ≥ 8/10 detections, each within ~1 s of saying the word.
|
||||
|
||||
If you get fewer: the issue is in the inference pipeline. Common causes:
|
||||
- `x/10 + 2` mel transform wasn't applied (search `WakewordModel.cs` for `/10` — if missing, that's it).
|
||||
- Off-by-one in the mel window slice (`-EmbeddingWindowMelFrames` vs `-EmbeddingWindowMelFrames-1`).
|
||||
- Dtype mismatch — passing int16 directly instead of converting to float32.
|
||||
|
||||
- [ ] **Step 4: Verify pass criterion 2 (false-positive rate)**
|
||||
|
||||
While the probe is still running, read a newspaper or book aloud at normal volume from ~1 m for **3 continuous minutes** (anything that isn't "alexa"). Count any spurious `DETECTED ...` lines.
|
||||
|
||||
**Pass criterion 2:** ≤ 1 false positive per minute (i.e., ≤ 3 over the 3-minute test).
|
||||
|
||||
If you get more: the threshold (0.5) was right for the Python probe; if C# is over-triggering, the pipeline is producing systematically higher scores than Python — likely a normalization or transform bug (revisit Step 3 troubleshooting).
|
||||
|
||||
- [ ] **Step 5: Verify pass criterion 3 (CPU budget)**
|
||||
|
||||
In a **second terminal**, while the probe is still running:
|
||||
|
||||
```sh
|
||||
sshpass -p assistant ssh pi@192.168.50.115 htop
|
||||
```
|
||||
|
||||
Find the `Probe` process row. Note the `CPU%` column over ~30 seconds while the probe is doing inference (speak occasionally to keep it busy).
|
||||
|
||||
**Pass criterion 3:** CPU% stays under 50% of one core (the Pi 4 has 4 cores, so htop shows "100%" per core — pass = stays below 50 in that column).
|
||||
|
||||
If higher: confirm `WakewordModel`'s `SessionOptions` set `IntraOpNumThreads = 1` and `InterOpNumThreads = 1` (Task 3 Step 1). If they're set and it's still hot, ONNX Runtime may be ignoring the limit — set the env var via `Libc.setenv("OMP_NUM_THREADS", "1", 1)` at the top of `Program.cs` alongside the others, redeploy, recheck.
|
||||
|
||||
Ctrl-C the probe when done. Quit htop.
|
||||
|
||||
- [ ] **Step 6: Commit the beep code**
|
||||
|
||||
```sh
|
||||
git add tests/02-wakeword-cs/Program.cs
|
||||
git commit -m "Beep on detection: fire-and-forget OutputStream, input keeps flowing"
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Append Test 2 outcome section to `findings.md`**
|
||||
|
||||
Open `findings.md` and append, after the existing `## C# probe outcome (2026-06-12 — Test 1 ported to C#)` section, a new section in the same shape:
|
||||
|
||||
```markdown
|
||||
## C# wakeword probe outcome (2026-06-12 — Test 2 ported to C#)
|
||||
|
||||
The Python Test 2 (openwakeword "alexa" listener with beep on detect) was re-implemented in C# / .NET 9, with the openwakeword `Model.predict()` streaming pipeline ported to `Microsoft.ML.OnnxRuntime`. Probe lives at `tests/02-wakeword-cs/`; deploy script `bin/probe-cs-2`. All three pass criteria from the original spec held on hardware.
|
||||
|
||||
### What we proved
|
||||
|
||||
- **Microsoft.ML.OnnxRuntime <ORT_VER>** runs on linux-arm64 from a self-contained `dotnet publish`. NuGet [bundles / does not bundle — confirm] `libonnxruntime.so` for the RID.
|
||||
- The three openwakeword ONNX models (`melspectrogram.onnx`, `embedding_model.onnx`, `alexa.onnx`, vendored from openwakeword 0.6.0) load and chain correctly when fed real audio.
|
||||
- **Two PortAudio streams on one USB device works**: the input stream stays open while a per-detection fire-and-forget output stream plays the beep. No errors, no input overflow during playback (fixing the bug `findings.md` § A flagged in the Python probe).
|
||||
- True-positive rate: <X>/10, false-positive rate: <Y>/3 min, CPU: <Z>% of one core. (Fill in observed values.)
|
||||
|
||||
### Key gotchas
|
||||
|
||||
- (Capture anything that took > 15 minutes to debug. Likely candidates: env-var name for ORT log level on this version; whether the mel transform `x/10 + 2` actually mattered in C#; whether `IntraOpNumThreads = 1` was honoured; whether shape assertions caught anything during development.)
|
||||
|
||||
### Open questions for the main assistant
|
||||
|
||||
- **Custom wakeword model.** Stock "alexa" works as a probe but the shipped assistant needs a custom "hey assistant" (or similar) model. openwakeword has a Colab notebook for synthetic training; budget ~1 hour to train + verify on the same pipeline this probe validates.
|
||||
- **Speaker-is-mic self-trigger.** Not in probe scope, but the main assistant will need detector gating during TTS playback (findings.md § B), or hardware/software AEC.
|
||||
- **Inference latency.** If `WakewordModel.Predict` ever exceeds 80 ms, the queue backs up; sample real measurements from this probe before sizing the queue / consumer thread in the main assistant.
|
||||
```
|
||||
|
||||
Replace the bracketed placeholders (`<ORT_VER>`, `<X>`, `<Y>`, `<Z>`, the bundling note, the gotchas list) with the actual values you observed. **Do not commit with placeholders.**
|
||||
|
||||
- [ ] **Step 8: Commit findings**
|
||||
|
||||
```sh
|
||||
git add findings.md
|
||||
git commit -m "Findings: C# wakeword probe outcome"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Notes on what is intentionally **not** in this plan
|
||||
|
||||
- **No automated tests.** The spec is explicit; verification is the listening / counting test on real hardware. The silence smoke test in Task 3 is a pipeline sanity check, not a model-correctness test.
|
||||
- **No numerical parity test vs Python.** If hardware passes, the port is good enough for the probe. If hardware fails, the diagnostic levers in the spec (Section "Diagnostic levers if it fails") cover the likely causes.
|
||||
- **No detector gating during beep playback.** The 200 ms beep at 880 Hz is far enough from human speech to not self-trigger the "alexa" classifier; if it did, the spec's § B (gate detector during TTS) is the main-assistant solution, not the probe's.
|
||||
- **No retry / restart on PortAudio init failure.** Probe — crash and surface.
|
||||
- **No abstraction over PortAudio.** Same one-file Program.cs shape as Test 1.
|
||||
- **No custom wakeword.** Stock alexa only. Custom is a separate task per `findings.md` § C.
|
||||
- **No bin/probe-cs-2 → bin/probe-cs unification.** Decided in brainstorm — kept as separate scripts.
|
||||
|
||||
If during implementation you feel the urge to add any of the above, stop and re-read the spec. The point of this probe is to be cheap and disposable.
|
||||
@@ -1,758 +0,0 @@
|
||||
# Test 3 — Full-cycle C# probe Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build a single-binary C# probe at `tests/03-full-cycle-cs/` that runs the full assistant cycle on the Pi: idle → "alexa" → beep → record 5 s → play back → idle, with detector gating during non-idle states and a `WakewordModel.Reset()` on every IDLE re-entry.
|
||||
|
||||
**Architecture:** One persistent PortAudio input stream feeds 80 ms frames into a `BlockingCollection`; a single consumer thread runs a four-state machine (IDLE / BEEPING / RECORDING / PLAYBACK). Beep and playback are fire-and-forget output streams using Test 2's `Start()`-before-`Task.Run` + 2 s timeout lifetime pattern. `WakewordModel` is copied verbatim from Test 2 with a new `Reset()` method called on PLAYBACK→IDLE.
|
||||
|
||||
**Tech Stack:** .NET 9 (self-contained linux-arm64), PortAudioSharp2 1.0.6, Microsoft.ML.OnnxRuntime 1.26.0. Deploy via `dotnet publish` + `scp` + `ssh -t` to Pi at `pi@192.168.50.115`.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-06-12-test-3-full-cycle-csharp-design.md`. Read it before starting.
|
||||
|
||||
**Branch:** `fresh-start` (already current; do NOT create a new branch).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Scaffold `tests/03-full-cycle-cs/` project skeleton
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/03-full-cycle-cs/Probe.csproj`
|
||||
- Create: `tests/03-full-cycle-cs/models/` (directory; populated in Task 2)
|
||||
|
||||
- [ ] **Step 1: Create the project directory**
|
||||
|
||||
```bash
|
||||
mkdir -p tests/03-full-cycle-cs/models
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write `Probe.csproj`**
|
||||
|
||||
Create `tests/03-full-cycle-cs/Probe.csproj` with this exact content (identical to `tests/02-wakeword-cs/Probe.csproj` except `RootNamespace`):
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<RootNamespace>FullCycleProbe</RootNamespace>
|
||||
<AssemblyName>Probe</AssemblyName>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
|
||||
<SelfContained>true</SelfContained>
|
||||
<PublishSingleFile>false</PublishSingleFile>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="PortAudioSharp2" Version="1.0.6" />
|
||||
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.26.0" />
|
||||
</ItemGroup>
|
||||
<ItemGroup>
|
||||
<Content Include="models/*.onnx">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</Content>
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit the scaffold**
|
||||
|
||||
```bash
|
||||
git add tests/03-full-cycle-cs/Probe.csproj
|
||||
git commit -m "Test 3 scaffold: csproj for tests/03-full-cycle-cs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Copy models and `WakewordModel.cs` from Test 2 (with namespace rename and `Reset()`)
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/03-full-cycle-cs/WakewordModel.cs` (copied from Test 2, namespace renamed, `Reset()` added)
|
||||
- Create: `tests/03-full-cycle-cs/models/melspectrogram.onnx`
|
||||
- Create: `tests/03-full-cycle-cs/models/embedding_model.onnx`
|
||||
- Create: `tests/03-full-cycle-cs/models/alexa.onnx`
|
||||
|
||||
- [ ] **Step 1: Copy the three ONNX model files**
|
||||
|
||||
```bash
|
||||
cp tests/02-wakeword-cs/models/melspectrogram.onnx tests/03-full-cycle-cs/models/
|
||||
cp tests/02-wakeword-cs/models/embedding_model.onnx tests/03-full-cycle-cs/models/
|
||||
cp tests/02-wakeword-cs/models/alexa.onnx tests/03-full-cycle-cs/models/
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Copy `WakewordModel.cs`**
|
||||
|
||||
```bash
|
||||
cp tests/02-wakeword-cs/WakewordModel.cs tests/03-full-cycle-cs/WakewordModel.cs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Rename the namespace in the copy**
|
||||
|
||||
Edit `tests/03-full-cycle-cs/WakewordModel.cs`: change line 4 from `namespace WakewordProbe;` to `namespace FullCycleProbe;`. Leave everything else unchanged.
|
||||
|
||||
- [ ] **Step 4: Add `Reset()` method to the copy**
|
||||
|
||||
Edit `tests/03-full-cycle-cs/WakewordModel.cs`. Insert the following method immediately before the `public void Dispose()` method near the bottom of the file:
|
||||
|
||||
```csharp
|
||||
public void Reset()
|
||||
{
|
||||
_rawRingFill = 0;
|
||||
_melRing.Clear();
|
||||
_embRing.Clear();
|
||||
_framesSeen = 0;
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
(Note the trailing blank line. The result should be a method block sitting between `return clsOut[0, 0]; }` (end of `Predict`) and `public void Dispose()`.)
|
||||
|
||||
- [ ] **Step 5: Verify the project builds**
|
||||
|
||||
Run from repo root:
|
||||
|
||||
```bash
|
||||
dotnet build tests/03-full-cycle-cs -c Release
|
||||
```
|
||||
|
||||
Expected: `Build succeeded` with 0 errors, 0 warnings. (At this point `Program.cs` doesn't exist yet; csproj is `<OutputType>Exe</OutputType>` but the build will produce a no-entry-point warning or error. If it errors with "CS5001: Program does not contain a static 'Main' method", that's expected at this stage — proceed to Task 3 which adds `Program.cs`.)
|
||||
|
||||
If the build errors for any reason other than the missing entry point, fix before continuing.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add tests/03-full-cycle-cs/WakewordModel.cs tests/03-full-cycle-cs/models/
|
||||
git commit -m "Test 3: copy WakewordModel + models from Test 2, add Reset()"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Write `Program.cs` with the full state machine
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/03-full-cycle-cs/Program.cs`
|
||||
|
||||
The whole file is shown below. It lifts from `tests/02-wakeword-cs/Program.cs` (env-var setup, OrtEnv init, FindUsbDevice, input stream + queue, FireAndForgetBeep, Libc, MakeBeep) and adds:
|
||||
|
||||
- `volatile State _state` enum-as-state-machine in the consumer loop.
|
||||
- `FireAndForgetPlayback` helper (same shape as `FireAndForgetBeep` but reads from `recordBuf` and signals via `_playbackDoneFlag` instead of a `ManualResetEventSlim`).
|
||||
- `model.Reset()` call on PLAYBACK→IDLE transition.
|
||||
|
||||
- [ ] **Step 1: Write `Program.cs` in full**
|
||||
|
||||
Create `tests/03-full-cycle-cs/Program.cs` with exactly this content:
|
||||
|
||||
```csharp
|
||||
using System.Collections.Concurrent;
|
||||
using System.Diagnostics;
|
||||
using System.Runtime.InteropServices;
|
||||
using FullCycleProbe;
|
||||
using Microsoft.ML.OnnxRuntime;
|
||||
using PortAudioSharp;
|
||||
using Stream = PortAudioSharp.Stream;
|
||||
|
||||
const int SampleRate = WakewordModel.SampleRate; // 16 000
|
||||
const int Channels = 1;
|
||||
const uint BlockFrames = WakewordModel.FrameSamples; // 1280 (80 ms)
|
||||
const float Threshold = 0.5f;
|
||||
const double CooldownSeconds = 1.0;
|
||||
const double BeepHz = 880.0;
|
||||
const double BeepSeconds = 0.2;
|
||||
|
||||
const int BeepingFrames = 3; // 3 * 80 ms = 240 ms (covers 200 ms beep + 40 ms drain/reverb margin)
|
||||
const int RecordingFrames = 63; // 63 * 80 ms = 5.04 s (rounded up from 5 s to avoid partial-frame handling)
|
||||
const int RecordBufSamples = RecordingFrames * (int)BlockFrames; // 80640
|
||||
|
||||
// .NET-side mirror for any readers that go through Environment.GetEnvironmentVariable,
|
||||
// AND libc setenv so PortAudio / onnxruntime (which use getenv()) see the values.
|
||||
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
|
||||
Libc.setenv("PA_ALSA_PLUGHW", "1", 1);
|
||||
Environment.SetEnvironmentVariable("ORT_LOGGING_LEVEL", "3");
|
||||
Libc.setenv("ORT_LOGGING_LEVEL", "3", 1);
|
||||
// ORT 1.26.0: the env var does NOT suppress early GPU-discovery warnings from device_discovery.cc.
|
||||
// CreateInstanceWithOptions sets the log level at OrtEnv creation time, before EP discovery runs.
|
||||
var ortEnvOpts = new EnvironmentCreationOptions { logId = "FullCycleProbe", logLevel = OrtLoggingLevel.ORT_LOGGING_LEVEL_ERROR };
|
||||
OrtEnv.CreateInstanceWithOptions(ref ortEnvOpts);
|
||||
|
||||
PortAudio.Initialize();
|
||||
WakewordModel? model = null;
|
||||
try
|
||||
{
|
||||
int device = FindUsbDevice();
|
||||
Console.WriteLine($"Using device {device} ('{PortAudio.GetDeviceInfo(device).name}')");
|
||||
|
||||
Console.WriteLine("Loading WakewordModel...");
|
||||
var t0 = DateTime.UtcNow;
|
||||
string modelsDir = Path.Combine(AppContext.BaseDirectory, "models");
|
||||
model = new WakewordModel(
|
||||
Path.Combine(modelsDir, "melspectrogram.onnx"),
|
||||
Path.Combine(modelsDir, "embedding_model.onnx"),
|
||||
Path.Combine(modelsDir, "alexa.onnx"));
|
||||
Console.WriteLine($"Loaded in {(DateTime.UtcNow - t0).TotalSeconds:0.00}s.");
|
||||
|
||||
var queue = new BlockingCollection<short[]>(boundedCapacity: 16);
|
||||
using var cts = new CancellationTokenSource();
|
||||
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
|
||||
|
||||
var inParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowInputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback callback = (IntPtr input, IntPtr _, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags status, IntPtr _3) =>
|
||||
{
|
||||
if (status.HasFlag(StreamCallbackFlags.InputOverflow))
|
||||
Console.Error.WriteLine("[status] input overflow");
|
||||
|
||||
if (frameCount != BlockFrames)
|
||||
{
|
||||
Console.Error.WriteLine($"[status] unexpected callback frameCount={frameCount} (want {BlockFrames})");
|
||||
return StreamCallbackResult.Continue;
|
||||
}
|
||||
|
||||
var frame = new short[BlockFrames];
|
||||
unsafe
|
||||
{
|
||||
short* src = (short*)input.ToPointer();
|
||||
fixed (short* dst = &frame[0])
|
||||
Buffer.MemoryCopy(src, dst, BlockFrames * sizeof(short), BlockFrames * sizeof(short));
|
||||
}
|
||||
if (!queue.TryAdd(frame, 0))
|
||||
Console.Error.WriteLine("[status] consumer behind, dropping frame");
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
using var stream = new Stream(
|
||||
inParams, null, SampleRate, BlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
|
||||
stream.Start();
|
||||
short[] beepBuffer = MakeBeep(BeepHz, BeepSeconds, SampleRate);
|
||||
Console.WriteLine("Listening. Say 'alexa'. Ctrl-C to exit.");
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
TimeSpan lastTrigger = TimeSpan.FromSeconds(-CooldownSeconds);
|
||||
|
||||
// State machine (single-writer: this consumer thread).
|
||||
State state = State.Idle;
|
||||
int beepFrames = 0;
|
||||
int recordOffset = 0;
|
||||
short[] recordBuf = null!; // assigned on BEEPING→RECORDING transition
|
||||
var playbackDone = new PlaybackDoneFlag();
|
||||
|
||||
try
|
||||
{
|
||||
while (!cts.IsCancellationRequested)
|
||||
{
|
||||
short[] frame = queue.Take(cts.Token);
|
||||
|
||||
switch (state)
|
||||
{
|
||||
case State.Idle:
|
||||
{
|
||||
float score = model.Predict(frame);
|
||||
var now = sw.Elapsed;
|
||||
if (score >= Threshold && (now - lastTrigger).TotalSeconds >= CooldownSeconds)
|
||||
{
|
||||
Console.WriteLine($"DETECTED alexa score={score:0.000} t={now.TotalSeconds:0.0}s");
|
||||
lastTrigger = now;
|
||||
try
|
||||
{
|
||||
FireAndForgetBeep(device, beepBuffer);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] beep stream failed to start: {ex.Message}");
|
||||
// Stay in IDLE — no cycle to abort yet.
|
||||
break;
|
||||
}
|
||||
Console.WriteLine("[state] BEEPING");
|
||||
state = State.Beeping;
|
||||
beepFrames = 0;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case State.Beeping:
|
||||
{
|
||||
beepFrames++;
|
||||
if (beepFrames >= BeepingFrames)
|
||||
{
|
||||
Console.WriteLine("[state] RECORDING");
|
||||
state = State.Recording;
|
||||
recordBuf = new short[RecordBufSamples];
|
||||
recordOffset = 0;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case State.Recording:
|
||||
{
|
||||
unsafe
|
||||
{
|
||||
fixed (short* src = &frame[0])
|
||||
fixed (short* dst = &recordBuf[recordOffset])
|
||||
Buffer.MemoryCopy(src, dst, BlockFrames * sizeof(short), BlockFrames * sizeof(short));
|
||||
}
|
||||
recordOffset += (int)BlockFrames;
|
||||
if (recordOffset >= RecordBufSamples)
|
||||
{
|
||||
try
|
||||
{
|
||||
FireAndForgetPlayback(device, recordBuf, playbackDone);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] playback stream failed to start: {ex.Message}");
|
||||
// Drop the recording, reset model, return to IDLE.
|
||||
model.Reset();
|
||||
Console.WriteLine("[state] IDLE");
|
||||
state = State.Idle;
|
||||
break;
|
||||
}
|
||||
Console.WriteLine("[state] PLAYBACK");
|
||||
state = State.Playback;
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
case State.Playback:
|
||||
{
|
||||
// Discard input during playback (gate the detector).
|
||||
if (playbackDone.Consume())
|
||||
{
|
||||
model.Reset();
|
||||
Console.WriteLine("[state] IDLE");
|
||||
state = State.Idle;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { /* Ctrl-C path */ }
|
||||
|
||||
stream.Stop();
|
||||
Console.WriteLine("\nbye");
|
||||
}
|
||||
finally
|
||||
{
|
||||
model?.Dispose();
|
||||
PortAudio.Terminate();
|
||||
}
|
||||
|
||||
static int FindUsbDevice()
|
||||
{
|
||||
int count = PortAudio.DeviceCount;
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
if (info.name.ToLowerInvariant().Contains("usb") && info.maxInputChannels >= 1)
|
||||
return i;
|
||||
}
|
||||
Console.Error.WriteLine("USB audio device not found. Devices:");
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
Console.Error.WriteLine(
|
||||
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels}");
|
||||
}
|
||||
Environment.Exit(1);
|
||||
return -1; // unreachable
|
||||
}
|
||||
|
||||
static short[] MakeBeep(double freqHz, double durationS, int sampleRate)
|
||||
{
|
||||
int n = (int)(sampleRate * durationS);
|
||||
var buf = new short[n];
|
||||
for (int i = 0; i < n; i++)
|
||||
{
|
||||
double t = i / (double)sampleRate;
|
||||
double v = 0.3 * Math.Sin(2.0 * Math.PI * freqHz * t);
|
||||
buf[i] = (short)(v * short.MaxValue);
|
||||
}
|
||||
return buf;
|
||||
}
|
||||
|
||||
static void FireAndForgetBeep(int device, short[] beepBuffer)
|
||||
{
|
||||
int offset = 0;
|
||||
var done = new ManualResetEventSlim(false);
|
||||
|
||||
var outParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = 1,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback playCb = (IntPtr _, IntPtr output, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
|
||||
{
|
||||
int remaining = beepBuffer.Length - offset;
|
||||
int take = (int)Math.Min(frameCount, (uint)remaining);
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
if (take > 0)
|
||||
{
|
||||
fixed (short* src = &beepBuffer[offset])
|
||||
Buffer.MemoryCopy(src, dst, take * sizeof(short), take * sizeof(short));
|
||||
offset += take;
|
||||
}
|
||||
for (int i = take; i < frameCount; i++) dst[i] = 0;
|
||||
}
|
||||
if (offset >= beepBuffer.Length)
|
||||
{
|
||||
done.Set();
|
||||
return StreamCallbackResult.Complete;
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
var stream = new Stream(
|
||||
null, outParams, WakewordModel.SampleRate, 1024, StreamFlags.NoFlag, playCb, IntPtr.Zero);
|
||||
try
|
||||
{
|
||||
stream.Start();
|
||||
}
|
||||
catch
|
||||
{
|
||||
stream.Dispose();
|
||||
done.Dispose();
|
||||
throw;
|
||||
}
|
||||
Task.Run(() =>
|
||||
{
|
||||
// 2 s timeout so a failed callback (which PortAudio silently swallows
|
||||
// and would leave 'done' unset) eventually releases the stream + handle
|
||||
// instead of leaking them for the process lifetime.
|
||||
done.Wait(TimeSpan.FromSeconds(2));
|
||||
Thread.Sleep(200);
|
||||
stream.Stop();
|
||||
stream.Dispose();
|
||||
done.Dispose();
|
||||
});
|
||||
}
|
||||
|
||||
static void FireAndForgetPlayback(int device, short[] recordBuf, PlaybackDoneFlag doneFlag)
|
||||
{
|
||||
int offset = 0;
|
||||
var done = new ManualResetEventSlim(false);
|
||||
|
||||
var outParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = 1,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback playCb = (IntPtr _, IntPtr output, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
|
||||
{
|
||||
int remaining = recordBuf.Length - offset;
|
||||
int take = (int)Math.Min(frameCount, (uint)remaining);
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
if (take > 0)
|
||||
{
|
||||
fixed (short* src = &recordBuf[offset])
|
||||
Buffer.MemoryCopy(src, dst, take * sizeof(short), take * sizeof(short));
|
||||
offset += take;
|
||||
}
|
||||
for (int i = take; i < frameCount; i++) dst[i] = 0;
|
||||
}
|
||||
if (offset >= recordBuf.Length)
|
||||
{
|
||||
done.Set();
|
||||
return StreamCallbackResult.Complete;
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
var stream = new Stream(
|
||||
null, outParams, WakewordModel.SampleRate, 1024, StreamFlags.NoFlag, playCb, IntPtr.Zero);
|
||||
try
|
||||
{
|
||||
stream.Start();
|
||||
}
|
||||
catch
|
||||
{
|
||||
stream.Dispose();
|
||||
done.Dispose();
|
||||
throw;
|
||||
}
|
||||
Task.Run(() =>
|
||||
{
|
||||
// Same 2 s safety timeout as the beep cleanup.
|
||||
done.Wait(TimeSpan.FromSeconds(2));
|
||||
Thread.Sleep(200);
|
||||
stream.Stop();
|
||||
stream.Dispose();
|
||||
done.Dispose();
|
||||
doneFlag.Set(); // tell the consumer it's safe to flip PLAYBACK→IDLE
|
||||
});
|
||||
}
|
||||
|
||||
enum State { Idle, Beeping, Recording, Playback }
|
||||
|
||||
// Volatile flag set by the playback cleanup task and consumed by the consumer thread.
|
||||
// Wrapped in a tiny class so we can pass it by reference into the helper.
|
||||
sealed class PlaybackDoneFlag
|
||||
{
|
||||
private volatile bool _set;
|
||||
public void Set() => _set = true;
|
||||
public bool Consume()
|
||||
{
|
||||
if (!_set) return false;
|
||||
_set = false;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
static class Libc
|
||||
{
|
||||
[DllImport("libc", EntryPoint = "setenv")]
|
||||
public static extern int setenv(string name, string value, int overwrite);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify the project builds cleanly**
|
||||
|
||||
```bash
|
||||
dotnet build tests/03-full-cycle-cs -c Release
|
||||
```
|
||||
|
||||
Expected: `Build succeeded` with 0 errors, 0 warnings.
|
||||
|
||||
If `<AllowUnsafeBlocks>` is missing from the csproj, the build will fail with `CS0227: Unsafe code may only appear if compiling with /unsafe`. Confirm Task 1's csproj has `<AllowUnsafeBlocks>true</AllowUnsafeBlocks>`.
|
||||
|
||||
If you see `CS0246: The type or namespace 'Stream' could not be found` or a clash with `System.IO.Stream`, confirm the `using Stream = PortAudioSharp.Stream;` line is at the top of `Program.cs`.
|
||||
|
||||
- [ ] **Step 3: Verify `dotnet publish` produces a runnable binary**
|
||||
|
||||
```bash
|
||||
dotnet publish tests/03-full-cycle-cs -c Release -r linux-arm64 --self-contained -o /tmp/probe-cs-3-out
|
||||
ls /tmp/probe-cs-3-out/Probe
|
||||
ls /tmp/probe-cs-3-out/models/
|
||||
```
|
||||
|
||||
Expected:
|
||||
- `Probe` (executable) exists.
|
||||
- `models/` directory contains `melspectrogram.onnx`, `embedding_model.onnx`, `alexa.onnx`.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add tests/03-full-cycle-cs/Program.cs
|
||||
git commit -m "Test 3: full-cycle state machine (IDLE/BEEPING/RECORDING/PLAYBACK)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Write `bin/probe-cs-3` deploy script
|
||||
|
||||
**Files:**
|
||||
- Create: `bin/probe-cs-3`
|
||||
|
||||
- [ ] **Step 1: Write the deploy script**
|
||||
|
||||
Create `bin/probe-cs-3` with exactly this content (it's `bin/probe-cs-2` with three string swaps: `02-wakeword-cs` → `03-full-cycle-cs`, `probe-cs-2` → `probe-cs-3`, `/tmp/probe-cs-2-out` → `/tmp/probe-cs-3-out`):
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-3-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/03-full-cycle-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-3/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-3 && mkdir ~/probe-cs-3'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-3/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-3 && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Make it executable**
|
||||
|
||||
```bash
|
||||
chmod +x bin/probe-cs-3
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add bin/probe-cs-3
|
||||
git commit -m "Test 3: bin/probe-cs-3 deploy script"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Hardware verification on the Pi
|
||||
|
||||
This task is the actual pass/fail gate. No automated tests; verification is the human ear + console output + `top` reading on the Pi. If anything in this task fails, do NOT mark the implementation complete — investigate and either fix in code or update the spec to reflect what actually works.
|
||||
|
||||
**Files:** (none — this is a runtime verification task)
|
||||
|
||||
- [ ] **Step 1: Confirm the Pi is reachable**
|
||||
|
||||
Run from workstation:
|
||||
|
||||
```bash
|
||||
ping -c 2 192.168.50.115
|
||||
```
|
||||
|
||||
Expected: 2/2 replies. If unreachable, stop and ask the user.
|
||||
|
||||
- [ ] **Step 2: Deploy + run**
|
||||
|
||||
Run from workstation:
|
||||
|
||||
```bash
|
||||
./bin/probe-cs-3
|
||||
```
|
||||
|
||||
Expected console output (abridged):
|
||||
|
||||
```
|
||||
>> dotnet publish (linux-arm64, self-contained)
|
||||
... (build output) ...
|
||||
>> scp to pi@192.168.50.115:~/probe-cs-3/ (wipe-and-replace)
|
||||
>> ssh + run on Pi (Ctrl-C from this terminal stops the probe)
|
||||
[wakeword-model] melspectrogram.onnx
|
||||
input ...
|
||||
[wakeword-model] embedding_model.onnx ...
|
||||
[wakeword-model] alexa.onnx ...
|
||||
Loaded in 0.6s.
|
||||
Using device <N> ('USB Speaker Phone...')
|
||||
Listening. Say 'alexa'. Ctrl-C to exit.
|
||||
```
|
||||
|
||||
If the publish step errors with `setenv: command not found` or similar libc-related runtime crash, confirm Task 3's `Libc.setenv` `[DllImport]` is present and the binary was self-contained-published.
|
||||
|
||||
If the Pi run errors with `Cannot open shared library libonnxruntime.so` or `libportaudio.so`, the self-contained publish didn't bundle the native libs — confirm `<SelfContained>true</SelfContained>` and `<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>` in Task 1's csproj.
|
||||
|
||||
- [ ] **Step 3: Run 3 full cycles back-to-back**
|
||||
|
||||
Say "alexa" clearly at normal volume from ~1 m. Observe:
|
||||
|
||||
1. Beep plays within ~100 ms of saying "alexa".
|
||||
2. Console prints `DETECTED alexa score=…`, then `[state] BEEPING`, then `[state] RECORDING`.
|
||||
3. Say something distinctive (e.g. "hello world, this is cycle one") within the 5 s recording window.
|
||||
4. Console prints `[state] PLAYBACK`.
|
||||
5. Speaker plays back what you said. It should be recognisable.
|
||||
6. Console prints `[state] IDLE`.
|
||||
7. Pause ~1.5 s, then say "alexa" again. Verify the cycle starts over.
|
||||
|
||||
Repeat for cycle 2 and cycle 3. Record on paper (or transcript) which cycles succeeded.
|
||||
|
||||
Pass criterion 1: 3 full cycles back-to-back; each playback recognisable as what was spoken.
|
||||
|
||||
- [ ] **Step 4: Measure idle CPU on the Pi**
|
||||
|
||||
In a second workstation terminal:
|
||||
|
||||
```bash
|
||||
sshpass -p 'assistant' ssh pi@192.168.50.115 'top -b -n 3 -d 1 -p $(pgrep -f Probe)'
|
||||
```
|
||||
|
||||
Expected: `Probe` process appears with `%CPU` < 50 across the three samples (Test 2 measured 17–31%, Test 3 should be in the same range during IDLE).
|
||||
|
||||
Pass criterion 4: CPU < 50% of one core during IDLE.
|
||||
|
||||
- [ ] **Step 5: Test post-playback re-detection latency**
|
||||
|
||||
Speak "alexa" immediately after each `[state] IDLE` log line. Note how soon detection fires.
|
||||
|
||||
Pass criterion 2: Next "alexa" after playback detected within ~2 s of playback ending. (Below ~1.3 s is impossible due to WarmupFrames; above ~2 s indicates a regression.)
|
||||
|
||||
- [ ] **Step 6: Scan for warnings**
|
||||
|
||||
Re-read the console output from Step 3. Confirm there are **no** lines of the form:
|
||||
|
||||
- `[status] input overflow`
|
||||
- `[status] consumer behind, dropping frame`
|
||||
- `[status] unexpected callback frameCount=...`
|
||||
- `[error] beep stream failed to start: ...`
|
||||
- `[error] playback stream failed to start: ...`
|
||||
|
||||
A single occurrence under abnormal load may be tolerable; sustained occurrences during normal cycles fail criterion 5.
|
||||
|
||||
- [ ] **Step 7: Test Ctrl-C clean exit**
|
||||
|
||||
Press Ctrl-C in the workstation terminal that's running the probe.
|
||||
|
||||
Expected:
|
||||
|
||||
```
|
||||
^C
|
||||
bye
|
||||
$
|
||||
```
|
||||
|
||||
Pass criterion 7: process exits 0, no stack trace, terminal returns to prompt. Verify with `echo $?` → `0`.
|
||||
|
||||
- [ ] **Step 8: Wakeword detection rate during IDLE**
|
||||
|
||||
Restart the probe (`./bin/probe-cs-3`). Wait until `Listening. Say 'alexa'. Ctrl-C to exit.` appears. Say "alexa" 10 times with a ~2 s gap between each. Count how many times `DETECTED alexa score=…` appears.
|
||||
|
||||
Pass criterion 3: ≥ 8/10. (Test 2 demonstrated this rate; Test 3 should match.)
|
||||
|
||||
- [ ] **Step 9: Update `findings.md` with the outcome**
|
||||
|
||||
Add a new section `## Test 3 full-cycle outcome (2026-06-12)` at the bottom of `findings.md` summarising:
|
||||
|
||||
- Pass / fail per criterion.
|
||||
- Any new gotchas discovered during implementation or hardware verification (especially any that contradicted the spec's predictions).
|
||||
- CPU measurement (peak + typical), detection latency post-playback, observed pass rate.
|
||||
- Reference paths (spec + plan + code + deploy script).
|
||||
|
||||
Use the existing Test 1 and Test 2 outcome sections in `findings.md` as the template.
|
||||
|
||||
- [ ] **Step 10: Commit the findings update**
|
||||
|
||||
```bash
|
||||
git add findings.md
|
||||
git commit -m "Findings: Test 3 full-cycle probe outcome"
|
||||
```
|
||||
|
||||
- [ ] **Step 11: Final status report**
|
||||
|
||||
Report to the user:
|
||||
|
||||
- Pass/fail per criterion (1 through 7 from spec).
|
||||
- Any criteria that failed and what was discovered.
|
||||
- A one-line summary of whether Test 3 unblocks the main Pi-client implementation.
|
||||
|
||||
---
|
||||
|
||||
## Self-review notes
|
||||
|
||||
- **Spec coverage:** every section of the spec maps to a task — Architecture/State machine/Components → Tasks 1–4. Why-Reset-matters → Task 2's `Reset()` + Task 3's PLAYBACK→IDLE branch. Error handling → Task 3's try/catch on the FireAndForget calls + `[status]` / `[error]` log lines. Gotchas-carried-forward → preserved verbatim in Task 3's code. Verification → Task 5.
|
||||
- **Placeholder scan:** no TBD/TODO; every code step shows the actual code; every command shows expected output.
|
||||
- **Type consistency:** `State` enum (`Idle`, `Beeping`, `Recording`, `Playback`) used consistently; `PlaybackDoneFlag.Set()` / `.Consume()` used consistently between `FireAndForgetPlayback` and the consumer's PLAYBACK branch; `WakewordModel.Reset()` called in two places (Recording-failure recovery and Playback→Idle), signature is parameterless in both.
|
||||
@@ -1,897 +0,0 @@
|
||||
# Test 4a — OpenAI Realtime API one-shot voice round-trip (C#) Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build a single-binary throwaway C# probe at `tests/04a-realtime-oneshot-cs/` that has one round-trip spoken conversation with the OpenAI Realtime API: boot → beep → user speaks one sentence → server VAD ends the turn → assistant audio reply plays back through the USB Speaker Phone → exit 0.
|
||||
|
||||
**Architecture:** One persistent `ClientWebSocket` to `wss://api.openai.com/v1/realtime?model=gpt-realtime`. Two `Task.Run` loops: ReceiveLoop parses server events with `JsonDocument` and writes decoded audio chunks to a `Channel<short[]>`; UpstreamSendLoop drains a separate `Channel<short[]>` of mic frames and sends them as `input_audio_buffer.append`. PortAudio input stream (24 kHz mono Int16, 1920 frames/cb) writes to the upstream channel via callback; output stream (24 kHz, 1024 frames/cb) drains the downstream channel into the speaker, zero-filling underruns. Completion (`response.done` received AND downstream channel drained AND working chunk empty) is signaled via `ManualResetEventSlim` to the main thread, which closes the WS and exits.
|
||||
|
||||
**Tech Stack:** .NET 9 (self-contained linux-arm64), PortAudioSharp2 1.0.6, `System.Net.WebSockets.ClientWebSocket`, `System.Text.Json`, `System.Threading.Channels`. Deploy via `dotnet publish` + `scp` + `ssh -t` to Pi at `pi@192.168.50.115`. No onnxruntime — no wakeword in 4a.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-06-12-test-4a-openai-realtime-design.md`. Read it before starting.
|
||||
|
||||
**Branch:** `fresh-start` (already current; do NOT create a new branch).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Scaffold `tests/04a-realtime-oneshot-cs/` project skeleton
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/04a-realtime-oneshot-cs/Probe.csproj`
|
||||
|
||||
- [ ] **Step 1: Create the project directory**
|
||||
|
||||
```bash
|
||||
mkdir -p tests/04a-realtime-oneshot-cs
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write `Probe.csproj`**
|
||||
|
||||
Create `tests/04a-realtime-oneshot-cs/Probe.csproj` with this exact content. Note the differences from Test 3's csproj: `RootNamespace=RealtimeOneShotProbe`, no `Microsoft.ML.OnnxRuntime` reference (no wakeword in 4a), no `models/*.onnx` content block.
|
||||
|
||||
```xml
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net9.0</TargetFramework>
|
||||
<RootNamespace>RealtimeOneShotProbe</RootNamespace>
|
||||
<AssemblyName>Probe</AssemblyName>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
|
||||
<SelfContained>true</SelfContained>
|
||||
<PublishSingleFile>false</PublishSingleFile>
|
||||
<InvariantGlobalization>true</InvariantGlobalization>
|
||||
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<PackageReference Include="PortAudioSharp2" Version="1.0.6" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
```
|
||||
|
||||
`ClientWebSocket` (from `System.Net.WebSockets`), `JsonDocument` / `JsonSerializer` (from `System.Text.Json`), and `Channel<T>` (from `System.Threading.Channels`) all come from the BCL — no PackageReference needed.
|
||||
|
||||
- [ ] **Step 3: Commit the scaffold**
|
||||
|
||||
```bash
|
||||
git add tests/04a-realtime-oneshot-cs/Probe.csproj
|
||||
git commit -m "Test 4a scaffold: csproj for tests/04a-realtime-oneshot-cs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Write `Program.cs` in full
|
||||
|
||||
**Files:**
|
||||
- Create: `tests/04a-realtime-oneshot-cs/Program.cs`
|
||||
|
||||
The whole file is shown below. It is self-contained — no helper files, no copied modules. The structure follows Test 3's `Program.cs` shape (top-level statements; helpers, classes, and `static class Libc` at the bottom of the file) with these notable differences:
|
||||
|
||||
- No onnxruntime, no `WakewordModel`, no models directory.
|
||||
- A `ClientWebSocket` + two background `Task.Run` loops (ReceiveLoop, UpstreamSendLoop) replace the consumer-thread state machine.
|
||||
- The input stream callback's job is to push mic frames to a `Channel<short[]>` (not to call `Predict`).
|
||||
- The output stream callback drains a different `Channel<short[]>`, zero-fills underruns, and signals completion via a `ManualResetEventSlim` (replacing Test 3's per-cycle `PlaybackDoneFlag`).
|
||||
- A `SharedState` class holds the three `volatile bool` flags (`MicArmed`, `StopSending`, `NoMoreDeltas`) plus the output callback's per-call drain state (`CurrentChunk`, `ChunkOffset`).
|
||||
- `LoadApiKey()` reads `~/.openai_key` with mode-600 validation.
|
||||
|
||||
- [ ] **Step 1: Write `Program.cs` in full**
|
||||
|
||||
Create `tests/04a-realtime-oneshot-cs/Program.cs` with exactly this content:
|
||||
|
||||
```csharp
|
||||
using System.Net.WebSockets;
|
||||
using System.Runtime.InteropServices;
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using System.Threading.Channels;
|
||||
using PortAudioSharp;
|
||||
using Stream = PortAudioSharp.Stream;
|
||||
|
||||
// .NET-side mirror for any readers that go through Environment.GetEnvironmentVariable,
|
||||
// AND libc setenv so PortAudio (which uses getenv()) sees the value.
|
||||
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
|
||||
Libc.setenv("PA_ALSA_PLUGHW", "1", 1);
|
||||
|
||||
string apiKey = LoadApiKey();
|
||||
|
||||
PortAudio.Initialize();
|
||||
|
||||
using var cts = new CancellationTokenSource();
|
||||
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
|
||||
|
||||
var upstreamChannel = Channel.CreateBounded<short[]>(new BoundedChannelOptions(C.UpstreamChannelCapacity)
|
||||
{
|
||||
FullMode = BoundedChannelFullMode.Wait,
|
||||
SingleWriter = true,
|
||||
SingleReader = true,
|
||||
});
|
||||
var downstreamChannel = Channel.CreateBounded<short[]>(new BoundedChannelOptions(C.DownstreamChannelCapacity)
|
||||
{
|
||||
FullMode = BoundedChannelFullMode.Wait,
|
||||
SingleWriter = false,
|
||||
SingleReader = true,
|
||||
});
|
||||
|
||||
var doneFlag = new ManualResetEventSlim(false);
|
||||
var state = new SharedState();
|
||||
|
||||
var ws = new ClientWebSocket();
|
||||
ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
|
||||
ws.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
|
||||
|
||||
Stream? inputStream = null;
|
||||
Stream? outputStream = null;
|
||||
Task? receiveTask = null;
|
||||
Task? sendTask = null;
|
||||
|
||||
try
|
||||
{
|
||||
var uri = new Uri($"wss://api.openai.com/v1/realtime?model={C.ModelName}");
|
||||
Console.WriteLine($"[ws] connecting to {uri}");
|
||||
try
|
||||
{
|
||||
await ws.ConnectAsync(uri, cts.Token);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] WS connect failed: {ex.Message}");
|
||||
if (ex.InnerException is HttpRequestException httpEx)
|
||||
Console.Error.WriteLine($"[error] inner: HTTP {(int?)httpEx.StatusCode}");
|
||||
Environment.ExitCode = 3;
|
||||
return;
|
||||
}
|
||||
Console.WriteLine("[ws] connected");
|
||||
|
||||
receiveTask = Task.Run(() => ReceiveLoop(ws, downstreamChannel, cts, state));
|
||||
|
||||
await SendSessionUpdate(ws, cts.Token);
|
||||
|
||||
int device = FindUsbDevice();
|
||||
Console.WriteLine($"Using device {device} ('{PortAudio.GetDeviceInfo(device).name}')");
|
||||
|
||||
inputStream = OpenInputStream(device, upstreamChannel, state);
|
||||
try { inputStream.Start(); }
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] input stream start failed: {ex.Message}");
|
||||
Environment.ExitCode = 5;
|
||||
return;
|
||||
}
|
||||
|
||||
outputStream = OpenOutputStream(device, downstreamChannel, state, doneFlag);
|
||||
try { outputStream.Start(); }
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] output stream start failed: {ex.Message}");
|
||||
Environment.ExitCode = 6;
|
||||
return;
|
||||
}
|
||||
|
||||
sendTask = Task.Run(() => UpstreamSendLoop(ws, upstreamChannel, cts, state));
|
||||
|
||||
// Startup beep — queued onto downstreamChannel before the receive loop produces any audio.
|
||||
short[] beep = MakeBeep(C.BeepHz, C.BeepSeconds, C.SampleRate);
|
||||
Console.WriteLine("[beep]");
|
||||
await downstreamChannel.Writer.WriteAsync(beep, cts.Token);
|
||||
|
||||
// Wait for beep to drain (200 ms beep + 100 ms margin) so it doesn't get captured
|
||||
// by the mic and shipped to OpenAI.
|
||||
Thread.Sleep((int)(C.BeepSeconds * 1000) + C.BeepDrainMs);
|
||||
state.MicArmed = true;
|
||||
Console.WriteLine("Speak now.");
|
||||
|
||||
try
|
||||
{
|
||||
doneFlag.Wait(cts.Token);
|
||||
}
|
||||
catch (OperationCanceledException) { /* Ctrl-C path */ }
|
||||
|
||||
Console.WriteLine("[ws] closing");
|
||||
}
|
||||
finally
|
||||
{
|
||||
// Cancel any background work first so receive/send loops exit cleanly.
|
||||
cts.Cancel();
|
||||
|
||||
if (ws.State == WebSocketState.Open || ws.State == WebSocketState.CloseReceived)
|
||||
{
|
||||
try
|
||||
{
|
||||
using var closeTimeout = new CancellationTokenSource(TimeSpan.FromSeconds(2));
|
||||
await ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "done", closeTimeout.Token);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[ws] close failed: {ex.Message}");
|
||||
}
|
||||
}
|
||||
|
||||
try { inputStream?.Stop(); } catch { }
|
||||
try { outputStream?.Stop(); } catch { }
|
||||
try { inputStream?.Dispose(); } catch { }
|
||||
try { outputStream?.Dispose(); } catch { }
|
||||
try { ws.Dispose(); } catch { }
|
||||
try { PortAudio.Terminate(); } catch { }
|
||||
|
||||
Console.WriteLine("bye");
|
||||
}
|
||||
|
||||
static string LoadApiKey()
|
||||
{
|
||||
string home = Environment.GetEnvironmentVariable("HOME") ?? "/root";
|
||||
string path = Path.Combine(home, ".openai_key");
|
||||
if (!File.Exists(path))
|
||||
{
|
||||
Console.Error.WriteLine($"[error] API key file not found at {path}");
|
||||
Environment.Exit(2);
|
||||
}
|
||||
|
||||
// Non-fatal mode-600 check.
|
||||
try
|
||||
{
|
||||
var mode = File.GetUnixFileMode(path);
|
||||
var leaky = mode & (UnixFileMode.GroupRead | UnixFileMode.GroupWrite | UnixFileMode.GroupExecute
|
||||
| UnixFileMode.OtherRead | UnixFileMode.OtherWrite | UnixFileMode.OtherExecute);
|
||||
if (leaky != UnixFileMode.None)
|
||||
Console.Error.WriteLine($"[warn] {path} mode includes group/other perms ({mode}); expected 600");
|
||||
}
|
||||
catch (PlatformNotSupportedException) { /* not Unix */ }
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[warn] could not check {path} mode: {ex.Message}");
|
||||
}
|
||||
|
||||
string key = File.ReadAllText(path).Trim();
|
||||
if (string.IsNullOrEmpty(key))
|
||||
{
|
||||
Console.Error.WriteLine($"[error] API key file empty");
|
||||
Environment.Exit(2);
|
||||
}
|
||||
if (!key.StartsWith("sk-"))
|
||||
{
|
||||
Console.Error.WriteLine($"[error] API key doesn't start with sk-");
|
||||
Environment.Exit(2);
|
||||
}
|
||||
return key;
|
||||
}
|
||||
|
||||
static int FindUsbDevice()
|
||||
{
|
||||
int count = PortAudio.DeviceCount;
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
if (info.name.ToLowerInvariant().Contains("usb") && info.maxInputChannels >= 1)
|
||||
return i;
|
||||
}
|
||||
Console.Error.WriteLine("USB audio device not found. Devices:");
|
||||
for (int i = 0; i < count; i++)
|
||||
{
|
||||
var info = PortAudio.GetDeviceInfo(i);
|
||||
Console.Error.WriteLine(
|
||||
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels}");
|
||||
}
|
||||
Environment.Exit(1);
|
||||
return -1; // unreachable
|
||||
}
|
||||
|
||||
static short[] MakeBeep(double freqHz, double durationS, int sampleRate)
|
||||
{
|
||||
int n = (int)(sampleRate * durationS);
|
||||
var buf = new short[n];
|
||||
for (int i = 0; i < n; i++)
|
||||
{
|
||||
double t = i / (double)sampleRate;
|
||||
double v = 0.3 * Math.Sin(2.0 * Math.PI * freqHz * t);
|
||||
buf[i] = (short)(v * short.MaxValue);
|
||||
}
|
||||
return buf;
|
||||
}
|
||||
|
||||
static async Task SendSessionUpdate(ClientWebSocket ws, CancellationToken ct)
|
||||
{
|
||||
var msg = new
|
||||
{
|
||||
type = "session.update",
|
||||
session = new
|
||||
{
|
||||
modalities = new[] { "audio", "text" },
|
||||
instructions = C.Instructions,
|
||||
voice = C.Voice,
|
||||
input_audio_format = "pcm16",
|
||||
output_audio_format = "pcm16",
|
||||
turn_detection = new
|
||||
{
|
||||
type = "server_vad",
|
||||
threshold = 0.5,
|
||||
prefix_padding_ms = 300,
|
||||
silence_duration_ms = 500,
|
||||
create_response = true,
|
||||
},
|
||||
},
|
||||
};
|
||||
byte[] bytes = JsonSerializer.SerializeToUtf8Bytes(msg);
|
||||
await ws.SendAsync(new ArraySegment<byte>(bytes), WebSocketMessageType.Text, endOfMessage: true, ct);
|
||||
}
|
||||
|
||||
static Stream OpenInputStream(int device, Channel<short[]> upstream, SharedState state)
|
||||
{
|
||||
var inParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = C.Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowInputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback callback = (IntPtr input, IntPtr _, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags status, IntPtr _3) =>
|
||||
{
|
||||
if (status.HasFlag(StreamCallbackFlags.InputOverflow))
|
||||
Console.Error.WriteLine("[status] input overflow");
|
||||
|
||||
if (!state.MicArmed || state.StopSending)
|
||||
return StreamCallbackResult.Continue;
|
||||
|
||||
if (frameCount != C.MicBlockFrames)
|
||||
{
|
||||
Console.Error.WriteLine($"[status] unexpected input frameCount={frameCount} (want {C.MicBlockFrames})");
|
||||
return StreamCallbackResult.Continue;
|
||||
}
|
||||
|
||||
var frame = new short[C.MicBlockFrames];
|
||||
unsafe
|
||||
{
|
||||
short* src = (short*)input.ToPointer();
|
||||
fixed (short* dst = &frame[0])
|
||||
Buffer.MemoryCopy(src, dst, C.MicBlockFrames * sizeof(short), C.MicBlockFrames * sizeof(short));
|
||||
}
|
||||
if (!upstream.Writer.TryWrite(frame))
|
||||
Console.Error.WriteLine("[status] upstream behind, dropping mic frame");
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
return new Stream(inParams, null, C.SampleRate, C.MicBlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
|
||||
}
|
||||
|
||||
static Stream OpenOutputStream(int device, Channel<short[]> downstream, SharedState state, ManualResetEventSlim doneFlag)
|
||||
{
|
||||
var outParams = new StreamParameters
|
||||
{
|
||||
device = device,
|
||||
channelCount = C.Channels,
|
||||
sampleFormat = SampleFormat.Int16,
|
||||
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
|
||||
hostApiSpecificStreamInfo = IntPtr.Zero,
|
||||
};
|
||||
|
||||
Stream.Callback callback = (IntPtr _, IntPtr output, uint frameCount,
|
||||
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
|
||||
{
|
||||
int fill = 0;
|
||||
|
||||
while (fill < frameCount)
|
||||
{
|
||||
// Pull next chunk if we have no working one.
|
||||
if (state.CurrentChunk == null)
|
||||
{
|
||||
if (!downstream.Reader.TryRead(out var nextChunk))
|
||||
break; // ring empty
|
||||
state.CurrentChunk = nextChunk;
|
||||
state.ChunkOffset = 0;
|
||||
}
|
||||
// Local non-null alias so the compiler doesn't warn on dereference inside `fixed`.
|
||||
short[] chunk = state.CurrentChunk!;
|
||||
int slack = (int)frameCount - fill;
|
||||
int remaining = chunk.Length - state.ChunkOffset;
|
||||
int take = Math.Min(remaining, slack);
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
fixed (short* src = &chunk[state.ChunkOffset])
|
||||
Buffer.MemoryCopy(src, dst + fill, take * sizeof(short), take * sizeof(short));
|
||||
}
|
||||
state.ChunkOffset += take;
|
||||
fill += take;
|
||||
if (state.ChunkOffset >= chunk.Length)
|
||||
state.CurrentChunk = null;
|
||||
}
|
||||
|
||||
unsafe
|
||||
{
|
||||
short* dst = (short*)output.ToPointer();
|
||||
for (int i = fill; i < (int)frameCount; i++) dst[i] = 0;
|
||||
}
|
||||
|
||||
// Completion = working chunk drained AND channel empty AND writer completed.
|
||||
if (state.CurrentChunk == null
|
||||
&& downstream.Reader.Count == 0
|
||||
&& downstream.Reader.Completion.IsCompleted)
|
||||
{
|
||||
doneFlag.Set();
|
||||
return StreamCallbackResult.Complete;
|
||||
}
|
||||
return StreamCallbackResult.Continue;
|
||||
};
|
||||
|
||||
return new Stream(null, outParams, C.SampleRate, C.OutBlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
|
||||
}
|
||||
|
||||
static async Task UpstreamSendLoop(ClientWebSocket ws, Channel<short[]> upstream, CancellationTokenSource cts, SharedState state)
|
||||
{
|
||||
try
|
||||
{
|
||||
await foreach (var frame in upstream.Reader.ReadAllAsync(cts.Token))
|
||||
{
|
||||
if (state.StopSending) continue;
|
||||
ReadOnlySpan<byte> bytes = MemoryMarshal.AsBytes(frame.AsSpan());
|
||||
string base64 = Convert.ToBase64String(bytes);
|
||||
var msg = new { type = "input_audio_buffer.append", audio = base64 };
|
||||
byte[] json = JsonSerializer.SerializeToUtf8Bytes(msg);
|
||||
await ws.SendAsync(new ArraySegment<byte>(json), WebSocketMessageType.Text, endOfMessage: true, cts.Token);
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { /* expected on Cancel */ }
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] upstream send: {ex.Message}");
|
||||
cts.Cancel();
|
||||
}
|
||||
}
|
||||
|
||||
static async Task ReceiveLoop(ClientWebSocket ws, Channel<short[]> downstream, CancellationTokenSource cts, SharedState state)
|
||||
{
|
||||
var transcript = new StringBuilder();
|
||||
var buffer = new byte[4096];
|
||||
bool firstDeltaSeen = false;
|
||||
DateTime speechStoppedTime = DateTime.MinValue;
|
||||
using var msgBuffer = new MemoryStream();
|
||||
|
||||
try
|
||||
{
|
||||
while (!cts.IsCancellationRequested)
|
||||
{
|
||||
msgBuffer.SetLength(0);
|
||||
WebSocketReceiveResult result;
|
||||
do
|
||||
{
|
||||
result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), cts.Token);
|
||||
if (result.MessageType == WebSocketMessageType.Close) return;
|
||||
msgBuffer.Write(buffer, 0, result.Count);
|
||||
} while (!result.EndOfMessage);
|
||||
|
||||
msgBuffer.Position = 0;
|
||||
using var doc = JsonDocument.Parse(msgBuffer);
|
||||
string type = doc.RootElement.GetProperty("type").GetString() ?? "";
|
||||
|
||||
switch (type)
|
||||
{
|
||||
case "session.created":
|
||||
{
|
||||
string? sid = doc.RootElement.TryGetProperty("session", out var sess)
|
||||
&& sess.TryGetProperty("id", out var idEl) ? idEl.GetString() : null;
|
||||
Console.WriteLine($"[ws] session.created {sid}");
|
||||
break;
|
||||
}
|
||||
case "session.updated":
|
||||
Console.WriteLine("[ws] session.updated — session pinned");
|
||||
break;
|
||||
case "input_audio_buffer.speech_started":
|
||||
Console.WriteLine("[vad] speech started");
|
||||
break;
|
||||
case "input_audio_buffer.speech_stopped":
|
||||
Console.WriteLine("[vad] speech stopped");
|
||||
speechStoppedTime = DateTime.UtcNow;
|
||||
state.StopSending = true;
|
||||
break;
|
||||
case "input_audio_buffer.committed":
|
||||
Console.WriteLine("[ws] input_audio_buffer.committed");
|
||||
break;
|
||||
case "response.created":
|
||||
Console.WriteLine("[ws] response.created");
|
||||
break;
|
||||
case "response.audio_transcript.delta":
|
||||
if (doc.RootElement.TryGetProperty("delta", out var tEl))
|
||||
transcript.Append(tEl.GetString());
|
||||
break;
|
||||
case "response.audio.delta":
|
||||
{
|
||||
string b64 = doc.RootElement.GetProperty("delta").GetString() ?? "";
|
||||
byte[] audioBytes = Convert.FromBase64String(b64);
|
||||
short[] samples = new short[audioBytes.Length / 2];
|
||||
Buffer.BlockCopy(audioBytes, 0, samples, 0, audioBytes.Length);
|
||||
if (!firstDeltaSeen)
|
||||
{
|
||||
firstDeltaSeen = true;
|
||||
double elapsed = speechStoppedTime == DateTime.MinValue
|
||||
? -1
|
||||
: (DateTime.UtcNow - speechStoppedTime).TotalSeconds;
|
||||
Console.WriteLine($"[audio] first delta received (t = {elapsed:0.00} s since speech_stopped)");
|
||||
}
|
||||
await downstream.Writer.WriteAsync(samples, cts.Token);
|
||||
break;
|
||||
}
|
||||
case "response.audio.done":
|
||||
Console.WriteLine("[ws] response.audio.done");
|
||||
break;
|
||||
case "response.done":
|
||||
state.NoMoreDeltas = true;
|
||||
downstream.Writer.Complete();
|
||||
Console.WriteLine("[ws] response.done");
|
||||
Console.WriteLine($"[transcript] {transcript}");
|
||||
break;
|
||||
case "error":
|
||||
Console.Error.WriteLine($"[error] WS error event: {doc.RootElement.GetRawText()}");
|
||||
cts.Cancel();
|
||||
break;
|
||||
default:
|
||||
Console.Error.WriteLine($"[ws] ignored type={type}");
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (OperationCanceledException) { /* expected on Cancel */ }
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.Error.WriteLine($"[error] receive loop: {ex.Message}");
|
||||
cts.Cancel();
|
||||
}
|
||||
}
|
||||
|
||||
sealed class SharedState
|
||||
{
|
||||
public volatile bool MicArmed;
|
||||
public volatile bool StopSending;
|
||||
public volatile bool NoMoreDeltas;
|
||||
// Output-callback-thread-local; do NOT touch from anywhere else.
|
||||
public short[]? CurrentChunk;
|
||||
public int ChunkOffset;
|
||||
}
|
||||
|
||||
static class C
|
||||
{
|
||||
public const int SampleRate = 24_000;
|
||||
public const int Channels = 1;
|
||||
public const uint MicBlockFrames = 1920; // 40 ms @ 24 kHz
|
||||
public const uint OutBlockFrames = 1024; // ~43 ms @ 24 kHz
|
||||
public const string ModelName = "gpt-realtime";
|
||||
public const string Voice = "alloy";
|
||||
public const string Instructions = "Reply in one short sentence.";
|
||||
public const double BeepHz = 880.0;
|
||||
public const double BeepSeconds = 0.2;
|
||||
public const int BeepDrainMs = 100;
|
||||
public const int UpstreamChannelCapacity = 64;
|
||||
public const int DownstreamChannelCapacity = 64;
|
||||
}
|
||||
|
||||
static class Libc
|
||||
{
|
||||
[DllImport("libc", EntryPoint = "setenv")]
|
||||
public static extern int setenv(string name, string value, int overwrite);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Verify the project builds cleanly**
|
||||
|
||||
```bash
|
||||
dotnet build tests/04a-realtime-oneshot-cs -c Release
|
||||
```
|
||||
|
||||
Expected: `Build succeeded` with 0 errors. A handful of nullable-reference warnings on `state.CurrentChunk` access inside the `unsafe` block is fine if it appears; the runtime checks above ensure non-null.
|
||||
|
||||
If `<AllowUnsafeBlocks>` is missing from the csproj, the build will fail with `CS0227: Unsafe code may only appear if compiling with /unsafe`. Confirm Task 1's csproj has `<AllowUnsafeBlocks>true</AllowUnsafeBlocks>`.
|
||||
|
||||
If you see `CS0246: The type or namespace 'Stream' could not be found` or a clash with `System.IO.Stream`, confirm the `using Stream = PortAudioSharp.Stream;` line is at the top of `Program.cs`.
|
||||
|
||||
If you see `error CS8803: Top-level statements must precede namespace and type declarations` or `CS8805: Program using top-level statements must be an entry point`, the file ordering is wrong. The exact required order in `Program.cs` is:
|
||||
|
||||
1. `using` directives
|
||||
2. Top-level statements (the imperative body — `Environment.SetEnvironmentVariable`, `apiKey = LoadApiKey()`, `PortAudio.Initialize`, the `try {} finally {}` block)
|
||||
3. Top-level static method declarations (`LoadApiKey`, `FindUsbDevice`, `MakeBeep`, `SendSessionUpdate`, `OpenInputStream`, `OpenOutputStream`, `UpstreamSendLoop`, `ReceiveLoop`)
|
||||
4. Top-level type declarations (`sealed class SharedState`, `static class C`, `static class Libc`) — NOT wrapped in a `namespace` block.
|
||||
|
||||
The file above follows this order; do not reorder.
|
||||
|
||||
- [ ] **Step 3: Verify `dotnet publish` produces a runnable binary**
|
||||
|
||||
```bash
|
||||
dotnet publish tests/04a-realtime-oneshot-cs -c Release -r linux-arm64 --self-contained -o /tmp/probe-cs-4a-out
|
||||
ls /tmp/probe-cs-4a-out/Probe
|
||||
```
|
||||
|
||||
Expected:
|
||||
- `Probe` (executable) exists.
|
||||
- No `models/` directory needed.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add tests/04a-realtime-oneshot-cs/Program.cs
|
||||
git commit -m "Test 4a: WS plumbing + audio streams + server VAD round-trip"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Write `bin/probe-cs-4a` deploy script with API key precondition
|
||||
|
||||
**Files:**
|
||||
- Create: `bin/probe-cs-4a`
|
||||
|
||||
This is `bin/probe-cs-3` with three string swaps (`03-full-cycle-cs` → `04a-realtime-oneshot-cs`, `probe-cs-3` → `probe-cs-4a`, `/tmp/probe-cs-3-out` → `/tmp/probe-cs-4a-out`) plus ONE new precondition step: check that `~/.openai_key` exists on the Pi before publishing. This catches the silent "key missing" failure mode at the deploy step instead of 30 s into publish + scp.
|
||||
|
||||
- [ ] **Step 1: Write the deploy script**
|
||||
|
||||
Create `bin/probe-cs-4a` with exactly this content:
|
||||
|
||||
```bash
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
PI_HOST="${PI_HOST:-192.168.50.115}"
|
||||
PI_USER="${PI_USER:-pi}"
|
||||
PI_PASS="${PI_PASS:-assistant}"
|
||||
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-4a-out}"
|
||||
|
||||
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
cd "$repo_root"
|
||||
|
||||
echo ">> precondition: ~/.openai_key present on Pi"
|
||||
if ! sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'test -f ~/.openai_key && test -r ~/.openai_key'; then
|
||||
echo "ERROR: ~/.openai_key missing or unreadable on Pi."
|
||||
echo "Create it on the Pi with:"
|
||||
echo " sshpass -p '$PI_PASS' ssh $PI_USER@$PI_HOST \\"
|
||||
echo " 'umask 077 && printf \"%s\\n\" \"sk-PASTE-YOUR-KEY\" > ~/.openai_key'"
|
||||
exit 1
|
||||
fi
|
||||
echo " OK."
|
||||
|
||||
echo ">> dotnet publish (linux-arm64, self-contained)"
|
||||
dotnet publish tests/04a-realtime-oneshot-cs \
|
||||
-c Release -r linux-arm64 --self-contained \
|
||||
-o "$PUBLISH_DIR"
|
||||
|
||||
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-4a/ (wipe-and-replace)"
|
||||
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-4a && mkdir ~/probe-cs-4a'
|
||||
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
|
||||
"$PI_USER@$PI_HOST:~/probe-cs-4a/"
|
||||
|
||||
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
|
||||
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
|
||||
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-4a && chmod +x Probe && ./Probe'
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Make it executable**
|
||||
|
||||
```bash
|
||||
chmod +x bin/probe-cs-4a
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add bin/probe-cs-4a
|
||||
git commit -m "Test 4a: bin/probe-cs-4a deploy script with ~/.openai_key precondition"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: One-time Pi setup — place `~/.openai_key` on the Pi
|
||||
|
||||
**Files:** (none — this is a one-time remote setup task)
|
||||
|
||||
This task only runs once per Pi. If the file already exists from a previous deploy or another test, skip steps 1–2 and verify with step 3.
|
||||
|
||||
- [ ] **Step 1: Ask the user for an API key**
|
||||
|
||||
Before running this task, the user must have an OpenAI API key with Realtime API access (`sk-…`). If the user hasn't given it to you yet, ask:
|
||||
|
||||
> "I need your OpenAI API key (`sk-…`) to write it to `~/.openai_key` on the Pi. Paste it and I'll redact it from the shell history immediately. Do not paste it into any other context."
|
||||
|
||||
The key the user provides MUST be treated as secret and MUST NOT appear in commit messages, file contents anywhere in the repo, console echoes, or logs. The only place it goes is the file on the Pi.
|
||||
|
||||
- [ ] **Step 2: Write the key to the Pi**
|
||||
|
||||
Replace `sk-PASTE-YOUR-KEY-HERE` below with the actual key value. The `umask 077` ensures the file is created with mode 600 (rw for owner only). The `printf "%s\n"` appends exactly one newline.
|
||||
|
||||
```bash
|
||||
sshpass -p 'assistant' ssh -o StrictHostKeyChecking=accept-new \
|
||||
pi@192.168.50.115 \
|
||||
'umask 077 && printf "%s\n" "sk-PASTE-YOUR-KEY-HERE" > ~/.openai_key'
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify file exists and is mode 600**
|
||||
|
||||
```bash
|
||||
sshpass -p 'assistant' ssh pi@192.168.50.115 \
|
||||
'ls -l ~/.openai_key && wc -c ~/.openai_key'
|
||||
```
|
||||
|
||||
Expected output (mode `-rw-------`, owner `pi`, size > 50 bytes):
|
||||
|
||||
```
|
||||
-rw------- 1 pi pi <size> <date> /home/pi/.openai_key
|
||||
<size> /home/pi/.openai_key
|
||||
```
|
||||
|
||||
If mode is not `-rw-------`, fix with `chmod 600 ~/.openai_key`.
|
||||
|
||||
If size is suspiciously small (< 20 bytes), the key wasn't pasted correctly — re-do step 2.
|
||||
|
||||
(No commit — this step modifies state on the Pi, not the repo.)
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Hardware verification on the Pi
|
||||
|
||||
This task is the actual pass/fail gate. No automated tests; verification is the human ear + console output. If anything in this task fails, do NOT mark the implementation complete — investigate and either fix in code or update the spec to reflect what actually works.
|
||||
|
||||
**Files:** (none — this is a runtime verification task)
|
||||
|
||||
- [ ] **Step 1: Confirm the Pi is reachable**
|
||||
|
||||
Run from workstation:
|
||||
|
||||
```bash
|
||||
ping -c 2 192.168.50.115
|
||||
```
|
||||
|
||||
Expected: 2/2 replies. If unreachable, stop and ask the user.
|
||||
|
||||
- [ ] **Step 2: Deploy + run the probe**
|
||||
|
||||
Run from workstation:
|
||||
|
||||
```bash
|
||||
./bin/probe-cs-4a
|
||||
```
|
||||
|
||||
Expected console output (abridged, with elision):
|
||||
|
||||
```
|
||||
>> precondition: ~/.openai_key present on Pi
|
||||
OK.
|
||||
>> dotnet publish (linux-arm64, self-contained)
|
||||
... (build output) ...
|
||||
>> scp to pi@192.168.50.115:~/probe-cs-4a/ (wipe-and-replace)
|
||||
>> ssh + run on Pi (Ctrl-C from this terminal stops the probe)
|
||||
[ws] connecting to wss://api.openai.com/v1/realtime?model=gpt-realtime
|
||||
[ws] connected
|
||||
[ws] session.created sess_xxxxxxxxxxxx
|
||||
[ws] session.updated — session pinned
|
||||
Using device <N> ('USB Speaker Phone (LISTENAI ...)')
|
||||
[beep]
|
||||
Speak now.
|
||||
```
|
||||
|
||||
At this point the probe is waiting for you to speak. Say a single short question into the USB Speaker Phone — for example: **"What's the capital of France?"** — at normal volume from ~1 m.
|
||||
|
||||
Continue with verification in step 3.
|
||||
|
||||
If the publish step errors with a runtime crash invoking `Libc.setenv`, confirm Task 2's `[DllImport]` line for `setenv` is present.
|
||||
|
||||
If the Pi run errors with `Cannot open shared library libportaudio.so`, the self-contained publish didn't bundle the native libs — confirm `<SelfContained>true</SelfContained>` and `<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>` in Task 1's csproj.
|
||||
|
||||
If `[error] WS connect failed: ... HTTP 401` appears, the API key was rejected. Re-check Task 4 step 3 and confirm the key on the Pi starts with `sk-` and the user pasted the right value.
|
||||
|
||||
- [ ] **Step 3: Verify the full conversational round-trip**
|
||||
|
||||
After saying your question in step 2, observe:
|
||||
|
||||
1. `[vad] speech started` appears in the console as you begin speaking.
|
||||
2. `[vad] speech stopped` appears shortly after you stop speaking (within ~0.5–1 s, governed by `silence_duration_ms = 500` in session.update).
|
||||
3. `[ws] input_audio_buffer.committed` and `[ws] response.created` follow.
|
||||
4. `[audio] first delta received (t = X.XX s since speech_stopped)` appears with a small latency value (~1–2 s).
|
||||
5. The assistant's spoken reply comes out of the USB Speaker Phone speaker — clearly, intelligibly, recognisable as English speech.
|
||||
6. `[ws] response.audio.done` and `[ws] response.done` follow.
|
||||
7. `[transcript] <the assistant's reply, as text>` appears — should match what you heard.
|
||||
8. `[ws] closing` and `bye` appear; the probe exits.
|
||||
|
||||
**Pass criterion 1:** End-to-end round-trip works once. Probe exits 0.
|
||||
**Pass criterion 2:** Response is intelligible (no clipping, no dropouts longer than one ~43 ms callback).
|
||||
**Pass criterion 8:** Probe exits within ~500 ms of the last audio sample being heard.
|
||||
|
||||
- [ ] **Step 4: Check latency from the log line**
|
||||
|
||||
Look at the `[audio] first delta received (t = X.XX s since speech_stopped)` line printed in step 3.
|
||||
|
||||
**Pass criterion 3:** `X.XX` is under ~2 s. Slower is acceptable if logged here; over ~5 s is a fail.
|
||||
|
||||
- [ ] **Step 5: Scan for warnings**
|
||||
|
||||
Re-read the full console output. Confirm there are **no** lines of these forms during normal operation (between `[beep]` and `[ws] response.done`):
|
||||
|
||||
- `[status] upstream behind, dropping mic frame`
|
||||
- `[status] input overflow`
|
||||
- `[status] unexpected input frameCount=...`
|
||||
- `[error] WS ...`
|
||||
- `[ws] ignored type=...` (one or two of these are OK — the API may send events we don't handle; sustained occurrences during normal flow indicate a missing handler we should add)
|
||||
|
||||
**Pass criterion 4:** No `[status] upstream behind` warnings.
|
||||
**Pass criterion 5:** No `[error] WS …` events.
|
||||
**Pass criterion 6:** No PortAudio `InputOverflow` status flags.
|
||||
|
||||
- [ ] **Step 6: Test Ctrl-C clean exit (separate run)**
|
||||
|
||||
Restart the probe:
|
||||
|
||||
```bash
|
||||
./bin/probe-cs-4a
|
||||
```
|
||||
|
||||
When `Speak now.` appears, press Ctrl-C in the workstation terminal **without** speaking.
|
||||
|
||||
Expected:
|
||||
|
||||
```
|
||||
^C
|
||||
[ws] closing
|
||||
bye
|
||||
$
|
||||
```
|
||||
|
||||
Verify with `echo $?` — exit code 0.
|
||||
|
||||
**Pass criterion 7:** Ctrl-C produces a clean exit (no stack trace, terminal returns to prompt, exit code 0 when invoked during the steady listening phase).
|
||||
|
||||
- [ ] **Step 7: Update `findings.md` with the outcome**
|
||||
|
||||
Add a new section `## Test 4a outcome (2026-06-12 — OpenAI Realtime one-shot voice round-trip in C#)` at the bottom of `findings.md` summarising:
|
||||
|
||||
- Pass / fail per criterion (1–8).
|
||||
- Latency measured (the `X.XX` value from step 4).
|
||||
- Any new gotchas discovered during implementation or hardware verification — especially:
|
||||
- Any OpenAI Realtime API surface differences from the spec (event types not in our switch, session.update fields rejected, model name issues).
|
||||
- Any audio-format quirks (24 kHz playback dropouts, sample-rate-mismatch artifacts).
|
||||
- Any concurrency surprises (channel deadlocks, send-loop wedges).
|
||||
- Any errors in our LoadApiKey / mode-600 check.
|
||||
- Implications for 4b (what we still need to debug there vs. what 4a proved out).
|
||||
- Reference paths (spec + plan + code + deploy script).
|
||||
|
||||
Use the existing Test 1, 2, and 3 outcome sections in `findings.md` as the template.
|
||||
|
||||
- [ ] **Step 8: Commit the findings update**
|
||||
|
||||
```bash
|
||||
git add findings.md
|
||||
git commit -m "Findings: Test 4a OpenAI Realtime one-shot voice round-trip outcome"
|
||||
```
|
||||
|
||||
- [ ] **Step 9: Final status report**
|
||||
|
||||
Report to the user:
|
||||
|
||||
- Pass/fail per criterion (1 through 8 from spec).
|
||||
- Any criteria that failed and what was discovered.
|
||||
- The measured round-trip latency value.
|
||||
- A one-line summary of whether Test 4a unblocks Test 4b (i.e. whether the OpenAI Realtime surface + audio plumbing is proven, so 4b can focus on wakeword + loop + 16↔24 kHz coexistence).
|
||||
|
||||
---
|
||||
|
||||
## Self-review notes
|
||||
|
||||
- **Spec coverage:**
|
||||
- Goal (single-binary one-shot probe) → Tasks 1+2+3+5.
|
||||
- `~/.openai_key` precondition + file-on-Pi storage → Task 3 (script) + Task 4 (one-time setup) + Task 2 (`LoadApiKey` with mode-600 warning).
|
||||
- WS connect with `Authorization` + `OpenAI-Beta` headers → Task 2 (`ws.Options.SetRequestHeader` + `ConnectAsync`).
|
||||
- `session.update` payload with `server_vad` → Task 2 (`SendSessionUpdate` helper, exact JSON shape).
|
||||
- Mic input stream at 24 kHz with `_micArmed` gate → Task 2 (`OpenInputStream`, `state.MicArmed` check in callback).
|
||||
- UpstreamSendLoop (base64 + `input_audio_buffer.append`) → Task 2 (`UpstreamSendLoop`).
|
||||
- ReceiveLoop with `JsonDocument` switch on `type` → Task 2 (`ReceiveLoop`, complete handler table).
|
||||
- Output stream drain-or-zero-fill + `_noMoreDeltas` + `Reader.Completion.IsCompleted` completion → Task 2 (`OpenOutputStream` callback).
|
||||
- Startup beep through `downstreamChannel` → Task 2 (lines after `outputStream.Start()`).
|
||||
- `ManualResetEventSlim doneFlag` + clean shutdown via `finally` → Task 2 (try/finally block).
|
||||
- Ctrl-C → `Console.CancelKeyPress` → `cts.Cancel` → `OperationCanceledException` → finally → exit 0 → Tasks 2 + 5 step 6.
|
||||
- `bin/probe-cs-4a` with ssh -t + wipe-and-replace + key precondition → Task 3.
|
||||
- Pass criteria 1–8 → Task 5 steps 3, 4, 5, 6.
|
||||
- Findings update → Task 5 steps 7–8.
|
||||
- **Placeholder scan:** no TBD/TODO; every code step shows the actual code; every command shows expected output. The single user-substitution placeholder is `sk-PASTE-YOUR-KEY-HERE` in Task 4 step 2, which is explicitly called out as a replacement target.
|
||||
- **Type consistency:**
|
||||
- `SharedState` fields (`MicArmed`, `StopSending`, `NoMoreDeltas`, `CurrentChunk`, `ChunkOffset`) referenced consistently from `OpenInputStream`, `OpenOutputStream`, `UpstreamSendLoop`, `ReceiveLoop`, and the top-level statements.
|
||||
- `Channel<short[]>` used for both upstream and downstream; the `short[]` element type is consistent everywhere.
|
||||
- `ManualResetEventSlim doneFlag` passed by reference into `OpenOutputStream` so the callback can `Set()` it; main thread awaits it via `Wait(cts.Token)`.
|
||||
- `CancellationTokenSource cts` is the single source of cancellation across all three loops + the main thread.
|
||||
- **Scope:** the spec is a single subsystem (one probe). One plan is the right granularity.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user