Assistant/docs/superpowers/plans/2026-06-11-smart-assistant-device-hub-realtime-tools.md

# Smart Assistant — Device Hub + Realtime Relay + Tools Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Stand up the per-device WebSocket hub at `/device`, the OpenAI Realtime relay (one upstream WSS per conversation), the in-process C# tools registry (`get_current_time`, `end_session`, `set_volume`), and the `Conversations` / `Turns` persistence — enough that a paired client can `wake`, hold a multi-turn conversation against OpenAI, end via tool-call or idle timeout, and have every utterance + tool call written to SQLite. The Python client itself is **Plan 3** — this plan ships with a minimal `wscat`-like manual smoke script and integration tests against a scripted upstream double.

**Architecture:** A single persistent client WS per paired device, authenticated by the bearer device token. The handler (`DeviceHubEndpoint`) routes text envelopes (`hello`/`ping`/`wake`/`cancel`/`playback_done`/`tool_result`) and binary frames (PCM16 uplink). On `wake`, the hub creates a `RealtimeSession` that opens its own upstream WSS to OpenAI (`/v1/realtime?model=…`), drives a `session.update` with per-device config + tools, and pumps events both ways. The relay is layered behind `IRealtimeUpstream` so tests inject a scripted double — no real OpenAI calls in unit tests. Tools implement `ITool` and run in-process; `set_volume` routes back to the Pi via `IDeviceChannel.CallPiToolAsync(...)`. Idle timeout, `end_session` sequencing, and transcript persistence are all part of `RealtimeSession`.

**Tech Stack:** .NET 9 ASP.NET Core (WebSockets middleware, `System.Net.WebSockets.ClientWebSocket` for upstream), System.Text.Json with `JsonNamingPolicy.SnakeCaseLower`, EF Core 9 + SQLite (already in Plan 1), `System.Threading.Channels` for queues, xUnit + `Microsoft.AspNetCore.Mvc.Testing` for integration tests.

**Scope (in):**
- `Conversation` + `Turn` entities + migration.
- `ConversationLog` (write-only persistence helper).
- `ITool` + `ToolRegistry` + three built-ins: `get_current_time`, `end_session`, `set_volume`.
- `OpenAIKeyProvider`.
- `IRealtimeUpstream` abstraction + `ScriptedRealtimeUpstream` test double + real `OpenAIRealtimeUpstream`.
- `RealtimeSession` (open → session.update → pump → tools → idle/end → close, with transcript writes).
- `ActiveDevice` + `IDeviceChannel` + `DeviceRegistry`.
- `DeviceHubEndpoint` at `/device` with bearer-token auth, handshake, ping, wake, cancel, playback_done, tool_result.
- Full integration test (real WS in `WebApplicationFactory` + scripted upstream).
- Coolify deploy + smoke check on the live `/device` endpoint.

**Scope (out, later plans):**
- Python client implementation (Plan 3 — replaces the placeholders shipped in Plan 1 with real audio loop, wakeword, state machine, playback, pairing CLI).
- Admin UI / `DeviceConfig` edit endpoints / `config_updated` push from a UI save (Plan 4 — the relay still consumes `DeviceConfig` from the DB on each `wake`, so changing rows directly works today).
- `play_audio(kind=notification)` (backend pushing audio outside a session) — envelope types are defined here but no production flow uses them yet.
- Long-term memory, audio recording persistence, barge-in (all non-goals per spec).
- React UI (Plan 4).

**Reference spec:** `docs/superpowers/specs/2026-06-11-smart-assistant-design.md`
**Previous plan:** `docs/superpowers/plans/2026-06-11-smart-assistant-backend-foundation.md`

---

## File structure

This plan creates the following files. Each file owns one concern; cross-references are noted under each task.

```
Assistant/
├── backend/
│   ├── Conversations/
│   │   ├── Conversation.cs                  # entity
│   │   ├── Turn.cs                          # entity
│   │   └── ConversationLog.cs               # scoped writer
│   ├── Tools/
│   │   ├── ITool.cs                         # interface, DeviceContext, ToolInvocation, ToolResult
│   │   ├── IDeviceChannel.cs                # surface tools see to call Pi-side tools
│   │   ├── ToolRegistry.cs                  # singleton with all ITool impls
│   │   ├── GetCurrentTimeTool.cs            # server-side, no params
│   │   ├── EndSessionTool.cs                # marks RealtimeSession to close after response.done
│   │   └── SetVolumeTool.cs                 # routes to Pi via IDeviceChannel
│   ├── Realtime/
│   │   ├── OpenAIKeyProvider.cs             # reads OPENAI_API_KEY env
│   │   ├── RealtimeSettings.cs              # per-conversation config (model/voice/prompt/tools/timeout)
│   │   ├── RealtimeEvents.cs                # records for upstream JSON envelopes (session.update etc.)
│   │   ├── IRealtimeUpstream.cs             # abstraction: send JSON, receive JSON, close
│   │   ├── OpenAIRealtimeUpstream.cs        # real ClientWebSocket implementation
│   │   ├── RealtimeSession.cs               # owns one conversation; pumps both ways
│   │   └── RealtimeSessionFactory.cs        # creates RealtimeSession with DI'd deps
│   └── DeviceHub/
│       ├── HubEnvelopes.cs                  # records for client-side wire types
│       ├── ActiveDevice.cs                  # per-WS state; implements IDeviceChannel; outbound writer
│       ├── DeviceRegistry.cs                # singleton map deviceId→ActiveDevice
│       ├── DeviceAuth.cs                    # bearer-token → device lookup helper
│       └── DeviceHubEndpoint.cs             # /device WS handler
├── backend.tests/
│   ├── ConversationLogTests.cs
│   ├── ToolRegistryTests.cs
│   ├── GetCurrentTimeToolTests.cs
│   ├── EndSessionToolTests.cs
│   ├── SetVolumeToolTests.cs
│   ├── ScriptedRealtimeUpstream.cs          # test double
│   ├── FakeDeviceChannel.cs                 # test double
│   ├── RealtimeSessionLifecycleTests.cs
│   ├── RealtimeSessionAudioTests.cs
│   ├── RealtimeSessionTranscriptTests.cs
│   ├── RealtimeSessionToolsTests.cs
│   ├── RealtimeSessionEndSessionTests.cs
│   ├── RealtimeSessionIdleTimeoutTests.cs
│   ├── DeviceRegistryTests.cs
│   ├── DeviceHubHandshakeTests.cs
│   └── DeviceHubWakeFlowTests.cs
```

---

## Task 1: Conversation + Turn entities + migration

**Files:**
- Create: `backend/Conversations/Conversation.cs`
- Create: `backend/Conversations/Turn.cs`
- Modify: `backend/Data/AppDbContext.cs`

- [ ] **Step 1: Define `Conversation`**

Create `backend/Conversations/Conversation.cs`:

```csharp
namespace backend.Conversations;

public enum EndReason
{
    Unknown = 0,
    Tool = 1,
    Idle = 2,
    Error = 3,
}

public class Conversation
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid DeviceId { get; set; }
    public DateTimeOffset StartedAt { get; set; }
    public DateTimeOffset? EndedAt { get; set; }
    public EndReason EndReason { get; set; } = EndReason.Unknown;
    public List<Turn> Turns { get; set; } = new();
}
```

- [ ] **Step 2: Define `Turn`**

Create `backend/Conversations/Turn.cs`:

```csharp
namespace backend.Conversations;

public enum TurnRole
{
    User = 0,
    Assistant = 1,
    Tool = 2,
}

public class Turn
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public Guid ConversationId { get; set; }
    public int Index { get; set; }
    public TurnRole Role { get; set; }
    public string? Text { get; set; }
    public string? ToolName { get; set; }
    public string? ToolArgsJson { get; set; }
    public string? ToolResultJson { get; set; }
    public DateTimeOffset CreatedAt { get; set; }
}
```

- [ ] **Step 3: Register entities in `AppDbContext`**

In `backend/Data/AppDbContext.cs`, add `using backend.Conversations;` at the top, then add the DbSets and the mapping in `OnModelCreating`:

```csharp
public DbSet<Conversation> Conversations => Set<Conversation>();
public DbSet<Turn> Turns => Set<Turn>();
```

Inside `OnModelCreating` (after the existing mappings, before the closing brace):

```csharp
b.Entity<Conversation>().HasIndex(c => c.DeviceId);
b.Entity<Conversation>()
    .HasMany(c => c.Turns)
    .WithOne()
    .HasForeignKey(t => t.ConversationId)
    .OnDelete(DeleteBehavior.Cascade);
b.Entity<Turn>().HasIndex(t => new { t.ConversationId, t.Index }).IsUnique();
```

- [ ] **Step 4: Generate the migration**

```bash
dotnet ef migrations add AddConversationsAndTurns --project backend --startup-project backend
```

Expected: a new file under `backend/Migrations/` ending in `_AddConversationsAndTurns.cs` that creates `Conversations` and `Turns` tables.

- [ ] **Step 5: Verify build + previous tests still pass**

Run: `dotnet test`
Expected: every Plan 1 test still passes; no new failures.

- [ ] **Step 6: Commit**

```bash
git add backend/Conversations backend/Data/AppDbContext.cs backend/Migrations
git commit -m "Add Conversation + Turn entities and migration"
```

---

## Task 2: ConversationLog (scoped persistence helper)

**Files:**
- Create: `backend/Conversations/ConversationLog.cs`
- Create: `backend.tests/ConversationLogTests.cs`
- Modify: `backend/Program.cs`

- [ ] **Step 1: Write the failing test**

Create `backend.tests/ConversationLogTests.cs`:

```csharp
using backend.Conversations;
using backend.Data;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class ConversationLogTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    private async Task<(AppDbContext db, ConversationLog log, Guid deviceId)> ArrangeAsync()
    {
        // Use the factory's DI container so we hit the same SQLite file.
        var scope = _factory.Services.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var deviceId = Guid.NewGuid();
        return (db, log, deviceId);
    }

    [Fact]
    public async Task Start_creates_conversation_row()
    {
        var (db, log, deviceId) = await ArrangeAsync();

        var convo = await log.StartAsync(deviceId, default);

        var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
        row.DeviceId.Should().Be(deviceId);
        row.EndedAt.Should().BeNull();
        row.EndReason.Should().Be(EndReason.Unknown);
    }

    [Fact]
    public async Task Turns_are_persisted_with_monotonic_index_per_conversation()
    {
        var (db, log, deviceId) = await ArrangeAsync();
        var convo = await log.StartAsync(deviceId, default);

        await log.AppendUserAsync(convo.Id, "hi", default);
        await log.AppendAssistantAsync(convo.Id, "hello there", default);
        await log.AppendToolAsync(convo.Id, "get_current_time", "{}", "{\"now\":\"2026-06-11T00:00:00Z\"}", default);

        var turns = await db.Turns
            .Where(t => t.ConversationId == convo.Id)
            .OrderBy(t => t.Index)
            .ToListAsync();

        turns.Should().HaveCount(3);
        turns.Select(t => t.Index).Should().Equal(0, 1, 2);
        turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant, TurnRole.Tool);
        turns[0].Text.Should().Be("hi");
        turns[1].Text.Should().Be("hello there");
        turns[2].ToolName.Should().Be("get_current_time");
    }

    [Fact]
    public async Task End_sets_ended_at_and_reason()
    {
        var (db, log, deviceId) = await ArrangeAsync();
        var convo = await log.StartAsync(deviceId, default);

        await log.EndAsync(convo.Id, EndReason.Idle, default);

        var row = await db.Conversations.FirstAsync(c => c.Id == convo.Id);
        row.EndedAt.Should().NotBeNull();
        row.EndReason.Should().Be(EndReason.Idle);
    }
}
```

- [ ] **Step 2: Run the test to see it fail**

Run: `dotnet test --filter ConversationLogTests`
Expected: build error (`ConversationLog` not defined).

- [ ] **Step 3: Implement the writer**

Create `backend/Conversations/ConversationLog.cs`:

```csharp
using backend.Data;
using Microsoft.EntityFrameworkCore;

namespace backend.Conversations;

public class ConversationLog(AppDbContext db)
{
    public async Task<Conversation> StartAsync(Guid deviceId, CancellationToken ct)
    {
        var c = new Conversation
        {
            DeviceId = deviceId,
            StartedAt = DateTimeOffset.UtcNow,
        };
        db.Conversations.Add(c);
        await db.SaveChangesAsync(ct);
        return c;
    }

    public Task AppendUserAsync(Guid conversationId, string text, CancellationToken ct)
        => AppendAsync(conversationId, TurnRole.User, text: text, ct: ct);

    public Task AppendAssistantAsync(Guid conversationId, string text, CancellationToken ct)
        => AppendAsync(conversationId, TurnRole.Assistant, text: text, ct: ct);

    public Task AppendToolAsync(
        Guid conversationId, string toolName, string argsJson, string resultJson, CancellationToken ct)
        => AppendAsync(conversationId, TurnRole.Tool,
            toolName: toolName, argsJson: argsJson, resultJson: resultJson, ct: ct);

    public async Task EndAsync(Guid conversationId, EndReason reason, CancellationToken ct)
    {
        var row = await db.Conversations.FirstAsync(c => c.Id == conversationId, ct);
        row.EndedAt = DateTimeOffset.UtcNow;
        row.EndReason = reason;
        await db.SaveChangesAsync(ct);
    }

    private async Task AppendAsync(
        Guid conversationId, TurnRole role,
        string? text = null, string? toolName = null,
        string? argsJson = null, string? resultJson = null,
        CancellationToken ct = default)
    {
        var next = await db.Turns
            .Where(t => t.ConversationId == conversationId)
            .Select(t => (int?)t.Index)
            .MaxAsync(ct) ?? -1;

        db.Turns.Add(new Turn
        {
            ConversationId = conversationId,
            Index = next + 1,
            Role = role,
            Text = text,
            ToolName = toolName,
            ToolArgsJson = argsJson,
            ToolResultJson = resultJson,
            CreatedAt = DateTimeOffset.UtcNow,
        });
        await db.SaveChangesAsync(ct);
    }
}
```

- [ ] **Step 4: Register in DI**

In `backend/Program.cs`, in the existing service-registration block (above `var app = builder.Build();`), add:

```csharp
builder.Services.AddScoped<backend.Conversations.ConversationLog>();
```

- [ ] **Step 5: Run the tests**

Run: `dotnet test --filter ConversationLogTests`
Expected: `Passed: 3`.

- [ ] **Step 6: Commit**

```bash
git add backend/Conversations/ConversationLog.cs backend.tests/ConversationLogTests.cs backend/Program.cs
git commit -m "Add ConversationLog (start/append/end with monotonic per-convo index)"
```

---

## Task 3: ITool, IDeviceChannel, ToolRegistry, GetCurrentTimeTool

**Files:**
- Create: `backend/Tools/ITool.cs`
- Create: `backend/Tools/IDeviceChannel.cs`
- Create: `backend/Tools/ToolRegistry.cs`
- Create: `backend/Tools/GetCurrentTimeTool.cs`
- Create: `backend.tests/ToolRegistryTests.cs`
- Create: `backend.tests/GetCurrentTimeToolTests.cs`

- [ ] **Step 1: Write the failing tests**

Create `backend.tests/ToolRegistryTests.cs`:

```csharp
using backend.Tools;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class ToolRegistryTests
{
    private readonly ITool[] _all = new ITool[]
    {
        new GetCurrentTimeTool(),
    };

    [Fact]
    public void Get_returns_tool_by_exact_name()
    {
        var reg = new ToolRegistry(_all);
        reg.Get("get_current_time").Should().NotBeNull();
        reg.Get("does_not_exist").Should().BeNull();
    }

    [Fact]
    public void Filter_returns_only_enabled_tools_preserving_order()
    {
        var reg = new ToolRegistry(_all);
        var enabled = new HashSet<string> { "get_current_time", "ghost" };
        reg.EnabledFor(enabled).Select(t => t.Name).Should().Equal("get_current_time");
    }
}
```

Create `backend.tests/GetCurrentTimeToolTests.cs`:

```csharp
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class GetCurrentTimeToolTests
{
    [Fact]
    public async Task Returns_now_field_with_iso_utc_timestamp()
    {
        var tool = new GetCurrentTimeTool();
        var args = JsonDocument.Parse("{}").RootElement;
        var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());

        var res = await tool.ExecuteAsync(new ToolInvocation("call_1", args), ctx, default);

        res.Output.TryGetProperty("now", out var now).Should().BeTrue();
        now.GetString().Should().MatchRegex(@"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}");
        now.GetString()!.Should().EndWith("Z");
    }

    private class NoopChannel : IDeviceChannel
    {
        public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
            => throw new NotSupportedException();
    }
}
```

- [ ] **Step 2: Run to see them fail**

Run: `dotnet test --filter "ToolRegistryTests|GetCurrentTimeToolTests"`
Expected: build error (`ITool`, `ToolRegistry`, `GetCurrentTimeTool`, `IDeviceChannel` not defined).

- [ ] **Step 3: Define the tool interface**

Create `backend/Tools/ITool.cs`:

```csharp
using System.Text.Json;

namespace backend.Tools;

public record DeviceContext(Guid DeviceId, Guid ConversationId, IDeviceChannel Channel);
public record ToolInvocation(string CallId, JsonElement Arguments);
public record ToolResult(JsonElement Output);

public interface ITool
{
    string Name { get; }
    string Description { get; }
    JsonElement ParameterSchema { get; }
    bool RunsDuringResponse { get; } // false → executes at response.done; end_session uses this
    Task<ToolResult> ExecuteAsync(
        ToolInvocation invocation,
        DeviceContext device,
        CancellationToken ct);
}
```

- [ ] **Step 4: Define IDeviceChannel**

Create `backend/Tools/IDeviceChannel.cs`:

```csharp
using System.Text.Json;

namespace backend.Tools;

public interface IDeviceChannel
{
    Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct);
}
```

- [ ] **Step 5: Define ToolRegistry**

Create `backend/Tools/ToolRegistry.cs`:

```csharp
namespace backend.Tools;

public class ToolRegistry
{
    private readonly Dictionary<string, ITool> _byName;
    private readonly List<ITool> _all;

    public ToolRegistry(IEnumerable<ITool> tools)
    {
        _all = tools.ToList();
        _byName = _all.ToDictionary(t => t.Name);
    }

    public IReadOnlyList<ITool> All => _all;

    public ITool? Get(string name) => _byName.GetValueOrDefault(name);

    public IEnumerable<ITool> EnabledFor(IReadOnlySet<string> enabled)
        => _all.Where(t => enabled.Contains(t.Name));
}
```

- [ ] **Step 6: Implement GetCurrentTimeTool**

Create `backend/Tools/GetCurrentTimeTool.cs`:

```csharp
using System.Text.Json;

namespace backend.Tools;

public class GetCurrentTimeTool : ITool
{
    public string Name => "get_current_time";
    public string Description => "Returns the current UTC time as an ISO-8601 string.";
    public bool RunsDuringResponse => false;

    public JsonElement ParameterSchema { get; } =
        JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;

    public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
    {
        var now = DateTimeOffset.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ");
        var output = JsonDocument.Parse($$"""{"now":"{{now}}"}""").RootElement;
        return Task.FromResult(new ToolResult(output));
    }
}
```

- [ ] **Step 7: Run the tests**

Run: `dotnet test --filter "ToolRegistryTests|GetCurrentTimeToolTests"`
Expected: `Passed: 3`.

- [ ] **Step 8: Commit**

```bash
git add backend/Tools/ITool.cs backend/Tools/IDeviceChannel.cs backend/Tools/ToolRegistry.cs backend/Tools/GetCurrentTimeTool.cs backend.tests/ToolRegistryTests.cs backend.tests/GetCurrentTimeToolTests.cs
git commit -m "Add ITool + ToolRegistry + GetCurrentTimeTool"
```

---

## Task 4: EndSessionTool

**Files:**
- Create: `backend/Tools/EndSessionTool.cs`
- Create: `backend.tests/EndSessionToolTests.cs`

`end_session` has zero parameters and zero side effects. Its return value is `{"ok":true}` for cleanliness; the actual "close" behaviour is wired in `RealtimeSession` (Task 12), which special-cases this tool by name. The tool itself is simple: return ok. We mark `RunsDuringResponse = false` because the relay must wait for `response.done` before acting on it.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/EndSessionToolTests.cs`:

```csharp
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class EndSessionToolTests
{
    [Fact]
    public async Task Returns_ok_true()
    {
        var tool = new EndSessionTool();
        var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), new NoopChannel());

        var res = await tool.ExecuteAsync(
            new ToolInvocation("call_1", JsonDocument.Parse("{}").RootElement), ctx, default);

        res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
    }

    [Fact]
    public void Name_and_runs_during_response_match_spec()
    {
        var tool = new EndSessionTool();
        tool.Name.Should().Be("end_session");
        tool.RunsDuringResponse.Should().BeFalse();
    }

    private class NoopChannel : IDeviceChannel
    {
        public Task<JsonElement> CallPiToolAsync(string n, JsonElement a, CancellationToken ct)
            => throw new NotSupportedException();
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter EndSessionToolTests`
Expected: build error (`EndSessionTool` not defined).

- [ ] **Step 3: Implement the tool**

Create `backend/Tools/EndSessionTool.cs`:

```csharp
using System.Text.Json;

namespace backend.Tools;

public class EndSessionTool : ITool
{
    public string Name => "end_session";
    public string Description =>
        "Call this when the user has said goodbye or otherwise indicates the conversation is over.";
    public bool RunsDuringResponse => false;

    public JsonElement ParameterSchema { get; } =
        JsonDocument.Parse("""{"type":"object","properties":{},"required":[]}""").RootElement;

    public Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
    {
        var output = JsonDocument.Parse("""{"ok":true}""").RootElement;
        return Task.FromResult(new ToolResult(output));
    }
}
```

- [ ] **Step 4: Run the tests**

Run: `dotnet test --filter EndSessionToolTests`
Expected: `Passed: 2`.

- [ ] **Step 5: Commit**

```bash
git add backend/Tools/EndSessionTool.cs backend.tests/EndSessionToolTests.cs
git commit -m "Add EndSessionTool (no params, returns ok; relay closes on this tool name)"
```

---

## Task 5: SetVolumeTool

**Files:**
- Create: `backend/Tools/SetVolumeTool.cs`
- Create: `backend.tests/SetVolumeToolTests.cs`

`set_volume` runs server-side but immediately delegates to the Pi via `IDeviceChannel.CallPiToolAsync`. The level is validated (0..100); out-of-range returns `{"ok":false,"error":"…"}` without ever touching the device.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/SetVolumeToolTests.cs`:

```csharp
using System.Text.Json;
using backend.Tools;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class SetVolumeToolTests
{
    [Fact]
    public async Task Forwards_call_to_pi_and_returns_its_result()
    {
        var fake = new RecordingChannel(
            JsonDocument.Parse("""{"ok":true}""").RootElement);
        var tool = new SetVolumeTool();
        var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);

        var args = JsonDocument.Parse("""{"level":42}""").RootElement;
        var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);

        fake.LastName.Should().Be("set_volume");
        fake.LastArgs.GetProperty("level").GetInt32().Should().Be(42);
        res.Output.GetProperty("ok").GetBoolean().Should().BeTrue();
    }

    [Fact]
    public async Task Rejects_out_of_range_level_without_calling_pi()
    {
        var fake = new RecordingChannel(JsonDocument.Parse("""{"ok":true}""").RootElement);
        var tool = new SetVolumeTool();
        var ctx = new DeviceContext(Guid.NewGuid(), Guid.NewGuid(), fake);

        var args = JsonDocument.Parse("""{"level":150}""").RootElement;
        var res = await tool.ExecuteAsync(new ToolInvocation("c1", args), ctx, default);

        fake.LastName.Should().BeNull();
        res.Output.GetProperty("ok").GetBoolean().Should().BeFalse();
        res.Output.GetProperty("error").GetString().Should().Contain("0..100");
    }

    private class RecordingChannel(JsonElement reply) : IDeviceChannel
    {
        public string? LastName { get; private set; }
        public JsonElement LastArgs { get; private set; }
        public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
        {
            LastName = name;
            LastArgs = args;
            return Task.FromResult(reply);
        }
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter SetVolumeToolTests`
Expected: build error (`SetVolumeTool` not defined).

- [ ] **Step 3: Implement the tool**

Create `backend/Tools/SetVolumeTool.cs`:

```csharp
using System.Text.Json;

namespace backend.Tools;

public class SetVolumeTool : ITool
{
    public string Name => "set_volume";
    public string Description =>
        "Sets the assistant's audio output volume on the device. Level is 0..100.";
    public bool RunsDuringResponse => true;

    public JsonElement ParameterSchema { get; } = JsonDocument.Parse(
        """
        {
          "type":"object",
          "properties":{"level":{"type":"integer","minimum":0,"maximum":100}},
          "required":["level"]
        }
        """).RootElement;

    public async Task<ToolResult> ExecuteAsync(ToolInvocation invocation, DeviceContext device, CancellationToken ct)
    {
        if (!invocation.Arguments.TryGetProperty("level", out var lvlEl)
            || !lvlEl.TryGetInt32(out var level)
            || level < 0 || level > 100)
        {
            var err = JsonDocument.Parse("""{"ok":false,"error":"level must be an integer in 0..100"}""").RootElement;
            return new ToolResult(err);
        }

        var piReply = await device.Channel.CallPiToolAsync(Name, invocation.Arguments, ct);
        return new ToolResult(piReply);
    }
}
```

- [ ] **Step 4: Run the tests**

Run: `dotnet test --filter SetVolumeToolTests`
Expected: `Passed: 2`.

- [ ] **Step 5: Commit**

```bash
git add backend/Tools/SetVolumeTool.cs backend.tests/SetVolumeToolTests.cs
git commit -m "Add SetVolumeTool (validates 0..100, delegates to Pi via IDeviceChannel)"
```

---

## Task 6: OpenAIKeyProvider + RealtimeSettings

**Files:**
- Create: `backend/Realtime/OpenAIKeyProvider.cs`
- Create: `backend/Realtime/RealtimeSettings.cs`

The provider exposes the OpenAI API key sourced from `OPENAI_API_KEY` env (set as a Coolify secret per spec). `RealtimeSettings` is the per-conversation snapshot of `DeviceConfig` that drives `session.update`.

- [ ] **Step 1: Implement OpenAIKeyProvider**

Create `backend/Realtime/OpenAIKeyProvider.cs`:

```csharp
namespace backend.Realtime;

public class OpenAIKeyProvider
{
    private readonly string? _key;

    public OpenAIKeyProvider(IConfiguration cfg)
    {
        _key = cfg["OPENAI_API_KEY"] ?? cfg["OpenAI:ApiKey"];
    }

    public bool HasKey => !string.IsNullOrWhiteSpace(_key);

    public string GetRequired()
    {
        if (string.IsNullOrWhiteSpace(_key))
            throw new InvalidOperationException(
                "OPENAI_API_KEY is not configured. Set the env var on the Coolify app.");
        return _key!;
    }
}
```

- [ ] **Step 2: Implement RealtimeSettings**

Create `backend/Realtime/RealtimeSettings.cs`:

```csharp
namespace backend.Realtime;

public record RealtimeSettings(
    string Model,
    string Voice,
    string SystemPrompt,
    int IdleTimeoutSeconds,
    IReadOnlySet<string> EnabledTools);
```

- [ ] **Step 3: Register the provider in DI**

In `backend/Program.cs`, add to the existing service-registration block:

```csharp
builder.Services.AddSingleton<backend.Realtime.OpenAIKeyProvider>();
```

- [ ] **Step 4: Verify build + tests still pass**

Run: `dotnet test`
Expected: every prior test passes; nothing new broken.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/OpenAIKeyProvider.cs backend/Realtime/RealtimeSettings.cs backend/Program.cs
git commit -m "Add OpenAIKeyProvider + RealtimeSettings record"
```

---

## Task 7: IRealtimeUpstream interface + envelope helpers

**Files:**
- Create: `backend/Realtime/RealtimeEvents.cs`
- Create: `backend/Realtime/IRealtimeUpstream.cs`

The relay only sends a tiny handful of upstream event shapes (`session.update`, `input_audio_buffer.append`, `conversation.item.create` with `function_call_output`, `response.create`) and reads about a dozen back. Define those as plain `JsonNode` helpers, not strict types — OpenAI evolves the schema and we never need to model the whole event.

- [ ] **Step 1: Define event helpers**

Create `backend/Realtime/RealtimeEvents.cs`:

```csharp
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Tools;

namespace backend.Realtime;

public static class RealtimeEvents
{
    public static JsonObject SessionUpdate(RealtimeSettings cfg, IEnumerable<ITool> enabledTools)
    {
        var tools = new JsonArray();
        foreach (var t in enabledTools)
        {
            tools.Add(new JsonObject
            {
                ["type"] = "function",
                ["name"] = t.Name,
                ["description"] = t.Description,
                ["parameters"] = JsonNode.Parse(t.ParameterSchema.GetRawText())!,
            });
        }

        return new JsonObject
        {
            ["type"] = "session.update",
            ["session"] = new JsonObject
            {
                ["voice"] = cfg.Voice,
                ["instructions"] = cfg.SystemPrompt,
                ["modalities"] = new JsonArray("text", "audio"),
                ["input_audio_format"] = "pcm16",
                ["output_audio_format"] = "pcm16",
                ["input_audio_transcription"] = new JsonObject { ["model"] = "whisper-1" },
                ["turn_detection"] = new JsonObject
                {
                    ["type"] = "server_vad",
                    ["threshold"] = 0.5,
                    ["prefix_padding_ms"] = 300,
                    ["silence_duration_ms"] = 500,
                },
                ["tools"] = tools,
                ["tool_choice"] = "auto",
            },
        };
    }

    public static JsonObject InputAudioBufferAppend(ReadOnlySpan<byte> pcm16)
    {
        return new JsonObject
        {
            ["type"] = "input_audio_buffer.append",
            ["audio"] = Convert.ToBase64String(pcm16),
        };
    }

    public static JsonObject FunctionCallOutput(string callId, string outputJson)
    {
        return new JsonObject
        {
            ["type"] = "conversation.item.create",
            ["item"] = new JsonObject
            {
                ["type"] = "function_call_output",
                ["call_id"] = callId,
                ["output"] = outputJson,
            },
        };
    }

    public static JsonObject ResponseCreate() => new() { ["type"] = "response.create" };
}
```

- [ ] **Step 2: Define IRealtimeUpstream**

Create `backend/Realtime/IRealtimeUpstream.cs`:

```csharp
using System.Text.Json.Nodes;

namespace backend.Realtime;

public interface IRealtimeUpstream : IAsyncDisposable
{
    Task SendJsonAsync(JsonObject envelope, CancellationToken ct);
    Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct); // null on graceful close
}
```

- [ ] **Step 3: Verify build still passes**

Run: `dotnet build`
Expected: 0 errors.

- [ ] **Step 4: Commit**

```bash
git add backend/Realtime/RealtimeEvents.cs backend/Realtime/IRealtimeUpstream.cs
git commit -m "Add IRealtimeUpstream + RealtimeEvents helpers for upstream JSON"
```

---

## Task 8: ScriptedRealtimeUpstream test double

**Files:**
- Create: `backend.tests/ScriptedRealtimeUpstream.cs`
- Create: `backend.tests/FakeDeviceChannel.cs`

`ScriptedRealtimeUpstream` lets each test queue the upstream events the relay will see and capture what the relay sends. It's a pure in-memory `Channel<JsonObject>` pair.

- [ ] **Step 1: Implement the scripted upstream**

Create `backend.tests/ScriptedRealtimeUpstream.cs`:

```csharp
using System.Text.Json.Nodes;
using System.Threading.Channels;
using backend.Realtime;

namespace backend.tests;

public class ScriptedRealtimeUpstream : IRealtimeUpstream
{
    private readonly Channel<JsonObject> _toRelay =
        Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleReader = true });

    private readonly Channel<JsonObject> _fromRelay =
        Channel.CreateUnbounded<JsonObject>(new UnboundedChannelOptions { SingleWriter = true });

    public IList<JsonObject> Sent { get; } = new List<JsonObject>();

    public Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
    {
        Sent.Add(envelope);
        return _fromRelay.Writer.WriteAsync(envelope, ct).AsTask();
    }

    public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
    {
        try
        {
            return await _toRelay.Reader.ReadAsync(ct);
        }
        catch (ChannelClosedException)
        {
            return null;
        }
    }

    public void Push(JsonObject evt) => _toRelay.Writer.TryWrite(evt);
    public void CloseUpstream() => _toRelay.Writer.TryComplete();

    public async Task<JsonObject> WaitForSentAsync(string type, CancellationToken ct)
    {
        await foreach (var item in _fromRelay.Reader.ReadAllAsync(ct))
        {
            if ((string?)item["type"] == type) return item;
        }
        throw new TimeoutException($"never saw outbound {type}");
    }

    public ValueTask DisposeAsync()
    {
        _toRelay.Writer.TryComplete();
        _fromRelay.Writer.TryComplete();
        return ValueTask.CompletedTask;
    }
}
```

- [ ] **Step 2: Implement the fake device channel**

Create `backend.tests/FakeDeviceChannel.cs`:

```csharp
using System.Collections.Concurrent;
using System.Text.Json;
using backend.DeviceHub;
using backend.Tools;

namespace backend.tests;

public class FakeDeviceChannel : IDeviceChannel, IDeviceClient
{
    public List<HubEnvelope> SentEnvelopes { get; } = new();
    public List<byte[]> SentBinary { get; } = new();
    public ConcurrentQueue<JsonElement> PiToolReplies { get; } = new();

    public Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
    {
        SentEnvelopes.Add(envelope);
        return Task.CompletedTask;
    }

    public Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
    {
        SentBinary.Add(bytes.ToArray());
        return Task.CompletedTask;
    }

    public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
    {
        if (PiToolReplies.TryDequeue(out var reply))
            return Task.FromResult(reply);
        // Default: succeed silently.
        return Task.FromResult(JsonDocument.Parse("""{"ok":true}""").RootElement);
    }
}
```

Note: `IDeviceClient` and `HubEnvelope` are introduced in Tasks 11 and 16 respectively. This file references them by name so it will compile only after those tasks land. To keep ordering tidy, write this file but `git commit` it together with whichever of these dependencies arrives first. (Subagent-driven execution should already do this automatically by running `dotnet build` before commit.) If you're executing inline and want the test double to compile right now, leave only the `IDeviceChannel` interface and `PiToolReplies` parts; re-add the `IDeviceClient` surface during Task 16.

- [ ] **Step 3: Verify the scripted upstream compiles (DeviceChannel will follow later)**

Run: `dotnet build backend.tests`
Expected: errors if `FakeDeviceChannel.cs` is included before Task 11. Acceptable workaround: temporarily comment out the `IDeviceClient`/`HubEnvelope` lines until Task 11 lands; the `IDeviceChannel` portion is enough for Tasks 9–14.

- [ ] **Step 4: Commit**

```bash
git add backend.tests/ScriptedRealtimeUpstream.cs backend.tests/FakeDeviceChannel.cs
git commit -m "Add ScriptedRealtimeUpstream + FakeDeviceChannel test doubles"
```

---

## Task 9: RealtimeSession — lifecycle (open → session.update → session_started → close)

**Files:**
- Create: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionLifecycleTests.cs`

The session owns one conversation. `RunAsync(CancellationToken)`:
1. Opens upstream (already constructed by the factory in Task 15; here we accept it via the ctor).
2. Sends `session.update`, awaits `session.updated`.
3. Inserts the `Conversations` row via `ConversationLog`.
4. Emits `session_started` to the device.
5. Pumps upstream events (later tasks add audio/transcript/tools).
6. On externally requested cancel OR `EndReason` set internally, closes upstream, calls `ConversationLog.EndAsync`, emits `session_ended`.

In this task we set up the skeleton and verify a happy-path lifecycle: opens, immediately receives `session.updated` from the scripted upstream, emits `session_started`, then a `CancellationToken` cancel triggers `session_ended(reason=error)`. Later tasks add tool/idle paths.

To avoid the WS dependency, this task introduces a tiny output sink interface `ISessionOutput` that `RealtimeSession` writes to (envelopes + binary). The DeviceHub adapter in Task 16 implements it by writing to the WS; tests implement it directly.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionLifecycleTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionLifecycleTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task Open_emits_session_update_then_session_started_then_session_ended_on_cancel()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var registry = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });

        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings(
            Model: "gpt-4o-realtime-preview",
            Voice: "alloy",
            SystemPrompt: "test",
            IdleTimeoutSeconds: 30,
            EnabledTools: new HashSet<string> { "get_current_time" });

        var session = new RealtimeSession(
            deviceId: Guid.NewGuid(),
            upstream, sink, log, registry, cfg,
            channel: new FakeDeviceChannel(),
            logger: NullLogger.Instance);

        // Push session.updated so the relay can proceed past the gate.
        upstream.Push(new JsonObject { ["type"] = "session.updated" });

        var cts = new CancellationTokenSource();
        var runTask = session.RunAsync(cts.Token);

        // Wait until session_started is emitted to the device.
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        cts.Cancel();
        await runTask;

        upstream.Sent.Select(e => (string?)e["type"]).Should().StartWith("session.update");
        sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_started");
        sink.Envelopes.Select(e => (string?)e["type"]).Should().Contain("session_ended");
    }
}
```

(The test references a `NullLogger` and `RecordingSink` — add them as helpers in the same file or in a small `TestHelpers.cs`. For brevity, inline:)

At the bottom of the file, add:

```csharp
internal class NullLogger : Microsoft.Extensions.Logging.ILogger
{
    public static readonly NullLogger Instance = new();
    public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
    public bool IsEnabled(Microsoft.Extensions.Logging.LogLevel logLevel) => false;
    public void Log<TState>(Microsoft.Extensions.Logging.LogLevel l, Microsoft.Extensions.Logging.EventId e,
        TState s, Exception? ex, Func<TState, Exception?, string> f) { }
}

internal class RecordingSink : ISessionOutput
{
    public List<JsonObject> Envelopes { get; } = new();
    public List<byte[]> Binary { get; } = new();
    public Task WriteEnvelopeAsync(JsonObject e, CancellationToken ct)
    {
        lock (Envelopes) Envelopes.Add(e);
        _gate.Release();
        return Task.CompletedTask;
    }
    public Task WriteBinaryAsync(ReadOnlyMemory<byte> b, CancellationToken ct)
    {
        Binary.Add(b.ToArray());
        return Task.CompletedTask;
    }
    private readonly SemaphoreSlim _gate = new(0, int.MaxValue);
    public async Task WaitForAsync(string type, TimeSpan timeout)
    {
        var deadline = DateTime.UtcNow + timeout;
        while (DateTime.UtcNow < deadline)
        {
            lock (Envelopes)
            {
                if (Envelopes.Any(e => (string?)e["type"] == type)) return;
            }
            await _gate.WaitAsync(timeout);
        }
        throw new TimeoutException($"never saw {type}");
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter RealtimeSessionLifecycleTests`
Expected: build error (`RealtimeSession`, `ISessionOutput` not defined).

- [ ] **Step 3: Implement ISessionOutput + RealtimeSession**

Create `backend/Realtime/RealtimeSession.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Tools;
using Microsoft.Extensions.Logging;

namespace backend.Realtime;

public interface ISessionOutput
{
    Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct);
    Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
}

public class RealtimeSession
{
    private readonly Guid _deviceId;
    private readonly IRealtimeUpstream _upstream;
    private readonly ISessionOutput _output;
    private readonly ConversationLog _log;
    private readonly ToolRegistry _registry;
    private readonly RealtimeSettings _cfg;
    private readonly IDeviceChannel _channel;
    private readonly ILogger _logger;

    private Conversation? _conversation;
    private EndReason _endReason = EndReason.Unknown;
    private bool _endRequested;

    public RealtimeSession(
        Guid deviceId,
        IRealtimeUpstream upstream,
        ISessionOutput output,
        ConversationLog log,
        ToolRegistry registry,
        RealtimeSettings cfg,
        IDeviceChannel channel,
        ILogger logger)
    {
        _deviceId = deviceId;
        _upstream = upstream;
        _output = output;
        _log = log;
        _registry = registry;
        _cfg = cfg;
        _channel = channel;
        _logger = logger;
    }

    public Guid? ConversationId => _conversation?.Id;

    public async Task RunAsync(CancellationToken ct)
    {
        try
        {
            var enabled = _registry.EnabledFor(_cfg.EnabledTools).ToList();
            await _upstream.SendJsonAsync(RealtimeEvents.SessionUpdate(_cfg, enabled), ct);

            // Wait for session.updated.
            var ack = await WaitForUpstreamTypeAsync("session.updated", ct);
            if (ack is null)
            {
                _endReason = EndReason.Error;
                return;
            }

            _conversation = await _log.StartAsync(_deviceId, ct);
            await _output.WriteEnvelopeAsync(new JsonObject
            {
                ["type"] = "session_started",
                ["conversation_id"] = _conversation.Id.ToString(),
            }, ct);

            await PumpAsync(ct);
        }
        catch (OperationCanceledException)
        {
            if (_endReason == EndReason.Unknown) _endReason = EndReason.Error;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "RealtimeSession crashed");
            _endReason = EndReason.Error;
        }
        finally
        {
            await _upstream.DisposeAsync();
            if (_conversation is not null)
            {
                await _log.EndAsync(_conversation.Id, _endReason, CancellationToken.None);
            }
            var endEnv = new JsonObject
            {
                ["type"] = "session_ended",
                ["reason"] = _endReason.ToString().ToLowerInvariant(),
            };
            await _output.WriteEnvelopeAsync(endEnv, CancellationToken.None);
        }
    }

    private async Task PumpAsync(CancellationToken ct)
    {
        // Filled in by Task 10+. Loop until cancel or _endRequested.
        while (!ct.IsCancellationRequested && !_endRequested)
        {
            var evt = await _upstream.ReceiveJsonAsync(ct);
            if (evt is null) { _endReason = EndReason.Error; return; }
            // Future tasks dispatch on (string?)evt["type"].
        }
    }

    private async Task<JsonObject?> WaitForUpstreamTypeAsync(string type, CancellationToken ct)
    {
        while (true)
        {
            var evt = await _upstream.ReceiveJsonAsync(ct);
            if (evt is null) return null;
            if ((string?)evt["type"] == type) return evt;
        }
    }
}
```

- [ ] **Step 4: Run the test**

Run: `dotnet test --filter RealtimeSessionLifecycleTests`
Expected: `Passed: 1`.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/RealtimeSession.cs backend.tests/RealtimeSessionLifecycleTests.cs
git commit -m "Add RealtimeSession lifecycle (open + session.update + session_started/ended)"
```

---

## Task 10: RealtimeSession — audio relay (uplink + downlink)

**Files:**
- Modify: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionAudioTests.cs`

Uplink: device binary frames push into a `Channel<byte[]>` exposed by `RealtimeSession.WriteUplinkAsync(...)`. A dedicated pump task base64-encodes each frame and sends it as `input_audio_buffer.append`. Downlink: when `response.audio.delta` arrives, decode the base64 `delta` field and forward as binary to `ISessionOutput`.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionAudioTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionAudioTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task Uplink_bytes_become_input_audio_buffer_append()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(Array.Empty<ITool>());
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var cts = new CancellationTokenSource();
        var run = session.RunAsync(cts.Token);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        var frame = new byte[3840];
        for (int i = 0; i < frame.Length; i++) frame[i] = (byte)(i % 256);
        await session.WriteUplinkAsync(frame, default);

        // Spin briefly so the pump can flush.
        await Task.Delay(100);

        var appended = upstream.Sent
            .FirstOrDefault(e => (string?)e["type"] == "input_audio_buffer.append");
        appended.Should().NotBeNull();
        var b64 = (string?)appended!["audio"];
        Convert.FromBase64String(b64!).Length.Should().Be(3840);

        cts.Cancel();
        await run;
    }

    [Fact]
    public async Task Audio_delta_from_upstream_becomes_binary_to_device()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(Array.Empty<ITool>());
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var cts = new CancellationTokenSource();
        var run = session.RunAsync(cts.Token);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        var payload = new byte[3840];
        for (int i = 0; i < payload.Length; i++) payload[i] = (byte)((i + 1) % 256);
        upstream.Push(new JsonObject
        {
            ["type"] = "response.audio.delta",
            ["delta"] = Convert.ToBase64String(payload),
        });

        await Task.Delay(100);
        sink.Binary.Should().NotBeEmpty();
        sink.Binary[0].Length.Should().Be(3840);
        sink.Binary[0][0].Should().Be(1);

        cts.Cancel();
        await run;
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter RealtimeSessionAudioTests`
Expected: build error (`WriteUplinkAsync` doesn't exist).

- [ ] **Step 3: Add uplink channel + audio.delta handling**

In `backend/Realtime/RealtimeSession.cs`, at the top of the class add fields:

```csharp
private readonly System.Threading.Channels.Channel<byte[]> _uplink =
    System.Threading.Channels.Channel.CreateBounded<byte[]>(
        new System.Threading.Channels.BoundedChannelOptions(64)
        {
            FullMode = System.Threading.Channels.BoundedChannelFullMode.DropOldest,
        });
```

Add a public uplink writer (used by the device hub):

```csharp
public ValueTask WriteUplinkAsync(byte[] frame, CancellationToken ct)
    => _uplink.Writer.WriteAsync(frame, ct);
```

In `RunAsync`, after emitting `session_started`, launch the uplink pump alongside `PumpAsync`:

```csharp
var uplinkTask = Task.Run(() => PumpUplinkAsync(ct), ct);
await PumpAsync(ct);
_uplink.Writer.TryComplete();
await uplinkTask;
```

Add the pump method:

```csharp
private async Task PumpUplinkAsync(CancellationToken ct)
{
    try
    {
        await foreach (var frame in _uplink.Reader.ReadAllAsync(ct))
        {
            await _upstream.SendJsonAsync(RealtimeEvents.InputAudioBufferAppend(frame), ct);
        }
    }
    catch (OperationCanceledException) { }
    catch (Exception ex) { _logger.LogWarning(ex, "uplink pump errored"); }
}
```

Extend `PumpAsync` to handle `response.audio.delta` — replace the existing loop body with:

```csharp
while (!ct.IsCancellationRequested && !_endRequested)
{
    var evt = await _upstream.ReceiveJsonAsync(ct);
    if (evt is null) { _endReason = EndReason.Error; return; }
    var type = (string?)evt["type"];
    switch (type)
    {
        case "response.audio.delta":
            await ForwardAudioDeltaAsync(evt, ct);
            break;
    }
}
```

And add the helper:

```csharp
private async Task ForwardAudioDeltaAsync(JsonObject evt, CancellationToken ct)
{
    var b64 = (string?)evt["delta"];
    if (string.IsNullOrEmpty(b64)) return;
    var bytes = Convert.FromBase64String(b64);
    await _output.WriteBinaryAsync(bytes, ct);
}
```

- [ ] **Step 4: Run the tests**

Run: `dotnet test --filter RealtimeSessionAudioTests`
Expected: `Passed: 2`.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/RealtimeSession.cs backend.tests/RealtimeSessionAudioTests.cs
git commit -m "RealtimeSession: uplink pump + downlink audio.delta forwarding"
```

---

## Task 11: RealtimeSession — transcript persistence + assistant_done

**Files:**
- Modify: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionTranscriptTests.cs`

Handle three upstream events:
- `response.audio_transcript.done` → `transcript` field, append assistant turn.
- `conversation.item.input_audio_transcription.completed` → `transcript` field, append user turn.
- `response.done` → emit `assistant_done` to the device.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionTranscriptTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionTranscriptTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task User_and_assistant_transcripts_are_persisted_and_response_done_emits_assistant_done()
    {
        using var scope = _factory.Services.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(Array.Empty<ITool>());
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string>());
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var cts = new CancellationTokenSource();
        var run = session.RunAsync(cts.Token);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        upstream.Push(new JsonObject
        {
            ["type"] = "conversation.item.input_audio_transcription.completed",
            ["transcript"] = "hello",
        });
        upstream.Push(new JsonObject
        {
            ["type"] = "response.audio_transcript.done",
            ["transcript"] = "hi there",
        });
        upstream.Push(new JsonObject { ["type"] = "response.done" });

        await sink.WaitForAsync("assistant_done", TimeSpan.FromSeconds(2));

        var convoId = session.ConversationId!.Value;
        var turns = await db.Turns
            .Where(t => t.ConversationId == convoId)
            .OrderBy(t => t.Index).ToListAsync();
        turns.Select(t => t.Role).Should().Equal(TurnRole.User, TurnRole.Assistant);
        turns[0].Text.Should().Be("hello");
        turns[1].Text.Should().Be("hi there");

        cts.Cancel();
        await run;
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter RealtimeSessionTranscriptTests`
Expected: 1 failure (events not yet handled).

- [ ] **Step 3: Extend the dispatch switch**

In `backend/Realtime/RealtimeSession.cs` `PumpAsync`'s switch, add:

```csharp
case "conversation.item.input_audio_transcription.completed":
{
    var text = (string?)evt["transcript"];
    if (_conversation is not null && !string.IsNullOrEmpty(text))
        await _log.AppendUserAsync(_conversation.Id, text!, ct);
    break;
}
case "response.audio_transcript.done":
{
    var text = (string?)evt["transcript"];
    if (_conversation is not null && !string.IsNullOrEmpty(text))
        await _log.AppendAssistantAsync(_conversation.Id, text!, ct);
    break;
}
case "response.done":
    await _output.WriteEnvelopeAsync(
        new JsonObject { ["type"] = "assistant_done" }, ct);
    break;
```

- [ ] **Step 4: Run the test**

Run: `dotnet test --filter RealtimeSessionTranscriptTests`
Expected: `Passed: 1`.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/RealtimeSession.cs backend.tests/RealtimeSessionTranscriptTests.cs
git commit -m "RealtimeSession: persist user/assistant transcripts; emit assistant_done"
```

---

## Task 12: RealtimeSession — server-side tool calls (get_current_time)

**Files:**
- Modify: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionToolsTests.cs`

A tool call surfaces on the upstream as `response.function_call_arguments.done` with `name`, `call_id`, and `arguments` (a stringified JSON). The relay must:
1. Collect these during the response (do not execute yet — wait for `response.done`).
2. At `response.done`:
   - If any collected call has `name == "end_session"`: defer to Task 13 (end-session sequencing).
   - Otherwise for each call: invoke the tool, persist a `Turn(role=tool)`, send `conversation.item.create` (function_call_output) + `response.create` upstream.

Track pending calls in a list on the session.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionToolsTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Data;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionToolsTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task Get_current_time_call_returns_function_call_output_and_persists_tool_turn()
    {
        using var scope = _factory.Services.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(new ITool[] { new GetCurrentTimeTool() });
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "get_current_time" });
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var cts = new CancellationTokenSource();
        var run = session.RunAsync(cts.Token);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        upstream.Push(new JsonObject
        {
            ["type"] = "response.function_call_arguments.done",
            ["call_id"] = "call_abc",
            ["name"] = "get_current_time",
            ["arguments"] = "{}",
        });
        upstream.Push(new JsonObject { ["type"] = "response.done" });

        // Wait for function_call_output + response.create to appear on upstream.
        var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(2);
        while (DateTime.UtcNow < deadline && !upstream.Sent.Any(e =>
            (string?)e["type"] == "conversation.item.create"))
        {
            await Task.Delay(20);
        }

        var item = upstream.Sent.First(e => (string?)e["type"] == "conversation.item.create");
        ((string?)item["item"]!["type"]).Should().Be("function_call_output");
        ((string?)item["item"]!["call_id"]).Should().Be("call_abc");
        ((string?)item["item"]!["output"]).Should().Contain("\"now\"");

        upstream.Sent.Should().Contain(e => (string?)e["type"] == "response.create");

        var convoId = session.ConversationId!.Value;
        var toolTurn = await db.Turns.FirstAsync(t =>
            t.ConversationId == convoId && t.Role == TurnRole.Tool);
        toolTurn.ToolName.Should().Be("get_current_time");
        toolTurn.ToolResultJson.Should().Contain("\"now\"");

        cts.Cancel();
        await run;
    }
}
```

- [ ] **Step 2: Run to see it fail**

Run: `dotnet test --filter RealtimeSessionToolsTests`
Expected: 1 failure.

- [ ] **Step 3: Track + execute pending tool calls**

In `backend/Realtime/RealtimeSession.cs`, add a field at the top of the class:

```csharp
private readonly List<(string CallId, string Name, string ArgsJson)> _pendingCalls = new();
```

In `PumpAsync`'s switch, add:

```csharp
case "response.function_call_arguments.done":
{
    var callId = (string?)evt["call_id"] ?? "";
    var name = (string?)evt["name"] ?? "";
    var args = (string?)evt["arguments"] ?? "{}";
    _pendingCalls.Add((callId, name, args));
    break;
}
```

Replace the existing `case "response.done"` with:

```csharp
case "response.done":
{
    await _output.WriteEnvelopeAsync(
        new JsonObject { ["type"] = "assistant_done" }, ct);

    if (_pendingCalls.Count == 0) break;
    var calls = _pendingCalls.ToList();
    _pendingCalls.Clear();

    if (calls.Any(c => c.Name == "end_session"))
    {
        // Handled in Task 13.
        _endReason = EndReason.Tool;
        _endRequested = true;
        break;
    }

    foreach (var c in calls)
    {
        await ExecuteToolAsync(c.CallId, c.Name, c.ArgsJson, ct);
    }
    break;
}
```

Add the helper:

```csharp
private async Task ExecuteToolAsync(string callId, string name, string argsJson, CancellationToken ct)
{
    var tool = _registry.Get(name);
    string outputJson;
    if (tool is null)
    {
        outputJson = $$"""{"ok":false,"error":"unknown tool: {{name}}"}""";
    }
    else
    {
        try
        {
            using var argsDoc = System.Text.Json.JsonDocument.Parse(argsJson);
            var ctx = new DeviceContext(_deviceId, _conversation!.Id, _channel);
            var res = await tool.ExecuteAsync(
                new ToolInvocation(callId, argsDoc.RootElement.Clone()), ctx, ct);
            outputJson = res.Output.GetRawText();
        }
        catch (Exception ex)
        {
            outputJson = System.Text.Json.JsonSerializer.Serialize(new
            {
                ok = false,
                error = ex.Message,
            });
        }
    }

    if (_conversation is not null)
    {
        await _log.AppendToolAsync(_conversation.Id, name, argsJson, outputJson, ct);
    }

    await _upstream.SendJsonAsync(RealtimeEvents.FunctionCallOutput(callId, outputJson), ct);
    await _upstream.SendJsonAsync(RealtimeEvents.ResponseCreate(), ct);
}
```

- [ ] **Step 4: Run the test**

Run: `dotnet test --filter RealtimeSessionToolsTests`
Expected: `Passed: 1`.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/RealtimeSession.cs backend.tests/RealtimeSessionToolsTests.cs
git commit -m "RealtimeSession: execute server-side tools at response.done"
```

---

## Task 13: RealtimeSession — end_session sequencing

**Files:**
- Modify: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionEndSessionTests.cs`

When `end_session` is one of the pending calls at `response.done`, the relay:
1. Sets `_endReason = EndReason.Tool` and `_endRequested = true`.
2. Does NOT send a `response.create`.
3. Closes the upstream and emits `session_ended(reason=tool)` (this is already handled by `RunAsync`'s `finally`).

Already wired in Task 12 — this task adds the test that proves it, and tightens the audio-draining behaviour so the assistant audio that arrived BEFORE `response.done` is forwarded (also already true, since audio.delta is forwarded eagerly).

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionEndSessionTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionEndSessionTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task End_session_call_closes_session_with_reason_tool_and_skips_response_create()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(new ITool[] { new EndSessionTool() });
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", 30, new HashSet<string> { "end_session" });
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var run = session.RunAsync(default);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        // Push some audio so we can confirm it is forwarded before close.
        upstream.Push(new JsonObject
        {
            ["type"] = "response.audio.delta",
            ["delta"] = Convert.ToBase64String(new byte[3840]),
        });
        upstream.Push(new JsonObject
        {
            ["type"] = "response.function_call_arguments.done",
            ["call_id"] = "c1",
            ["name"] = "end_session",
            ["arguments"] = "{}",
        });
        upstream.Push(new JsonObject { ["type"] = "response.done" });

        await run;

        // Audio forwarded before close.
        sink.Binary.Should().NotBeEmpty();

        // session_ended must have reason=tool.
        var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
        ((string?)ended["reason"]).Should().Be("tool");

        // No response.create was sent upstream.
        upstream.Sent.Should().NotContain(e => (string?)e["type"] == "response.create");
    }
}
```

- [ ] **Step 2: Run the test**

Run: `dotnet test --filter RealtimeSessionEndSessionTests`
Expected: `Passed: 1` (logic already in place from Task 12; if not, the test will pinpoint the gap to fix).

- [ ] **Step 3: Commit**

```bash
git add backend.tests/RealtimeSessionEndSessionTests.cs
git commit -m "Test end_session sequencing closes with reason=tool and skips response.create"
```

---

## Task 14: RealtimeSession — idle timeout

**Files:**
- Modify: `backend/Realtime/RealtimeSession.cs`
- Create: `backend.tests/RealtimeSessionIdleTimeoutTests.cs`

If `IdleTimeoutSeconds` elapses with no `input_audio_buffer.speech_started` upstream event, end the session with `reason=idle`. Each `speech_started` resets the timer.

Implementation: a `DateTime` field `_lastSpeechAt`, updated on `speech_started`. A background task in `RunAsync` checks every 500 ms; if `DateTime.UtcNow - _lastSpeechAt > timeout` AND no `response.done` is pending (i.e. we're not mid-assistant-speech), set `_endReason = Idle` and `_endRequested = true`.

For the test, we'll use a 1-second timeout to keep it fast.

- [ ] **Step 1: Write the failing test**

Create `backend.tests/RealtimeSessionIdleTimeoutTests.cs`:

```csharp
using System.Text.Json.Nodes;
using backend.Conversations;
using backend.Realtime;
using backend.Tools;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class RealtimeSessionIdleTimeoutTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    [Fact]
    public async Task Session_ends_with_reason_idle_when_no_speech_for_timeout()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(Array.Empty<ITool>());
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 1, new HashSet<string>());
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var run = session.RunAsync(default);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        // Wait > 1.5 s without sending any speech_started.
        await run.WaitAsync(TimeSpan.FromSeconds(4));

        var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
        ((string?)ended["reason"]).Should().Be("idle");
    }

    [Fact]
    public async Task Speech_started_resets_the_idle_timer()
    {
        using var scope = _factory.Services.CreateScope();
        var log = scope.ServiceProvider.GetRequiredService<ConversationLog>();
        var reg = new ToolRegistry(Array.Empty<ITool>());
        var upstream = new ScriptedRealtimeUpstream();
        var sink = new RecordingSink();
        var cfg = new RealtimeSettings("m", "v", "p", IdleTimeoutSeconds: 2, new HashSet<string>());
        var session = new RealtimeSession(Guid.NewGuid(), upstream, sink, log, reg, cfg,
            new FakeDeviceChannel(), NullLogger.Instance);

        upstream.Push(new JsonObject { ["type"] = "session.updated" });
        var run = session.RunAsync(default);
        await sink.WaitForAsync("session_started", TimeSpan.FromSeconds(2));

        // Wait 1 s, ping speech, wait 1.5 s — total 2.5 s, but timer reset at 1 s, so still alive at 2.5 s.
        await Task.Delay(1000);
        upstream.Push(new JsonObject { ["type"] = "input_audio_buffer.speech_started" });
        await Task.Delay(1500);

        sink.Envelopes.Any(e => (string?)e["type"] == "session_ended").Should().BeFalse();

        // Then let it actually idle out.
        await run.WaitAsync(TimeSpan.FromSeconds(4));
        var ended = sink.Envelopes.Last(e => (string?)e["type"] == "session_ended");
        ((string?)ended["reason"]).Should().Be("idle");
    }
}
```

- [ ] **Step 2: Run to see them fail**

Run: `dotnet test --filter RealtimeSessionIdleTimeoutTests`
Expected: 2 failures (no idle-timeout logic yet).

- [ ] **Step 3: Add the timer**

In `backend/Realtime/RealtimeSession.cs`, add a field:

```csharp
private DateTime _lastSpeechAt = DateTime.UtcNow;
```

In `PumpAsync`'s switch, add:

```csharp
case "input_audio_buffer.speech_started":
    _lastSpeechAt = DateTime.UtcNow;
    break;
```

Reset `_lastSpeechAt = DateTime.UtcNow;` immediately after emitting `session_started` in `RunAsync` (so the first turn has a fresh window).

In `RunAsync`, after `var uplinkTask = Task.Run(...)`, start the idle watchdog:

```csharp
var idleTask = Task.Run(() => WatchdogAsync(ct), ct);
```

And after `await PumpAsync(ct);`, before `_uplink.Writer.TryComplete();`, add:

```csharp
await idleTask;
```

Add the method:

```csharp
private async Task WatchdogAsync(CancellationToken ct)
{
    try
    {
        var timeout = TimeSpan.FromSeconds(_cfg.IdleTimeoutSeconds);
        while (!ct.IsCancellationRequested && !_endRequested)
        {
            await Task.Delay(250, ct);
            if (DateTime.UtcNow - _lastSpeechAt > timeout)
            {
                _endReason = EndReason.Idle;
                _endRequested = true;
                return;
            }
        }
    }
    catch (OperationCanceledException) { }
}
```

Because `PumpAsync` blocks in `ReceiveJsonAsync`, the watchdog setting `_endRequested = true` won't immediately exit the loop. Tighten `PumpAsync` so it polls upstream with a small linger:

Replace `await _upstream.ReceiveJsonAsync(ct)` with a wrapper that yields if the upstream has no traffic for 250 ms:

```csharp
var evt = await ReceiveOrIdleAsync(ct);
if (evt is null && _endRequested) return;
if (evt is null && ct.IsCancellationRequested) return;
if (evt is null) continue;
```

Add the helper:

```csharp
private async Task<JsonObject?> ReceiveOrIdleAsync(CancellationToken ct)
{
    using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
    var receive = _upstream.ReceiveJsonAsync(timeout.Token).AsTaskOrFromResult();
    var delay = Task.Delay(250, timeout.Token);
    var done = await Task.WhenAny(receive, delay);
    if (done == receive) return await receive;
    timeout.Cancel(); // unhook the receive task; relay tolerates a re-poll
    return null;
}
```

`IRealtimeUpstream.ReceiveJsonAsync` returns `Task<JsonObject?>` so `AsTaskOrFromResult()` is a tiny adapter — drop it; just use `_upstream.ReceiveJsonAsync(timeout.Token)`. Update the helper to:

```csharp
private async Task<JsonObject?> ReceiveOrIdleAsync(CancellationToken ct)
{
    using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
    var receive = _upstream.ReceiveJsonAsync(timeout.Token);
    var delay = Task.Delay(250, timeout.Token);
    var done = await Task.WhenAny(receive, delay);
    if (done == receive) return await receive;
    timeout.Cancel();
    try { await receive; } catch { /* ignore cancellation */ }
    return null;
}
```

(`ScriptedRealtimeUpstream.ReceiveJsonAsync` already throws / returns null on cancel, so the discarded await is safe.)

- [ ] **Step 4: Run the tests**

Run: `dotnet test --filter RealtimeSessionIdleTimeoutTests`
Expected: `Passed: 2`.

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/RealtimeSession.cs backend.tests/RealtimeSessionIdleTimeoutTests.cs
git commit -m "RealtimeSession: idle-timeout watchdog (resets on speech_started)"
```

---

## Task 15: OpenAIRealtimeUpstream (real ClientWebSocket implementation) + RealtimeSessionFactory

**Files:**
- Create: `backend/Realtime/OpenAIRealtimeUpstream.cs`
- Create: `backend/Realtime/RealtimeSessionFactory.cs`
- Modify: `backend/Program.cs`

No unit tests — exercised only by the live smoke (Task 18). Manual confidence: the upstream is small (open WSS with two headers, send/receive JSON text frames) and reading is a tight `ReceiveAsync` loop reassembling fragmented messages.

- [ ] **Step 1: Implement the upstream**

Create `backend/Realtime/OpenAIRealtimeUpstream.cs`:

```csharp
using System.Net.WebSockets;
using System.Text;
using System.Text.Json.Nodes;

namespace backend.Realtime;

public class OpenAIRealtimeUpstream : IRealtimeUpstream
{
    private readonly ClientWebSocket _ws = new();

    public async Task ConnectAsync(string apiKey, string model, CancellationToken ct)
    {
        _ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
        _ws.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
        var uri = new Uri($"wss://api.openai.com/v1/realtime?model={Uri.EscapeDataString(model)}");
        await _ws.ConnectAsync(uri, ct);
    }

    public async Task SendJsonAsync(JsonObject envelope, CancellationToken ct)
    {
        var json = envelope.ToJsonString();
        var bytes = Encoding.UTF8.GetBytes(json);
        await _ws.SendAsync(bytes, WebSocketMessageType.Text, endOfMessage: true, ct);
    }

    public async Task<JsonObject?> ReceiveJsonAsync(CancellationToken ct)
    {
        var buffer = new byte[8192];
        using var stream = new MemoryStream();
        while (true)
        {
            WebSocketReceiveResult result;
            try { result = await _ws.ReceiveAsync(buffer, ct); }
            catch (WebSocketException) { return null; }
            catch (OperationCanceledException) { return null; }

            if (result.MessageType == WebSocketMessageType.Close) return null;
            stream.Write(buffer, 0, result.Count);
            if (result.EndOfMessage)
            {
                var json = Encoding.UTF8.GetString(stream.ToArray());
                return JsonNode.Parse(json)?.AsObject();
            }
        }
    }

    public async ValueTask DisposeAsync()
    {
        if (_ws.State == WebSocketState.Open)
        {
            try { await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None); }
            catch { }
        }
        _ws.Dispose();
    }
}
```

- [ ] **Step 2: Implement RealtimeSessionFactory**

Create `backend/Realtime/RealtimeSessionFactory.cs`:

```csharp
using backend.Conversations;
using backend.Tools;
using Microsoft.Extensions.Logging;

namespace backend.Realtime;

public class RealtimeSessionFactory(
    OpenAIKeyProvider keys,
    ToolRegistry registry,
    ConversationLog log,
    ILoggerFactory loggerFactory)
{
    public async Task<RealtimeSession> CreateAsync(
        Guid deviceId,
        RealtimeSettings cfg,
        ISessionOutput output,
        IDeviceChannel channel,
        CancellationToken ct)
    {
        var upstream = new OpenAIRealtimeUpstream();
        await upstream.ConnectAsync(keys.GetRequired(), cfg.Model, ct);
        return new RealtimeSession(
            deviceId, upstream, output, log, registry, cfg, channel,
            loggerFactory.CreateLogger<RealtimeSession>());
    }
}
```

- [ ] **Step 3: Register in DI**

In `backend/Program.cs`, add:

```csharp
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.GetCurrentTimeTool>();
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.EndSessionTool>();
builder.Services.AddSingleton<backend.Tools.ITool, backend.Tools.SetVolumeTool>();
builder.Services.AddSingleton<backend.Tools.ToolRegistry>();
builder.Services.AddScoped<backend.Realtime.RealtimeSessionFactory>();
```

- [ ] **Step 4: Build verify**

Run: `dotnet build`
Expected: 0 errors. The previous tests continue to pass (no behaviour change to RealtimeSession).

- [ ] **Step 5: Commit**

```bash
git add backend/Realtime/OpenAIRealtimeUpstream.cs backend/Realtime/RealtimeSessionFactory.cs backend/Program.cs
git commit -m "Add OpenAIRealtimeUpstream + RealtimeSessionFactory + DI registrations"
```

---

## Task 16: ActiveDevice + DeviceRegistry + HubEnvelopes + IDeviceClient

**Files:**
- Create: `backend/DeviceHub/HubEnvelopes.cs`
- Create: `backend/DeviceHub/ActiveDevice.cs`
- Create: `backend/DeviceHub/DeviceRegistry.cs`
- Create: `backend.tests/DeviceRegistryTests.cs`
- Modify: `backend.tests/FakeDeviceChannel.cs` (uncomment the IDeviceClient/HubEnvelope references introduced in Task 8)

Envelopes are records with `[JsonPropertyName]` for the snake-case fields used on the wire. `ActiveDevice` owns one WS, implements `IDeviceChannel` (for tools) and `IDeviceClient` (for the relay → device output). It also manages `tool_result` correlation via a `ConcurrentDictionary<callId, TaskCompletionSource<JsonElement>>`.

- [ ] **Step 1: Define envelopes**

Create `backend/DeviceHub/HubEnvelopes.cs`:

```csharp
using System.Text.Json;
using System.Text.Json.Serialization;

namespace backend.DeviceHub;

public record HubEnvelope([property: JsonPropertyName("type")] string Type);

public record HelloEnvelope(string Type,
    [property: JsonPropertyName("device_id")] Guid DeviceId,
    [property: JsonPropertyName("client_version")] string ClientVersion) : HubEnvelope(Type);

public record HelloAckEnvelope(string Type,
    [property: JsonPropertyName("config")] JsonElement Config) : HubEnvelope(Type);

public record PongEnvelope(string Type) : HubEnvelope(Type);

public record WakeEnvelope(string Type,
    [property: JsonPropertyName("at")] long? At) : HubEnvelope(Type);

public record SessionStartedEnvelope(string Type,
    [property: JsonPropertyName("conversation_id")] Guid ConversationId) : HubEnvelope(Type);

public record SessionEndedEnvelope(string Type,
    [property: JsonPropertyName("reason")] string Reason) : HubEnvelope(Type);

public record AssistantDoneEnvelope(string Type) : HubEnvelope(Type);

public record ToolCallEnvelope(string Type,
    [property: JsonPropertyName("call_id")] string CallId,
    [property: JsonPropertyName("name")] string Name,
    [property: JsonPropertyName("arguments")] JsonElement Arguments) : HubEnvelope(Type);

public record ToolResultEnvelope(string Type,
    [property: JsonPropertyName("call_id")] string CallId,
    [property: JsonPropertyName("ok")] bool Ok,
    [property: JsonPropertyName("result")] JsonElement? Result,
    [property: JsonPropertyName("error")] string? Error) : HubEnvelope(Type);

public record ErrorEnvelope(string Type,
    [property: JsonPropertyName("code")] string Code,
    [property: JsonPropertyName("message")] string Message,
    [property: JsonPropertyName("fatal")] bool? Fatal) : HubEnvelope(Type);
```

- [ ] **Step 2: Define IDeviceClient**

In a new section of `backend/DeviceHub/ActiveDevice.cs` (file written next step), the relay calls these to push to the device. Declare interface inline at the top:

```csharp
public interface IDeviceClient
{
    Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope;
    Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
}
```

- [ ] **Step 3: Implement ActiveDevice**

Create `backend/DeviceHub/ActiveDevice.cs`:

```csharp
using System.Collections.Concurrent;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Realtime;
using backend.Tools;

namespace backend.DeviceHub;

public interface IDeviceClient
{
    Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope;
    Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct);
}

public class ActiveDevice : IDeviceChannel, IDeviceClient, ISessionOutput, IAsyncDisposable
{
    public Guid DeviceId { get; }
    private readonly WebSocket _ws;
    private readonly SemaphoreSlim _sendLock = new(1, 1);
    private readonly ConcurrentDictionary<string, TaskCompletionSource<JsonElement>> _pendingTools = new();

    public ActiveDevice(Guid deviceId, WebSocket ws)
    {
        DeviceId = deviceId;
        _ws = ws;
    }

    public async Task SendEnvelopeAsync<T>(T envelope, CancellationToken ct) where T : HubEnvelope
    {
        var json = JsonSerializer.Serialize(envelope, envelope.GetType());
        await _sendLock.WaitAsync(ct);
        try
        {
            await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
                WebSocketMessageType.Text, endOfMessage: true, ct);
        }
        finally { _sendLock.Release(); }
    }

    public async Task SendBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
    {
        await _sendLock.WaitAsync(ct);
        try
        {
            await _ws.SendAsync(bytes, WebSocketMessageType.Binary, endOfMessage: true, ct);
        }
        finally { _sendLock.Release(); }
    }

    public Task WriteEnvelopeAsync(JsonObject envelope, CancellationToken ct)
    {
        // Relay uses JsonObject for upstream; we marshal back to a string here.
        var json = envelope.ToJsonString();
        return WriteRawTextAsync(json, ct);
    }

    private async Task WriteRawTextAsync(string json, CancellationToken ct)
    {
        await _sendLock.WaitAsync(ct);
        try
        {
            await _ws.SendAsync(Encoding.UTF8.GetBytes(json),
                WebSocketMessageType.Text, endOfMessage: true, ct);
        }
        finally { _sendLock.Release(); }
    }

    public Task WriteBinaryAsync(ReadOnlyMemory<byte> bytes, CancellationToken ct)
        => SendBinaryAsync(bytes, ct);

    public Task<JsonElement> CallPiToolAsync(string name, JsonElement args, CancellationToken ct)
    {
        var callId = "srv_" + Guid.NewGuid().ToString("N")[..12];
        var tcs = new TaskCompletionSource<JsonElement>(TaskCreationOptions.RunContinuationsAsynchronously);
        _pendingTools[callId] = tcs;

        var envelope = new ToolCallEnvelope("tool_call", callId, name, args);
        _ = SendEnvelopeAsync(envelope, ct);

        ct.Register(() => tcs.TrySetCanceled());
        return tcs.Task;
    }

    public void CompleteToolResult(ToolResultEnvelope env)
    {
        if (!_pendingTools.TryRemove(env.CallId, out var tcs)) return;
        if (env.Ok && env.Result is JsonElement r) tcs.TrySetResult(r);
        else
        {
            var doc = JsonDocument.Parse($$"""{"ok":false,"error":"{{env.Error ?? "unknown"}}"}""");
            tcs.TrySetResult(doc.RootElement.Clone());
        }
    }

    public async ValueTask DisposeAsync()
    {
        try
        {
            if (_ws.State == WebSocketState.Open)
                await _ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "bye", CancellationToken.None);
        }
        catch { }
        _ws.Dispose();
        _sendLock.Dispose();
    }
}
```

- [ ] **Step 4: Implement DeviceRegistry**

Create `backend/DeviceHub/DeviceRegistry.cs`:

```csharp
using System.Collections.Concurrent;

namespace backend.DeviceHub;

public class DeviceRegistry
{
    private readonly ConcurrentDictionary<Guid, ActiveDevice> _online = new();

    public bool TryRegister(ActiveDevice dev)
        => _online.TryAdd(dev.DeviceId, dev);

    public bool Unregister(Guid deviceId)
        => _online.TryRemove(deviceId, out _);

    public ActiveDevice? Get(Guid deviceId)
        => _online.TryGetValue(deviceId, out var d) ? d : null;

    public bool IsOnline(Guid deviceId) => _online.ContainsKey(deviceId);

    public int OnlineCount => _online.Count;
}
```

- [ ] **Step 5: Write the registry test**

Create `backend.tests/DeviceRegistryTests.cs`:

```csharp
using System.Net.WebSockets;
using backend.DeviceHub;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class DeviceRegistryTests
{
    [Fact]
    public void Register_unregister_round_trip()
    {
        var reg = new DeviceRegistry();
        var id = Guid.NewGuid();
        var dev = new ActiveDevice(id, new ClientWebSocket()); // not connected — never sends
        reg.TryRegister(dev).Should().BeTrue();
        reg.TryRegister(dev).Should().BeFalse();
        reg.IsOnline(id).Should().BeTrue();
        reg.OnlineCount.Should().Be(1);

        reg.Unregister(id).Should().BeTrue();
        reg.IsOnline(id).Should().BeFalse();
        reg.OnlineCount.Should().Be(0);
    }
}
```

- [ ] **Step 6: Update FakeDeviceChannel (if you stubbed it in Task 8)**

If you stubbed out the `IDeviceClient`/`HubEnvelope` lines back in Task 8, uncomment / restore them now. The full file (Task 8 step 2) is the final version.

- [ ] **Step 7: Register in DI**

In `backend/Program.cs`, add:

```csharp
builder.Services.AddSingleton<backend.DeviceHub.DeviceRegistry>();
```

- [ ] **Step 8: Run tests**

Run: `dotnet test --filter DeviceRegistryTests`
Expected: `Passed: 1`. The full suite still passes.

- [ ] **Step 9: Commit**

```bash
git add backend/DeviceHub backend.tests/DeviceRegistryTests.cs backend.tests/FakeDeviceChannel.cs backend/Program.cs
git commit -m "Add ActiveDevice + DeviceRegistry + HubEnvelopes + IDeviceClient/Channel implementations"
```

---

## Task 17: DeviceHubEndpoint — bearer auth + WS upgrade + handshake + ping/pong

**Files:**
- Create: `backend/DeviceHub/DeviceAuth.cs`
- Create: `backend/DeviceHub/DeviceHubEndpoint.cs`
- Create: `backend.tests/DeviceHubHandshakeTests.cs`
- Modify: `backend/Program.cs`

The endpoint:
1. Reads `Authorization: Bearer <token>` from the upgrade request.
2. Looks up the device by `TokenHash` (DeviceTokenService.Hash + DB query).
3. Rejects with HTTP 401 if not found or revoked.
4. Upgrades the WS.
5. Reads the first text frame; expects `hello`; responds with `hello_ack` carrying the snapshot config.
6. Loops: dispatch by `type`. Implements `ping` → updates `LastSeenAt`, replies `pong`.
7. Cleanly closes on cancel / client disconnect; removes from `DeviceRegistry`.

This task ships handshake + ping/pong only. Wake/audio routing is Task 18.

- [ ] **Step 1: Define DeviceAuth helper**

Create `backend/DeviceHub/DeviceAuth.cs`:

```csharp
using backend.Data;
using backend.Devices;
using Microsoft.EntityFrameworkCore;

namespace backend.DeviceHub;

public class DeviceAuth(AppDbContext db, DeviceTokenService tokens)
{
    public async Task<Device?> AuthenticateAsync(string? authHeader, CancellationToken ct)
    {
        if (string.IsNullOrEmpty(authHeader)) return null;
        const string prefix = "Bearer ";
        if (!authHeader.StartsWith(prefix, StringComparison.Ordinal)) return null;
        var token = authHeader[prefix.Length..].Trim();
        if (string.IsNullOrEmpty(token)) return null;
        var hash = tokens.Hash(token);
        var device = await db.Devices.Include(d => d.Config)
            .FirstOrDefaultAsync(d => d.TokenHash == hash && !d.IsRevoked, ct);
        return device;
    }
}
```

- [ ] **Step 2: Implement the endpoint (handshake only)**

Create `backend/DeviceHub/DeviceHubEndpoint.cs`:

```csharp
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using backend.Data;
using backend.Devices;
using Microsoft.EntityFrameworkCore;

namespace backend.DeviceHub;

public static class DeviceHubEndpoint
{
    public static IEndpointRouteBuilder MapDeviceHub(this IEndpointRouteBuilder app)
    {
        app.Map("/device", async (HttpContext ctx, DeviceAuth auth, DeviceRegistry reg, AppDbContext db) =>
        {
            if (!ctx.WebSockets.IsWebSocketRequest)
            {
                ctx.Response.StatusCode = StatusCodes.Status400BadRequest;
                return;
            }

            var device = await auth.AuthenticateAsync(ctx.Request.Headers.Authorization.ToString(), ctx.RequestAborted);
            if (device is null)
            {
                ctx.Response.StatusCode = StatusCodes.Status401Unauthorized;
                return;
            }

            using var ws = await ctx.WebSockets.AcceptWebSocketAsync();
            await using var active = new ActiveDevice(device.Id, ws);
            if (!reg.TryRegister(active))
            {
                await ws.CloseAsync(WebSocketCloseStatus.PolicyViolation,
                    "device already connected", ctx.RequestAborted);
                return;
            }

            try
            {
                await HandleAsync(active, device, db, ctx.RequestAborted);
            }
            finally
            {
                reg.Unregister(device.Id);
            }
        });
        return app;
    }

    private static async Task HandleAsync(ActiveDevice active, Device device, AppDbContext db, CancellationToken ct)
    {
        var buffer = new byte[8192];
        var ws = ActiveWs(active);

        while (ws.State == WebSocketState.Open && !ct.IsCancellationRequested)
        {
            using var msg = new MemoryStream();
            WebSocketReceiveResult res;
            while (true)
            {
                res = await ws.ReceiveAsync(buffer, ct);
                if (res.MessageType == WebSocketMessageType.Close) return;
                msg.Write(buffer, 0, res.Count);
                if (res.EndOfMessage) break;
            }

            if (res.MessageType == WebSocketMessageType.Binary)
            {
                // Wake-path handling lives in Task 18. For now, drop.
                if (msg.Length != 3840) continue;
                continue;
            }

            var text = Encoding.UTF8.GetString(msg.ToArray());
            using var doc = JsonDocument.Parse(text);
            var type = doc.RootElement.GetProperty("type").GetString();
            switch (type)
            {
                case "hello":
                    await SendHelloAckAsync(active, device, ct);
                    break;

                case "ping":
                    device.LastSeenAt = DateTimeOffset.UtcNow;
                    await db.SaveChangesAsync(ct);
                    await active.SendEnvelopeAsync(new PongEnvelope("pong"), ct);
                    break;
            }
        }
    }

    private static WebSocket ActiveWs(ActiveDevice active)
    {
        // Reach into the private WS via reflection-free accessor: ActiveDevice exposes nothing else; we know the field.
        // To avoid touching internals, ActiveDevice gains a `WebSocket Ws { get; }` getter. (Update Task 16 file accordingly.)
        return active.Ws;
    }

    private static async Task SendHelloAckAsync(ActiveDevice active, Device device, CancellationToken ct)
    {
        var enabled = device.Config?.EnabledToolsJson ?? "[]";
        var config = JsonNode.Parse($$"""
        {
          "voice": "{{Escape(device.Config?.Voice)}}",
          "model": "{{Escape(device.Config?.Model)}}",
          "system_prompt": "{{Escape(device.Config?.SystemPrompt)}}",
          "idle_timeout_seconds": {{device.Config?.IdleTimeoutSeconds ?? 30}},
          "enabled_tools": {{enabled}}
        }
        """)!.AsObject();

        await active.WriteEnvelopeAsync(new JsonObject
        {
            ["type"] = "hello_ack",
            ["config"] = config,
        }, ct);
    }

    private static string Escape(string? s) =>
        (s ?? "").Replace("\\", "\\\\").Replace("\"", "\\\"");
}
```

Reflecting the note above: in `backend/DeviceHub/ActiveDevice.cs`, expose the WS:

```csharp
public WebSocket Ws => _ws;
```

(Add this as a public property next to `DeviceId`.)

- [ ] **Step 3: Wire it up**

In `backend/Program.cs`:

```csharp
builder.Services.AddScoped<backend.DeviceHub.DeviceAuth>();
```

After other endpoint registrations but BEFORE `app.Run();`, add:

```csharp
app.UseWebSockets();
app.MapDeviceHub();
```

(`UseWebSockets()` must run before `Map("/device")` to enable WS upgrades.)

- [ ] **Step 4: Write the integration test**

Create `backend.tests/DeviceHubHandshakeTests.cs`:

```csharp
using System.Net.Http.Json;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using backend.Devices;
using FluentAssertions;
using Microsoft.Extensions.DependencyInjection;
using Xunit;

namespace backend.tests;

public class DeviceHubHandshakeTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    private async Task<string> PairAndGetTokenAsync()
    {
        var http = _factory.CreateClient();
        await http.PostAsJsonAsync("/api/auth/register",
            new { email = "ws@example.com", password = "Passw0rd!" });
        await http.PostAsJsonAsync("/api/auth/login",
            new { email = "ws@example.com", password = "Passw0rd!" });
        var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "ws-test" });
        var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;

        var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
            new { code, hostname = "smoke", client_version = "0.1" });
        var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
        return pairBody!["device_token"].ToString()!;
    }

    [Fact]
    public async Task Connect_without_token_returns_401()
    {
        var client = _factory.Server.CreateWebSocketClient();
        var uri = new Uri(_factory.Server.BaseAddress, "device");
        var act = async () => await client.ConnectAsync(uri, CancellationToken.None);
        await act.Should().ThrowAsync<Exception>(); // 401 surfaces as a WS upgrade error
    }

    [Fact]
    public async Task Hello_ack_includes_config()
    {
        var token = await PairAndGetTokenAsync();
        var client = _factory.Server.CreateWebSocketClient();
        client.ConfigureRequest = req => req.Headers.Authorization =
            new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);

        var uri = new Uri(_factory.Server.BaseAddress, "device");
        var ws = await client.ConnectAsync(uri, CancellationToken.None);

        await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
        var ack = await ReceiveJson(ws);
        ack.GetProperty("type").GetString().Should().Be("hello_ack");
        ack.GetProperty("config").GetProperty("voice").GetString().Should().NotBeNullOrEmpty();
        ack.GetProperty("config").GetProperty("model").GetString().Should().NotBeNullOrEmpty();
        ack.GetProperty("config").GetProperty("enabled_tools").GetArrayLength().Should().BeGreaterThan(0);
    }

    [Fact]
    public async Task Ping_replies_with_pong_and_updates_last_seen()
    {
        var token = await PairAndGetTokenAsync();
        var client = _factory.Server.CreateWebSocketClient();
        client.ConfigureRequest = req => req.Headers.Authorization =
            new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
        var uri = new Uri(_factory.Server.BaseAddress, "device");
        var ws = await client.ConnectAsync(uri, CancellationToken.None);

        await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
        await ReceiveJson(ws); // drain ack
        await SendJson(ws, new { type = "ping" });
        var pong = await ReceiveJson(ws);
        pong.GetProperty("type").GetString().Should().Be("pong");
    }

    private static async Task SendJson(WebSocket ws, object payload)
    {
        var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
        await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
    }

    private static async Task<JsonElement> ReceiveJson(WebSocket ws)
    {
        var buf = new byte[8192];
        using var stream = new MemoryStream();
        while (true)
        {
            var r = await ws.ReceiveAsync(buf, CancellationToken.None);
            stream.Write(buf, 0, r.Count);
            if (r.EndOfMessage) break;
        }
        return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
    }
}
```

- [ ] **Step 5: Run the tests**

Run: `dotnet test --filter DeviceHubHandshakeTests`
Expected: `Passed: 3`.

- [ ] **Step 6: Commit**

```bash
git add backend/DeviceHub/DeviceAuth.cs backend/DeviceHub/DeviceHubEndpoint.cs backend/DeviceHub/ActiveDevice.cs backend/Program.cs backend.tests/DeviceHubHandshakeTests.cs
git commit -m "DeviceHub: /device WS with bearer auth + hello/hello_ack + ping/pong"
```

---

## Task 18: DeviceHubEndpoint — wake → session lifecycle, audio relay, tool_result fwd

**Files:**
- Modify: `backend/DeviceHub/DeviceHubEndpoint.cs`
- Create: `backend.tests/DeviceHubWakeFlowTests.cs`

Add to the dispatch:
- `wake`: if no active session, fetch `DeviceConfig` from the DB, build `RealtimeSettings`, ask `RealtimeSessionFactory.CreateAsync(...)`, kick off `session.RunAsync(...)` in the background. Track the session on the `ActiveDevice`.
- Binary frames: route to the active session's `WriteUplinkAsync`.
- `cancel`: cancel the session CTS.
- `playback_done`: log only — used by the Pi state machine; no server logic needed in v1.
- `tool_result`: route to `ActiveDevice.CompleteToolResult(...)`.

For the integration test, swap in a fake `RealtimeSessionFactory` that uses `ScriptedRealtimeUpstream` instead of OpenAI. Inject by overriding the DI registration in `ApiFactory`.

- [ ] **Step 1: Extend ActiveDevice with session tracking**

In `backend/DeviceHub/ActiveDevice.cs`, add:

```csharp
public RealtimeSession? CurrentSession { get; set; }
public CancellationTokenSource? SessionCts { get; set; }
```

(Imports: `using backend.Realtime;`.)

- [ ] **Step 2: Make RealtimeSessionFactory test-overridable**

In `backend/Realtime/RealtimeSessionFactory.cs`, mark the `CreateAsync` method `virtual` so tests can subclass and inject `ScriptedRealtimeUpstream`.

Change the signature:

```csharp
public virtual async Task<RealtimeSession> CreateAsync(...)
```

- [ ] **Step 3: Extend the dispatch**

In `backend/DeviceHub/DeviceHubEndpoint.cs`, after the existing `using backend.Data;`, add:

```csharp
using backend.Realtime;
using backend.Tools;
using backend.Conversations;
```

Change the `Map("/device", ...)` lambda signature to also pull `RealtimeSessionFactory` and the DI scope provider:

```csharp
app.Map("/device", async (
    HttpContext ctx,
    DeviceAuth auth,
    DeviceRegistry reg,
    AppDbContext db,
    RealtimeSessionFactory sessions,
    IServiceScopeFactory scopes) =>
```

Pass `sessions, scopes` down into `HandleAsync` (add to its parameters).

In the dispatch switch inside `HandleAsync`, add:

```csharp
case "wake":
    if (active.CurrentSession is null)
        await StartSessionAsync(active, device, db, sessions, scopes, ct);
    break;

case "cancel":
    active.SessionCts?.Cancel();
    break;

case "playback_done":
    break; // v1 client coordination only

case "tool_result":
    var env = JsonSerializer.Deserialize<ToolResultEnvelope>(text)!;
    active.CompleteToolResult(env);
    break;
```

And in the binary-frame branch, replace the drop with:

```csharp
if (msg.Length == 3840 && active.CurrentSession is not null)
{
    await active.CurrentSession.WriteUplinkAsync(msg.ToArray(), ct);
}
```

Add the helper:

```csharp
private static async Task StartSessionAsync(
    ActiveDevice active, Device device, AppDbContext db,
    RealtimeSessionFactory sessions, IServiceScopeFactory scopes, CancellationToken parentCt)
{
    var enabledTools = JsonSerializer.Deserialize<string[]>(device.Config?.EnabledToolsJson ?? "[]")
        ?? Array.Empty<string>();
    var cfg = new RealtimeSettings(
        Model: device.Config?.Model ?? "gpt-4o-realtime-preview",
        Voice: device.Config?.Voice ?? "alloy",
        SystemPrompt: device.Config?.SystemPrompt ?? "",
        IdleTimeoutSeconds: device.Config?.IdleTimeoutSeconds ?? 30,
        EnabledTools: new HashSet<string>(enabledTools));

    active.SessionCts = CancellationTokenSource.CreateLinkedTokenSource(parentCt);
    var ct = active.SessionCts.Token;

    var sessionScope = scopes.CreateAsyncScope();
    var scopedSessions = sessionScope.ServiceProvider.GetRequiredService<RealtimeSessionFactory>();

    RealtimeSession session;
    try
    {
        session = await scopedSessions.CreateAsync(
            device.Id, cfg, active, active, ct);
    }
    catch (Exception ex)
    {
        await active.SendEnvelopeAsync(
            new ErrorEnvelope("error", "upstream", ex.Message, true), parentCt);
        await sessionScope.DisposeAsync();
        return;
    }

    active.CurrentSession = session;
    _ = Task.Run(async () =>
    {
        try { await session.RunAsync(ct); }
        finally
        {
            active.CurrentSession = null;
            active.SessionCts?.Dispose();
            active.SessionCts = null;
            await sessionScope.DisposeAsync();
        }
    }, ct);
}
```

(Pull `using Microsoft.Extensions.DependencyInjection;` to the top.)

- [ ] **Step 4: Override the factory in tests**

In `backend.tests/ApiFactory.cs`, add a field `public ScriptedRealtimeUpstream? Upstream { get; set; }`. Override services:

```csharp
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
    builder.UseEnvironment("Test");
    builder.ConfigureAppConfiguration((_, cfg) =>
    {
        cfg.AddInMemoryCollection(new Dictionary<string, string?>
        {
            ["ConnectionStrings:Default"] = $"Data Source={_dbPath}",
            ["OPENAI_API_KEY"] = "test-key",
        });
    });

    builder.ConfigureTestServices(services =>
    {
        services.AddScoped<RealtimeSessionFactory, FakeRealtimeSessionFactory>();
    });
}
```

Add `using backend.Realtime;` and `using Microsoft.Extensions.DependencyInjection;` to the file. Then create `FakeRealtimeSessionFactory`:

```csharp
public class FakeRealtimeSessionFactory(
    OpenAIKeyProvider keys,
    ToolRegistry registry,
    ConversationLog log,
    ILoggerFactory loggerFactory)
    : RealtimeSessionFactory(keys, registry, log, loggerFactory)
{
    public static ScriptedRealtimeUpstream Upstream { get; } = new();

    public override Task<RealtimeSession> CreateAsync(
        Guid deviceId, RealtimeSettings cfg, ISessionOutput output,
        IDeviceChannel channel, CancellationToken ct)
    {
        Upstream.Push(new System.Text.Json.Nodes.JsonObject { ["type"] = "session.updated" });
        var s = new RealtimeSession(
            deviceId, Upstream, output, log, registry, cfg, channel,
            loggerFactory.CreateLogger<RealtimeSession>());
        return Task.FromResult(s);
    }
}
```

(Adjust the visibility of the protected ctor params of `RealtimeSessionFactory` if needed — or keep them on the base class as `protected internal` for the inheritance to compile. Easier: in `RealtimeSessionFactory` change the constructor params to a single `protected` ctor, and store them as `protected` fields the subclass uses.)

Update `RealtimeSessionFactory`:

```csharp
public class RealtimeSessionFactory
{
    protected readonly OpenAIKeyProvider keys;
    protected readonly ToolRegistry registry;
    protected readonly ConversationLog log;
    protected readonly ILoggerFactory loggerFactory;

    public RealtimeSessionFactory(OpenAIKeyProvider keys, ToolRegistry registry,
        ConversationLog log, ILoggerFactory loggerFactory)
    {
        this.keys = keys; this.registry = registry; this.log = log; this.loggerFactory = loggerFactory;
    }

    public virtual async Task<RealtimeSession> CreateAsync(...) // unchanged body
}
```

- [ ] **Step 5: Write the integration test**

Create `backend.tests/DeviceHubWakeFlowTests.cs`:

```csharp
using System.Net.Http.Json;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using FluentAssertions;
using Xunit;

namespace backend.tests;

public class DeviceHubWakeFlowTests(ApiFactory factory) : IClassFixture<ApiFactory>
{
    private readonly ApiFactory _factory = factory;

    private async Task<string> PairAndGetTokenAsync(string email)
    {
        var http = _factory.CreateClient();
        await http.PostAsJsonAsync("/api/auth/register",
            new { email, password = "Passw0rd!" });
        await http.PostAsJsonAsync("/api/auth/login",
            new { email, password = "Passw0rd!" });
        var codeRes = await http.PostAsJsonAsync("/api/pair-code", new { name = "wake-test" });
        var code = (await codeRes.Content.ReadFromJsonAsync<Dictionary<string, object>>())!["code"].ToString()!;
        var pairRes = await _factory.CreateClient().PostAsJsonAsync("/api/pair",
            new { code, hostname = "smoke", client_version = "0.1" });
        var pairBody = await pairRes.Content.ReadFromJsonAsync<Dictionary<string, object>>();
        return pairBody!["device_token"].ToString()!;
    }

    [Fact]
    public async Task Wake_opens_session_started_then_assistant_done_then_session_ended_on_idle()
    {
        var token = await PairAndGetTokenAsync("wake1@example.com");
        var client = _factory.Server.CreateWebSocketClient();
        client.ConfigureRequest = req => req.Headers.Authorization =
            new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
        var uri = new Uri(_factory.Server.BaseAddress, "device");
        var ws = await client.ConnectAsync(uri, CancellationToken.None);

        await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
        await ReceiveJson(ws); // ack

        await SendJson(ws, new { type = "wake", at = 0L });
        var started = await ReceiveJson(ws);
        started.GetProperty("type").GetString().Should().Be("session_started");

        FakeRealtimeSessionFactory.Upstream.Push(new JsonObject
        {
            ["type"] = "response.audio_transcript.done",
            ["transcript"] = "hello!",
        });
        FakeRealtimeSessionFactory.Upstream.Push(new JsonObject { ["type"] = "response.done" });

        var done = await ReceiveJson(ws);
        done.GetProperty("type").GetString().Should().Be("assistant_done");

        // Send cancel to end the session quickly (don't wait for idle-timeout).
        await SendJson(ws, new { type = "cancel" });
        var ended = await ReceiveJson(ws);
        ended.GetProperty("type").GetString().Should().Be("session_ended");
    }

    [Fact]
    public async Task Binary_uplink_after_wake_reaches_upstream_as_input_audio_buffer_append()
    {
        var token = await PairAndGetTokenAsync("wake2@example.com");
        var client = _factory.Server.CreateWebSocketClient();
        client.ConfigureRequest = req => req.Headers.Authorization =
            new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", token);
        var uri = new Uri(_factory.Server.BaseAddress, "device");
        var ws = await client.ConnectAsync(uri, CancellationToken.None);

        await SendJson(ws, new { type = "hello", device_id = Guid.NewGuid(), client_version = "0.1" });
        await ReceiveJson(ws);
        await SendJson(ws, new { type = "wake", at = 0L });
        await ReceiveJson(ws); // session_started

        var frame = new byte[3840];
        for (int i = 0; i < frame.Length; i++) frame[i] = 0x55;
        await ws.SendAsync(frame, WebSocketMessageType.Binary, true, CancellationToken.None);

        await Task.Delay(200);
        FakeRealtimeSessionFactory.Upstream.Sent
            .Any(e => (string?)e["type"] == "input_audio_buffer.append")
            .Should().BeTrue();

        await SendJson(ws, new { type = "cancel" });
        await ReceiveJson(ws);
    }

    private static async Task SendJson(WebSocket ws, object payload)
    {
        var bytes = Encoding.UTF8.GetBytes(JsonSerializer.Serialize(payload));
        await ws.SendAsync(bytes, WebSocketMessageType.Text, true, CancellationToken.None);
    }

    private static async Task<JsonElement> ReceiveJson(WebSocket ws)
    {
        var buf = new byte[16384];
        using var stream = new MemoryStream();
        while (true)
        {
            var r = await ws.ReceiveAsync(buf, CancellationToken.None);
            if (r.MessageType == WebSocketMessageType.Binary) { stream.Write(buf, 0, r.Count); continue; }
            stream.Write(buf, 0, r.Count);
            if (r.EndOfMessage) break;
        }
        return JsonDocument.Parse(stream.ToArray()).RootElement.Clone();
    }
}
```

- [ ] **Step 6: Run the tests**

Run: `dotnet test --filter DeviceHubWakeFlowTests`
Expected: `Passed: 2`.

- [ ] **Step 7: Run the full suite**

Run: `dotnet test`
Expected: every test (Plan 1 + everything above) passes.

- [ ] **Step 8: Commit**

```bash
git add backend/DeviceHub backend/Realtime/RealtimeSessionFactory.cs backend.tests/ApiFactory.cs backend.tests/DeviceHubWakeFlowTests.cs
git commit -m "DeviceHub: wake → session lifecycle + uplink relay + tool_result routing"
```

---

## Task 19: Coolify deploy + live smoke

**Files:**
- Modify: `README.md`

Coolify already runs the backend image (from Plan 1). Plan 2 only added code; no Dockerfile change needed. Trigger a deploy via the Coolify v4 API and confirm `/device` rejects unauthenticated WS upgrades on the live host.

- [ ] **Step 1: Ensure `OPENAI_API_KEY` is set on the Coolify app**

Use the `deploying-to-coolify-via-api` skill. If the env var is not already present on the app, PATCH it via the API. Do NOT log the value.

If unsure, ask the user to confirm the key is set and proceed once they do.

- [ ] **Step 2: Trigger a deploy (fire-and-forget per global CLAUDE.md)**

```bash
curl -sf -X POST \
  -H "Authorization: Bearer $COOLIFY_KEY" \
  "$COOLIFY_URL/api/v1/deploy?uuid=$APP_UUID"
```

Read `APP_UUID` from `deploy.json` if present, otherwise from the Coolify app list. Expected: a 2xx response with a deployment UUID. Print it and stop — do NOT poll the build.

- [ ] **Step 3: Live smoke — `/device` rejects unauthenticated WS upgrade**

After a few minutes (or whenever the user signals the deploy is live), confirm the endpoint is reachable. From the sandbox:

```bash
DOMAIN="https://assistant.volcanic.tes.gd"
# Just check the HTTP-level handshake fails closed.
curl -sI -H "Connection: Upgrade" -H "Upgrade: websocket" \
     -H "Sec-WebSocket-Version: 13" \
     -H "Sec-WebSocket-Key: $(head -c16 /dev/urandom | base64)" \
     "$DOMAIN/device" | head -1
```

Expected: a `401 Unauthorized` status line (no `Authorization` header was sent).

- [ ] **Step 4: Live smoke — authenticated handshake works**

Re-use the Plan 1 smoke recipe to pair a "smoke" device, then connect with the returned token using a tiny Python script. From the sandbox:

```bash
DOMAIN="https://assistant.volcanic.tes.gd"
COOKIE=$(mktemp)
curl -sf -c "$COOKIE" -X POST "$DOMAIN/api/auth/login" \
  -H 'content-type: application/json' \
  -d '{"email":"<your existing email>","password":"<your password>"}'
CODE=$(curl -sf -c "$COOKIE" -b "$COOKIE" -X POST "$DOMAIN/api/pair-code" \
  -H 'content-type: application/json' \
  -d '{"name":"plan2-smoke"}' | grep -oE '"code":"[^"]+"' | cut -d'"' -f4)
TOKEN=$(curl -sf -X POST "$DOMAIN/api/pair" \
  -H 'content-type: application/json' \
  -d "{\"code\":\"$CODE\",\"hostname\":\"smoke\",\"client_version\":\"0.1\"}" \
  | grep -oE '"device_token":"[^"]+"' | cut -d'"' -f4)
echo "TOKEN length: ${#TOKEN}"

python3 - <<PY
import asyncio, json, sys
import websockets
URL = "wss://assistant.volcanic.tes.gd/device"
TOKEN = "$TOKEN"

async def main():
    async with websockets.connect(URL, additional_headers={"Authorization": f"Bearer {TOKEN}"}) as ws:
        await ws.send(json.dumps({"type":"hello","device_id":"00000000-0000-0000-0000-000000000000","client_version":"0.1"}))
        ack = json.loads(await ws.recv())
        print("ack:", ack["type"])
        assert ack["type"] == "hello_ack"
        await ws.send(json.dumps({"type":"ping"}))
        pong = json.loads(await ws.recv())
        print("pong:", pong["type"])
        assert pong["type"] == "pong"
        print("OK")
asyncio.run(main())
PY
```

If `websockets` isn't available in the sandbox, install with `pip install --user websockets`. Expected output: `ack: hello_ack` then `pong: pong` then `OK`.

(Do NOT send `wake` from the smoke — it would open a real OpenAI session and burn budget for no return. The Plan 3 client exercises that end-to-end.)

- [ ] **Step 5: Update README with Plan 2 surface area**

Append the following section to `README.md`:

```markdown
## Plan 2: device hub + Realtime relay

- `wss://<domain>/device` — bearer-token WebSocket. Pair a device first to get a token.
- Handshake: client sends `{type:"hello", device_id, client_version}`, backend replies `hello_ack` with per-device config.
- Heartbeat: client sends `{type:"ping"}` every 15 s, backend replies `pong` and updates `Devices.LastSeenAt`.
- Wake: `{type:"wake"}` opens a Realtime session against OpenAI; backend emits `session_started`, then forwards `response.audio.delta` as binary, `response.audio_transcript.done` → assistant turn, etc. On `end_session` tool, idle timeout, or upstream error, emits `session_ended`.
- Tools shipped: `get_current_time` (server-side), `end_session` (closes session), `set_volume` (round-trips to Pi).

The Python Pi client (Plan 3) is the production consumer of this surface. The handshake-only smoke recipe in this README is enough to confirm the deploy is alive without paying OpenAI.
```

- [ ] **Step 6: Commit**

```bash
git add README.md
git commit -m "README: document Plan 2 device-hub surface area"
```

---

## Self-review notes

- **Spec coverage:**
  - DeviceHub WS + auth → Tasks 16, 17, 18
  - Realtime relay (open / session.update / audio relay / transcripts / tools / end_session / idle) → Tasks 9–14
  - Tools registry + 3 built-ins → Tasks 3, 4, 5
  - Conversations + Turns data model + log → Tasks 1, 2
  - OpenAI key from env → Task 6 + Coolify env var (Task 19)
  - Backend → device wire envelopes (`hello_ack`, `session_started`, `assistant_done`, `session_ended`, `tool_call`, `error`) → Task 16
  - Pi-side tool round-trip (`tool_call`/`tool_result`) → Tasks 5, 16, 18
  - Idle timeout reset on speech_started → Task 14
  - end_session sequencing (close only after response.done) → Tasks 12, 13
  - Conversation EndReason persisted → Tasks 1, 2, 9
  - Deploy fire-and-forget via Coolify v4 API → Task 19

- **Deferred (explicit, per the design spec section "Open items deferred to implementation plans"):**
  - Final voice / model defaults remain as currently seeded in `SystemSettings`; pick a different default by updating that singleton row in Plan 4's admin UI work.
  - `play_audio` push-from-server flow has envelope types (`HubEnvelopes`) but no production path; activate in v1.1 when per-device audio is added.
  - The Python client itself (`client/`) is still the Plan 1 placeholder; full implementation in Plan 3.

- **Type / name consistency:**
  - `RealtimeSettings` fields: `Model`, `Voice`, `SystemPrompt`, `IdleTimeoutSeconds`, `EnabledTools` — used the same in Tasks 7, 9, 10, 11, 12, 14, 15, 18.
  - `ToolResult.Output` is a `JsonElement` — preserved across Tasks 3, 4, 5, 12.
  - `IDeviceChannel.CallPiToolAsync(name, args, ct)` signature is identical in Tasks 3, 5, 16.
  - `RealtimeSession` ctor argument order matches between Task 9, Task 15 factory, and Task 18 fake factory.
  - `ConversationLog` methods: `StartAsync`, `AppendUserAsync`, `AppendAssistantAsync`, `AppendToolAsync`, `EndAsync` — same set used in Tasks 2, 11, 12, 13.

- **Plan 3 and 4 hand-off:**
  - Plan 3 (Python client) implements the wire protocol defined in spec § "Wire protocol" and Task 16's envelope set. It is the integration test for everything here.
  - Plan 4 (UI) adds the `DeviceConfig` edit endpoint + an admin push of `config_updated`. The relay already takes effect on the NEXT wake (it reads `DeviceConfig` fresh in Task 18 `StartSessionAsync`), so the design constraint "changes take effect on the next session" is already honoured without any extra wiring needed in Plan 2.