Test 4a: implementation plan

Five tasks: scaffold csproj → write full Program.cs (WS + audio
streams + channels + server VAD round-trip) → bin/probe-cs-4a with
~/.openai_key precondition → one-time Pi key setup → hardware
verification against the spec's 8 pass criteria.
This commit is contained in:
2026-06-12 17:34:25 +00:00
parent dea126f994
commit e96b6ffd95
@@ -0,0 +1,897 @@
# Test 4a — OpenAI Realtime API one-shot voice round-trip (C#) Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Build a single-binary throwaway C# probe at `tests/04a-realtime-oneshot-cs/` that has one round-trip spoken conversation with the OpenAI Realtime API: boot → beep → user speaks one sentence → server VAD ends the turn → assistant audio reply plays back through the USB Speaker Phone → exit 0.
**Architecture:** One persistent `ClientWebSocket` to `wss://api.openai.com/v1/realtime?model=gpt-realtime`. Two `Task.Run` loops: ReceiveLoop parses server events with `JsonDocument` and writes decoded audio chunks to a `Channel<short[]>`; UpstreamSendLoop drains a separate `Channel<short[]>` of mic frames and sends them as `input_audio_buffer.append`. PortAudio input stream (24 kHz mono Int16, 1920 frames/cb) writes to the upstream channel via callback; output stream (24 kHz, 1024 frames/cb) drains the downstream channel into the speaker, zero-filling underruns. Completion (`response.done` received AND downstream channel drained AND working chunk empty) is signaled via `ManualResetEventSlim` to the main thread, which closes the WS and exits.
**Tech Stack:** .NET 9 (self-contained linux-arm64), PortAudioSharp2 1.0.6, `System.Net.WebSockets.ClientWebSocket`, `System.Text.Json`, `System.Threading.Channels`. Deploy via `dotnet publish` + `scp` + `ssh -t` to Pi at `pi@192.168.50.115`. No onnxruntime — no wakeword in 4a.
**Spec:** `docs/superpowers/specs/2026-06-12-test-4a-openai-realtime-design.md`. Read it before starting.
**Branch:** `fresh-start` (already current; do NOT create a new branch).
---
### Task 1: Scaffold `tests/04a-realtime-oneshot-cs/` project skeleton
**Files:**
- Create: `tests/04a-realtime-oneshot-cs/Probe.csproj`
- [ ] **Step 1: Create the project directory**
```bash
mkdir -p tests/04a-realtime-oneshot-cs
```
- [ ] **Step 2: Write `Probe.csproj`**
Create `tests/04a-realtime-oneshot-cs/Probe.csproj` with this exact content. Note the differences from Test 3's csproj: `RootNamespace=RealtimeOneShotProbe`, no `Microsoft.ML.OnnxRuntime` reference (no wakeword in 4a), no `models/*.onnx` content block.
```xml
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net9.0</TargetFramework>
<RootNamespace>RealtimeOneShotProbe</RootNamespace>
<AssemblyName>Probe</AssemblyName>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>
<SelfContained>true</SelfContained>
<PublishSingleFile>false</PublishSingleFile>
<InvariantGlobalization>true</InvariantGlobalization>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="PortAudioSharp2" Version="1.0.6" />
</ItemGroup>
</Project>
```
`ClientWebSocket` (from `System.Net.WebSockets`), `JsonDocument` / `JsonSerializer` (from `System.Text.Json`), and `Channel<T>` (from `System.Threading.Channels`) all come from the BCL — no PackageReference needed.
- [ ] **Step 3: Commit the scaffold**
```bash
git add tests/04a-realtime-oneshot-cs/Probe.csproj
git commit -m "Test 4a scaffold: csproj for tests/04a-realtime-oneshot-cs"
```
---
### Task 2: Write `Program.cs` in full
**Files:**
- Create: `tests/04a-realtime-oneshot-cs/Program.cs`
The whole file is shown below. It is self-contained — no helper files, no copied modules. The structure follows Test 3's `Program.cs` shape (top-level statements; helpers, classes, and `static class Libc` at the bottom of the file) with these notable differences:
- No onnxruntime, no `WakewordModel`, no models directory.
- A `ClientWebSocket` + two background `Task.Run` loops (ReceiveLoop, UpstreamSendLoop) replace the consumer-thread state machine.
- The input stream callback's job is to push mic frames to a `Channel<short[]>` (not to call `Predict`).
- The output stream callback drains a different `Channel<short[]>`, zero-fills underruns, and signals completion via a `ManualResetEventSlim` (replacing Test 3's per-cycle `PlaybackDoneFlag`).
- A `SharedState` class holds the three `volatile bool` flags (`MicArmed`, `StopSending`, `NoMoreDeltas`) plus the output callback's per-call drain state (`CurrentChunk`, `ChunkOffset`).
- `LoadApiKey()` reads `~/.openai_key` with mode-600 validation.
- [ ] **Step 1: Write `Program.cs` in full**
Create `tests/04a-realtime-oneshot-cs/Program.cs` with exactly this content:
```csharp
using System.Net.WebSockets;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.Json;
using System.Threading.Channels;
using PortAudioSharp;
using Stream = PortAudioSharp.Stream;
// .NET-side mirror for any readers that go through Environment.GetEnvironmentVariable,
// AND libc setenv so PortAudio (which uses getenv()) sees the value.
Environment.SetEnvironmentVariable("PA_ALSA_PLUGHW", "1");
Libc.setenv("PA_ALSA_PLUGHW", "1", 1);
string apiKey = LoadApiKey();
PortAudio.Initialize();
using var cts = new CancellationTokenSource();
Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
var upstreamChannel = Channel.CreateBounded<short[]>(new BoundedChannelOptions(C.UpstreamChannelCapacity)
{
FullMode = BoundedChannelFullMode.Wait,
SingleWriter = true,
SingleReader = true,
});
var downstreamChannel = Channel.CreateBounded<short[]>(new BoundedChannelOptions(C.DownstreamChannelCapacity)
{
FullMode = BoundedChannelFullMode.Wait,
SingleWriter = false,
SingleReader = true,
});
var doneFlag = new ManualResetEventSlim(false);
var state = new SharedState();
var ws = new ClientWebSocket();
ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
ws.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
Stream? inputStream = null;
Stream? outputStream = null;
Task? receiveTask = null;
Task? sendTask = null;
try
{
var uri = new Uri($"wss://api.openai.com/v1/realtime?model={C.ModelName}");
Console.WriteLine($"[ws] connecting to {uri}");
try
{
await ws.ConnectAsync(uri, cts.Token);
}
catch (Exception ex)
{
Console.Error.WriteLine($"[error] WS connect failed: {ex.Message}");
if (ex.InnerException is HttpRequestException httpEx)
Console.Error.WriteLine($"[error] inner: HTTP {(int?)httpEx.StatusCode}");
Environment.ExitCode = 3;
return;
}
Console.WriteLine("[ws] connected");
receiveTask = Task.Run(() => ReceiveLoop(ws, downstreamChannel, cts, state));
await SendSessionUpdate(ws, cts.Token);
int device = FindUsbDevice();
Console.WriteLine($"Using device {device} ('{PortAudio.GetDeviceInfo(device).name}')");
inputStream = OpenInputStream(device, upstreamChannel, state);
try { inputStream.Start(); }
catch (Exception ex)
{
Console.Error.WriteLine($"[error] input stream start failed: {ex.Message}");
Environment.ExitCode = 5;
return;
}
outputStream = OpenOutputStream(device, downstreamChannel, state, doneFlag);
try { outputStream.Start(); }
catch (Exception ex)
{
Console.Error.WriteLine($"[error] output stream start failed: {ex.Message}");
Environment.ExitCode = 6;
return;
}
sendTask = Task.Run(() => UpstreamSendLoop(ws, upstreamChannel, cts, state));
// Startup beep — queued onto downstreamChannel before the receive loop produces any audio.
short[] beep = MakeBeep(C.BeepHz, C.BeepSeconds, C.SampleRate);
Console.WriteLine("[beep]");
await downstreamChannel.Writer.WriteAsync(beep, cts.Token);
// Wait for beep to drain (200 ms beep + 100 ms margin) so it doesn't get captured
// by the mic and shipped to OpenAI.
Thread.Sleep((int)(C.BeepSeconds * 1000) + C.BeepDrainMs);
state.MicArmed = true;
Console.WriteLine("Speak now.");
try
{
doneFlag.Wait(cts.Token);
}
catch (OperationCanceledException) { /* Ctrl-C path */ }
Console.WriteLine("[ws] closing");
}
finally
{
// Cancel any background work first so receive/send loops exit cleanly.
cts.Cancel();
if (ws.State == WebSocketState.Open || ws.State == WebSocketState.CloseReceived)
{
try
{
using var closeTimeout = new CancellationTokenSource(TimeSpan.FromSeconds(2));
await ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "done", closeTimeout.Token);
}
catch (Exception ex)
{
Console.Error.WriteLine($"[ws] close failed: {ex.Message}");
}
}
try { inputStream?.Stop(); } catch { }
try { outputStream?.Stop(); } catch { }
try { inputStream?.Dispose(); } catch { }
try { outputStream?.Dispose(); } catch { }
try { ws.Dispose(); } catch { }
try { PortAudio.Terminate(); } catch { }
Console.WriteLine("bye");
}
static string LoadApiKey()
{
string home = Environment.GetEnvironmentVariable("HOME") ?? "/root";
string path = Path.Combine(home, ".openai_key");
if (!File.Exists(path))
{
Console.Error.WriteLine($"[error] API key file not found at {path}");
Environment.Exit(2);
}
// Non-fatal mode-600 check.
try
{
var mode = File.GetUnixFileMode(path);
var leaky = mode & (UnixFileMode.GroupRead | UnixFileMode.GroupWrite | UnixFileMode.GroupExecute
| UnixFileMode.OtherRead | UnixFileMode.OtherWrite | UnixFileMode.OtherExecute);
if (leaky != UnixFileMode.None)
Console.Error.WriteLine($"[warn] {path} mode includes group/other perms ({mode}); expected 600");
}
catch (PlatformNotSupportedException) { /* not Unix */ }
catch (Exception ex)
{
Console.Error.WriteLine($"[warn] could not check {path} mode: {ex.Message}");
}
string key = File.ReadAllText(path).Trim();
if (string.IsNullOrEmpty(key))
{
Console.Error.WriteLine($"[error] API key file empty");
Environment.Exit(2);
}
if (!key.StartsWith("sk-"))
{
Console.Error.WriteLine($"[error] API key doesn't start with sk-");
Environment.Exit(2);
}
return key;
}
static int FindUsbDevice()
{
int count = PortAudio.DeviceCount;
for (int i = 0; i < count; i++)
{
var info = PortAudio.GetDeviceInfo(i);
if (info.name.ToLowerInvariant().Contains("usb") && info.maxInputChannels >= 1)
return i;
}
Console.Error.WriteLine("USB audio device not found. Devices:");
for (int i = 0; i < count; i++)
{
var info = PortAudio.GetDeviceInfo(i);
Console.Error.WriteLine(
$" [{i}] {info.name} in={info.maxInputChannels} out={info.maxOutputChannels}");
}
Environment.Exit(1);
return -1; // unreachable
}
static short[] MakeBeep(double freqHz, double durationS, int sampleRate)
{
int n = (int)(sampleRate * durationS);
var buf = new short[n];
for (int i = 0; i < n; i++)
{
double t = i / (double)sampleRate;
double v = 0.3 * Math.Sin(2.0 * Math.PI * freqHz * t);
buf[i] = (short)(v * short.MaxValue);
}
return buf;
}
static async Task SendSessionUpdate(ClientWebSocket ws, CancellationToken ct)
{
var msg = new
{
type = "session.update",
session = new
{
modalities = new[] { "audio", "text" },
instructions = C.Instructions,
voice = C.Voice,
input_audio_format = "pcm16",
output_audio_format = "pcm16",
turn_detection = new
{
type = "server_vad",
threshold = 0.5,
prefix_padding_ms = 300,
silence_duration_ms = 500,
create_response = true,
},
},
};
byte[] bytes = JsonSerializer.SerializeToUtf8Bytes(msg);
await ws.SendAsync(new ArraySegment<byte>(bytes), WebSocketMessageType.Text, endOfMessage: true, ct);
}
static Stream OpenInputStream(int device, Channel<short[]> upstream, SharedState state)
{
var inParams = new StreamParameters
{
device = device,
channelCount = C.Channels,
sampleFormat = SampleFormat.Int16,
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowInputLatency,
hostApiSpecificStreamInfo = IntPtr.Zero,
};
Stream.Callback callback = (IntPtr input, IntPtr _, uint frameCount,
ref StreamCallbackTimeInfo _2, StreamCallbackFlags status, IntPtr _3) =>
{
if (status.HasFlag(StreamCallbackFlags.InputOverflow))
Console.Error.WriteLine("[status] input overflow");
if (!state.MicArmed || state.StopSending)
return StreamCallbackResult.Continue;
if (frameCount != C.MicBlockFrames)
{
Console.Error.WriteLine($"[status] unexpected input frameCount={frameCount} (want {C.MicBlockFrames})");
return StreamCallbackResult.Continue;
}
var frame = new short[C.MicBlockFrames];
unsafe
{
short* src = (short*)input.ToPointer();
fixed (short* dst = &frame[0])
Buffer.MemoryCopy(src, dst, C.MicBlockFrames * sizeof(short), C.MicBlockFrames * sizeof(short));
}
if (!upstream.Writer.TryWrite(frame))
Console.Error.WriteLine("[status] upstream behind, dropping mic frame");
return StreamCallbackResult.Continue;
};
return new Stream(inParams, null, C.SampleRate, C.MicBlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
}
static Stream OpenOutputStream(int device, Channel<short[]> downstream, SharedState state, ManualResetEventSlim doneFlag)
{
var outParams = new StreamParameters
{
device = device,
channelCount = C.Channels,
sampleFormat = SampleFormat.Int16,
suggestedLatency = PortAudio.GetDeviceInfo(device).defaultLowOutputLatency,
hostApiSpecificStreamInfo = IntPtr.Zero,
};
Stream.Callback callback = (IntPtr _, IntPtr output, uint frameCount,
ref StreamCallbackTimeInfo _2, StreamCallbackFlags _3, IntPtr _4) =>
{
int fill = 0;
while (fill < frameCount)
{
// Pull next chunk if we have no working one.
if (state.CurrentChunk == null)
{
if (!downstream.Reader.TryRead(out var nextChunk))
break; // ring empty
state.CurrentChunk = nextChunk;
state.ChunkOffset = 0;
}
// Local non-null alias so the compiler doesn't warn on dereference inside `fixed`.
short[] chunk = state.CurrentChunk!;
int slack = (int)frameCount - fill;
int remaining = chunk.Length - state.ChunkOffset;
int take = Math.Min(remaining, slack);
unsafe
{
short* dst = (short*)output.ToPointer();
fixed (short* src = &chunk[state.ChunkOffset])
Buffer.MemoryCopy(src, dst + fill, take * sizeof(short), take * sizeof(short));
}
state.ChunkOffset += take;
fill += take;
if (state.ChunkOffset >= chunk.Length)
state.CurrentChunk = null;
}
unsafe
{
short* dst = (short*)output.ToPointer();
for (int i = fill; i < (int)frameCount; i++) dst[i] = 0;
}
// Completion = working chunk drained AND channel empty AND writer completed.
if (state.CurrentChunk == null
&& downstream.Reader.Count == 0
&& downstream.Reader.Completion.IsCompleted)
{
doneFlag.Set();
return StreamCallbackResult.Complete;
}
return StreamCallbackResult.Continue;
};
return new Stream(null, outParams, C.SampleRate, C.OutBlockFrames, StreamFlags.NoFlag, callback, IntPtr.Zero);
}
static async Task UpstreamSendLoop(ClientWebSocket ws, Channel<short[]> upstream, CancellationTokenSource cts, SharedState state)
{
try
{
await foreach (var frame in upstream.Reader.ReadAllAsync(cts.Token))
{
if (state.StopSending) continue;
ReadOnlySpan<byte> bytes = MemoryMarshal.AsBytes(frame.AsSpan());
string base64 = Convert.ToBase64String(bytes);
var msg = new { type = "input_audio_buffer.append", audio = base64 };
byte[] json = JsonSerializer.SerializeToUtf8Bytes(msg);
await ws.SendAsync(new ArraySegment<byte>(json), WebSocketMessageType.Text, endOfMessage: true, cts.Token);
}
}
catch (OperationCanceledException) { /* expected on Cancel */ }
catch (Exception ex)
{
Console.Error.WriteLine($"[error] upstream send: {ex.Message}");
cts.Cancel();
}
}
static async Task ReceiveLoop(ClientWebSocket ws, Channel<short[]> downstream, CancellationTokenSource cts, SharedState state)
{
var transcript = new StringBuilder();
var buffer = new byte[4096];
bool firstDeltaSeen = false;
DateTime speechStoppedTime = DateTime.MinValue;
using var msgBuffer = new MemoryStream();
try
{
while (!cts.IsCancellationRequested)
{
msgBuffer.SetLength(0);
WebSocketReceiveResult result;
do
{
result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), cts.Token);
if (result.MessageType == WebSocketMessageType.Close) return;
msgBuffer.Write(buffer, 0, result.Count);
} while (!result.EndOfMessage);
msgBuffer.Position = 0;
using var doc = JsonDocument.Parse(msgBuffer);
string type = doc.RootElement.GetProperty("type").GetString() ?? "";
switch (type)
{
case "session.created":
{
string? sid = doc.RootElement.TryGetProperty("session", out var sess)
&& sess.TryGetProperty("id", out var idEl) ? idEl.GetString() : null;
Console.WriteLine($"[ws] session.created {sid}");
break;
}
case "session.updated":
Console.WriteLine("[ws] session.updated — session pinned");
break;
case "input_audio_buffer.speech_started":
Console.WriteLine("[vad] speech started");
break;
case "input_audio_buffer.speech_stopped":
Console.WriteLine("[vad] speech stopped");
speechStoppedTime = DateTime.UtcNow;
state.StopSending = true;
break;
case "input_audio_buffer.committed":
Console.WriteLine("[ws] input_audio_buffer.committed");
break;
case "response.created":
Console.WriteLine("[ws] response.created");
break;
case "response.audio_transcript.delta":
if (doc.RootElement.TryGetProperty("delta", out var tEl))
transcript.Append(tEl.GetString());
break;
case "response.audio.delta":
{
string b64 = doc.RootElement.GetProperty("delta").GetString() ?? "";
byte[] audioBytes = Convert.FromBase64String(b64);
short[] samples = new short[audioBytes.Length / 2];
Buffer.BlockCopy(audioBytes, 0, samples, 0, audioBytes.Length);
if (!firstDeltaSeen)
{
firstDeltaSeen = true;
double elapsed = speechStoppedTime == DateTime.MinValue
? -1
: (DateTime.UtcNow - speechStoppedTime).TotalSeconds;
Console.WriteLine($"[audio] first delta received (t = {elapsed:0.00} s since speech_stopped)");
}
await downstream.Writer.WriteAsync(samples, cts.Token);
break;
}
case "response.audio.done":
Console.WriteLine("[ws] response.audio.done");
break;
case "response.done":
state.NoMoreDeltas = true;
downstream.Writer.Complete();
Console.WriteLine("[ws] response.done");
Console.WriteLine($"[transcript] {transcript}");
break;
case "error":
Console.Error.WriteLine($"[error] WS error event: {doc.RootElement.GetRawText()}");
cts.Cancel();
break;
default:
Console.Error.WriteLine($"[ws] ignored type={type}");
break;
}
}
}
catch (OperationCanceledException) { /* expected on Cancel */ }
catch (Exception ex)
{
Console.Error.WriteLine($"[error] receive loop: {ex.Message}");
cts.Cancel();
}
}
sealed class SharedState
{
public volatile bool MicArmed;
public volatile bool StopSending;
public volatile bool NoMoreDeltas;
// Output-callback-thread-local; do NOT touch from anywhere else.
public short[]? CurrentChunk;
public int ChunkOffset;
}
static class C
{
public const int SampleRate = 24_000;
public const int Channels = 1;
public const uint MicBlockFrames = 1920; // 40 ms @ 24 kHz
public const uint OutBlockFrames = 1024; // ~43 ms @ 24 kHz
public const string ModelName = "gpt-realtime";
public const string Voice = "alloy";
public const string Instructions = "Reply in one short sentence.";
public const double BeepHz = 880.0;
public const double BeepSeconds = 0.2;
public const int BeepDrainMs = 100;
public const int UpstreamChannelCapacity = 64;
public const int DownstreamChannelCapacity = 64;
}
static class Libc
{
[DllImport("libc", EntryPoint = "setenv")]
public static extern int setenv(string name, string value, int overwrite);
}
```
- [ ] **Step 2: Verify the project builds cleanly**
```bash
dotnet build tests/04a-realtime-oneshot-cs -c Release
```
Expected: `Build succeeded` with 0 errors. A handful of nullable-reference warnings on `state.CurrentChunk` access inside the `unsafe` block is fine if it appears; the runtime checks above ensure non-null.
If `<AllowUnsafeBlocks>` is missing from the csproj, the build will fail with `CS0227: Unsafe code may only appear if compiling with /unsafe`. Confirm Task 1's csproj has `<AllowUnsafeBlocks>true</AllowUnsafeBlocks>`.
If you see `CS0246: The type or namespace 'Stream' could not be found` or a clash with `System.IO.Stream`, confirm the `using Stream = PortAudioSharp.Stream;` line is at the top of `Program.cs`.
If you see `error CS8803: Top-level statements must precede namespace and type declarations` or `CS8805: Program using top-level statements must be an entry point`, the file ordering is wrong. The exact required order in `Program.cs` is:
1. `using` directives
2. Top-level statements (the imperative body — `Environment.SetEnvironmentVariable`, `apiKey = LoadApiKey()`, `PortAudio.Initialize`, the `try {} finally {}` block)
3. Top-level static method declarations (`LoadApiKey`, `FindUsbDevice`, `MakeBeep`, `SendSessionUpdate`, `OpenInputStream`, `OpenOutputStream`, `UpstreamSendLoop`, `ReceiveLoop`)
4. Top-level type declarations (`sealed class SharedState`, `static class C`, `static class Libc`) — NOT wrapped in a `namespace` block.
The file above follows this order; do not reorder.
- [ ] **Step 3: Verify `dotnet publish` produces a runnable binary**
```bash
dotnet publish tests/04a-realtime-oneshot-cs -c Release -r linux-arm64 --self-contained -o /tmp/probe-cs-4a-out
ls /tmp/probe-cs-4a-out/Probe
```
Expected:
- `Probe` (executable) exists.
- No `models/` directory needed.
- [ ] **Step 4: Commit**
```bash
git add tests/04a-realtime-oneshot-cs/Program.cs
git commit -m "Test 4a: WS plumbing + audio streams + server VAD round-trip"
```
---
### Task 3: Write `bin/probe-cs-4a` deploy script with API key precondition
**Files:**
- Create: `bin/probe-cs-4a`
This is `bin/probe-cs-3` with three string swaps (`03-full-cycle-cs``04a-realtime-oneshot-cs`, `probe-cs-3``probe-cs-4a`, `/tmp/probe-cs-3-out``/tmp/probe-cs-4a-out`) plus ONE new precondition step: check that `~/.openai_key` exists on the Pi before publishing. This catches the silent "key missing" failure mode at the deploy step instead of 30 s into publish + scp.
- [ ] **Step 1: Write the deploy script**
Create `bin/probe-cs-4a` with exactly this content:
```bash
#!/usr/bin/env bash
set -euo pipefail
PI_HOST="${PI_HOST:-192.168.50.115}"
PI_USER="${PI_USER:-pi}"
PI_PASS="${PI_PASS:-assistant}"
PUBLISH_DIR="${PUBLISH_DIR:-/tmp/probe-cs-4a-out}"
repo_root="$(cd "$(dirname "$0")/.." && pwd)"
cd "$repo_root"
echo ">> precondition: ~/.openai_key present on Pi"
if ! sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'test -f ~/.openai_key && test -r ~/.openai_key'; then
echo "ERROR: ~/.openai_key missing or unreadable on Pi."
echo "Create it on the Pi with:"
echo " sshpass -p '$PI_PASS' ssh $PI_USER@$PI_HOST \\"
echo " 'umask 077 && printf \"%s\\n\" \"sk-PASTE-YOUR-KEY\" > ~/.openai_key'"
exit 1
fi
echo " OK."
echo ">> dotnet publish (linux-arm64, self-contained)"
dotnet publish tests/04a-realtime-oneshot-cs \
-c Release -r linux-arm64 --self-contained \
-o "$PUBLISH_DIR"
echo ">> scp to $PI_USER@$PI_HOST:~/probe-cs-4a/ (wipe-and-replace)"
sshpass -p "$PI_PASS" ssh -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'rm -rf ~/probe-cs-4a && mkdir ~/probe-cs-4a'
sshpass -p "$PI_PASS" scp -r "$PUBLISH_DIR/." \
"$PI_USER@$PI_HOST:~/probe-cs-4a/"
echo ">> ssh + run on Pi (Ctrl-C from this terminal stops the probe)"
sshpass -p "$PI_PASS" ssh -t -o StrictHostKeyChecking=accept-new \
"$PI_USER@$PI_HOST" 'cd ~/probe-cs-4a && chmod +x Probe && ./Probe'
```
- [ ] **Step 2: Make it executable**
```bash
chmod +x bin/probe-cs-4a
```
- [ ] **Step 3: Commit**
```bash
git add bin/probe-cs-4a
git commit -m "Test 4a: bin/probe-cs-4a deploy script with ~/.openai_key precondition"
```
---
### Task 4: One-time Pi setup — place `~/.openai_key` on the Pi
**Files:** (none — this is a one-time remote setup task)
This task only runs once per Pi. If the file already exists from a previous deploy or another test, skip steps 12 and verify with step 3.
- [ ] **Step 1: Ask the user for an API key**
Before running this task, the user must have an OpenAI API key with Realtime API access (`sk-…`). If the user hasn't given it to you yet, ask:
> "I need your OpenAI API key (`sk-…`) to write it to `~/.openai_key` on the Pi. Paste it and I'll redact it from the shell history immediately. Do not paste it into any other context."
The key the user provides MUST be treated as secret and MUST NOT appear in commit messages, file contents anywhere in the repo, console echoes, or logs. The only place it goes is the file on the Pi.
- [ ] **Step 2: Write the key to the Pi**
Replace `sk-PASTE-YOUR-KEY-HERE` below with the actual key value. The `umask 077` ensures the file is created with mode 600 (rw for owner only). The `printf "%s\n"` appends exactly one newline.
```bash
sshpass -p 'assistant' ssh -o StrictHostKeyChecking=accept-new \
pi@192.168.50.115 \
'umask 077 && printf "%s\n" "sk-PASTE-YOUR-KEY-HERE" > ~/.openai_key'
```
- [ ] **Step 3: Verify file exists and is mode 600**
```bash
sshpass -p 'assistant' ssh pi@192.168.50.115 \
'ls -l ~/.openai_key && wc -c ~/.openai_key'
```
Expected output (mode `-rw-------`, owner `pi`, size > 50 bytes):
```
-rw------- 1 pi pi <size> <date> /home/pi/.openai_key
<size> /home/pi/.openai_key
```
If mode is not `-rw-------`, fix with `chmod 600 ~/.openai_key`.
If size is suspiciously small (< 20 bytes), the key wasn't pasted correctly — re-do step 2.
(No commit — this step modifies state on the Pi, not the repo.)
---
### Task 5: Hardware verification on the Pi
This task is the actual pass/fail gate. No automated tests; verification is the human ear + console output. If anything in this task fails, do NOT mark the implementation complete — investigate and either fix in code or update the spec to reflect what actually works.
**Files:** (none — this is a runtime verification task)
- [ ] **Step 1: Confirm the Pi is reachable**
Run from workstation:
```bash
ping -c 2 192.168.50.115
```
Expected: 2/2 replies. If unreachable, stop and ask the user.
- [ ] **Step 2: Deploy + run the probe**
Run from workstation:
```bash
./bin/probe-cs-4a
```
Expected console output (abridged, with elision):
```
>> precondition: ~/.openai_key present on Pi
OK.
>> dotnet publish (linux-arm64, self-contained)
... (build output) ...
>> scp to pi@192.168.50.115:~/probe-cs-4a/ (wipe-and-replace)
>> ssh + run on Pi (Ctrl-C from this terminal stops the probe)
[ws] connecting to wss://api.openai.com/v1/realtime?model=gpt-realtime
[ws] connected
[ws] session.created sess_xxxxxxxxxxxx
[ws] session.updated — session pinned
Using device <N> ('USB Speaker Phone (LISTENAI ...)')
[beep]
Speak now.
```
At this point the probe is waiting for you to speak. Say a single short question into the USB Speaker Phone — for example: **"What's the capital of France?"** — at normal volume from ~1 m.
Continue with verification in step 3.
If the publish step errors with a runtime crash invoking `Libc.setenv`, confirm Task 2's `[DllImport]` line for `setenv` is present.
If the Pi run errors with `Cannot open shared library libportaudio.so`, the self-contained publish didn't bundle the native libs — confirm `<SelfContained>true</SelfContained>` and `<RuntimeIdentifier>linux-arm64</RuntimeIdentifier>` in Task 1's csproj.
If `[error] WS connect failed: ... HTTP 401` appears, the API key was rejected. Re-check Task 4 step 3 and confirm the key on the Pi starts with `sk-` and the user pasted the right value.
- [ ] **Step 3: Verify the full conversational round-trip**
After saying your question in step 2, observe:
1. `[vad] speech started` appears in the console as you begin speaking.
2. `[vad] speech stopped` appears shortly after you stop speaking (within ~0.51 s, governed by `silence_duration_ms = 500` in session.update).
3. `[ws] input_audio_buffer.committed` and `[ws] response.created` follow.
4. `[audio] first delta received (t = X.XX s since speech_stopped)` appears with a small latency value (~12 s).
5. The assistant's spoken reply comes out of the USB Speaker Phone speaker — clearly, intelligibly, recognisable as English speech.
6. `[ws] response.audio.done` and `[ws] response.done` follow.
7. `[transcript] <the assistant's reply, as text>` appears — should match what you heard.
8. `[ws] closing` and `bye` appear; the probe exits.
**Pass criterion 1:** End-to-end round-trip works once. Probe exits 0.
**Pass criterion 2:** Response is intelligible (no clipping, no dropouts longer than one ~43 ms callback).
**Pass criterion 8:** Probe exits within ~500 ms of the last audio sample being heard.
- [ ] **Step 4: Check latency from the log line**
Look at the `[audio] first delta received (t = X.XX s since speech_stopped)` line printed in step 3.
**Pass criterion 3:** `X.XX` is under ~2 s. Slower is acceptable if logged here; over ~5 s is a fail.
- [ ] **Step 5: Scan for warnings**
Re-read the full console output. Confirm there are **no** lines of these forms during normal operation (between `[beep]` and `[ws] response.done`):
- `[status] upstream behind, dropping mic frame`
- `[status] input overflow`
- `[status] unexpected input frameCount=...`
- `[error] WS ...`
- `[ws] ignored type=...` (one or two of these are OK — the API may send events we don't handle; sustained occurrences during normal flow indicate a missing handler we should add)
**Pass criterion 4:** No `[status] upstream behind` warnings.
**Pass criterion 5:** No `[error] WS …` events.
**Pass criterion 6:** No PortAudio `InputOverflow` status flags.
- [ ] **Step 6: Test Ctrl-C clean exit (separate run)**
Restart the probe:
```bash
./bin/probe-cs-4a
```
When `Speak now.` appears, press Ctrl-C in the workstation terminal **without** speaking.
Expected:
```
^C
[ws] closing
bye
$
```
Verify with `echo $?` — exit code 0.
**Pass criterion 7:** Ctrl-C produces a clean exit (no stack trace, terminal returns to prompt, exit code 0 when invoked during the steady listening phase).
- [ ] **Step 7: Update `findings.md` with the outcome**
Add a new section `## Test 4a outcome (2026-06-12 — OpenAI Realtime one-shot voice round-trip in C#)` at the bottom of `findings.md` summarising:
- Pass / fail per criterion (18).
- Latency measured (the `X.XX` value from step 4).
- Any new gotchas discovered during implementation or hardware verification — especially:
- Any OpenAI Realtime API surface differences from the spec (event types not in our switch, session.update fields rejected, model name issues).
- Any audio-format quirks (24 kHz playback dropouts, sample-rate-mismatch artifacts).
- Any concurrency surprises (channel deadlocks, send-loop wedges).
- Any errors in our LoadApiKey / mode-600 check.
- Implications for 4b (what we still need to debug there vs. what 4a proved out).
- Reference paths (spec + plan + code + deploy script).
Use the existing Test 1, 2, and 3 outcome sections in `findings.md` as the template.
- [ ] **Step 8: Commit the findings update**
```bash
git add findings.md
git commit -m "Findings: Test 4a OpenAI Realtime one-shot voice round-trip outcome"
```
- [ ] **Step 9: Final status report**
Report to the user:
- Pass/fail per criterion (1 through 8 from spec).
- Any criteria that failed and what was discovered.
- The measured round-trip latency value.
- A one-line summary of whether Test 4a unblocks Test 4b (i.e. whether the OpenAI Realtime surface + audio plumbing is proven, so 4b can focus on wakeword + loop + 16↔24 kHz coexistence).
---
## Self-review notes
- **Spec coverage:**
- Goal (single-binary one-shot probe) → Tasks 1+2+3+5.
- `~/.openai_key` precondition + file-on-Pi storage → Task 3 (script) + Task 4 (one-time setup) + Task 2 (`LoadApiKey` with mode-600 warning).
- WS connect with `Authorization` + `OpenAI-Beta` headers → Task 2 (`ws.Options.SetRequestHeader` + `ConnectAsync`).
- `session.update` payload with `server_vad` → Task 2 (`SendSessionUpdate` helper, exact JSON shape).
- Mic input stream at 24 kHz with `_micArmed` gate → Task 2 (`OpenInputStream`, `state.MicArmed` check in callback).
- UpstreamSendLoop (base64 + `input_audio_buffer.append`) → Task 2 (`UpstreamSendLoop`).
- ReceiveLoop with `JsonDocument` switch on `type` → Task 2 (`ReceiveLoop`, complete handler table).
- Output stream drain-or-zero-fill + `_noMoreDeltas` + `Reader.Completion.IsCompleted` completion → Task 2 (`OpenOutputStream` callback).
- Startup beep through `downstreamChannel` → Task 2 (lines after `outputStream.Start()`).
- `ManualResetEventSlim doneFlag` + clean shutdown via `finally` → Task 2 (try/finally block).
- Ctrl-C → `Console.CancelKeyPress``cts.Cancel``OperationCanceledException` → finally → exit 0 → Tasks 2 + 5 step 6.
- `bin/probe-cs-4a` with ssh -t + wipe-and-replace + key precondition → Task 3.
- Pass criteria 18 → Task 5 steps 3, 4, 5, 6.
- Findings update → Task 5 steps 78.
- **Placeholder scan:** no TBD/TODO; every code step shows the actual code; every command shows expected output. The single user-substitution placeholder is `sk-PASTE-YOUR-KEY-HERE` in Task 4 step 2, which is explicitly called out as a replacement target.
- **Type consistency:**
- `SharedState` fields (`MicArmed`, `StopSending`, `NoMoreDeltas`, `CurrentChunk`, `ChunkOffset`) referenced consistently from `OpenInputStream`, `OpenOutputStream`, `UpstreamSendLoop`, `ReceiveLoop`, and the top-level statements.
- `Channel<short[]>` used for both upstream and downstream; the `short[]` element type is consistent everywhere.
- `ManualResetEventSlim doneFlag` passed by reference into `OpenOutputStream` so the callback can `Set()` it; main thread awaits it via `Wait(cts.Token)`.
- `CancellationTokenSource cts` is the single source of cancellation across all three loops + the main thread.
- **Scope:** the spec is a single subsystem (one probe). One plan is the right granularity.