# Test 4a — OpenAI Realtime API one-shot voice round-trip (C#)
A throwaway C# probe that has a single round-trip spoken conversation with the OpenAI Realtime API:
> Boot → beep → user speaks one sentence → server VAD ends the turn → assistant audio reply plays back through the same USB Speaker Phone → exit 0.
No wakeword. No loop. No state machine. One process, runs once, dies.
Test 4a's job is to debug the OpenAI Realtime API surface (auth, WS handshake, `session.update` payload, event JSON shapes, base64 audio decode, server VAD endpointing, streaming-length output-stream cleanup) **in isolation** from the harder concurrency story of Test 4b (wakeword + loop + 16↔24 kHz coexistence).
Read `findings.md` (§ "C# probe outcome", § "C# wakeword probe outcome", § "C# full-cycle outcome") before reading this. Everything below assumes those outcomes as ground truth. The patterns lifted directly from Test 3 are: `Libc.setenv`, the `using Stream = PortAudioSharp.Stream;` import disambiguation, `true`, the fire-and-forget output stream lifetime (Start under try/catch BEFORE the cleanup task launches), `FindUsbDevice`, the `MakeBeep` helper, `bin/probe-cs-3`'s wipe-and-replace publish shape with `ssh -t`.
## Goal
A single-binary C# probe that:
1. Reads the OpenAI API key from `~/.openai_key` on the Pi (mode 600).
2. Connects to `wss://api.openai.com/v1/realtime?model=gpt-realtime` over a WebSocket with `Authorization: Bearer …` + `OpenAI-Beta: realtime=v1` headers.
3. Sends one `session.update` event pinning audio formats, voice, instructions, and `turn_detection: server_vad`.
4. Plays a startup beep through the USB Speaker Phone.
5. Opens a 24 kHz mono Int16 input stream on the same device; streams mic audio to OpenAI as `input_audio_buffer.append` events.
6. Waits for the server VAD to fire `input_audio_buffer.speech_stopped` (server then auto-commits the buffer and creates the response).
7. Stops sending mic audio at that point. Receives `response.audio.delta` chunks, base64-decodes them, plays them through a 24 kHz output stream on the same device.
8. On `response.done` AND the output buffer drained, closes the WebSocket cleanly, terminates PortAudio, exits 0.
9. Exits cleanly on Ctrl-C from the workstation (SIGINT propagated via `ssh -t`) at any stage.
## Non-goals
- **No wakeword.** No `WakewordModel.cs`. No 16 kHz audio path anywhere in 4a. (Test 4b layers that on top of this proven base.)
- **No state machine, no loop.** Probe runs once and exits.
- **No barge-in.** Once `speech_stopped` fires, mic frames are dropped on the floor. The user cannot interrupt the assistant in 4a.
- **No client-side resampling.** Mic = 24 kHz, output = 24 kHz, OpenAI = 24 kHz pcm16. PortAudio plug-resamples 48↔24 at ALSA on the USB device.
- **No WAV capture.** Logging is stdout/stderr only.
- **No retry on WS errors, no reconnect, no health check.** Fail loud, exit non-zero, log enough to diagnose.
- **No automated tests.** Hardware verification only.
## Architecture
```
┌──────────────────────────────────────┐
│ ClientWebSocket │
│ wss://api.openai.com/v1/realtime? │
│ model=gpt-realtime │
│ Headers: Authorization: Bearer … │
│ OpenAI-Beta: realtime=v1 │
└────┬──────────────────────────┬──────┘
│ SendAsync │ ReceiveAsync
▼ ▼
┌──────────────────────────┐ ┌──────────────────────────────┐
│ UpstreamSendLoop │ │ ReceiveLoop (Task.Run) │
│ (Task.Run) │ │ while (!cts.Cancel) { │
│ drain upstreamChannel: │ │ msg = ReceiveAsync(…) │
│ pull short[1920] │ │ doc = JsonDocument.Parse │
│ → base64 │ │ switch (doc["type"]) │
│ → input_audio_ │ │ session.created → log │
│ buffer.append │ │ session.updated → log │
│ → ws.SendAsync │ │ speech_started → log │
│ exits when │ │ speech_stopped → │
│ _stopSending && empty│ │ _stopSending = true │
└──────────┬───────────────┘ │ response.audio.delta: │
│ (FullMode.Wait) │ base64 → short[] │
│ │ → downstreamChannel │
▲ │ .Writer.WriteAsync │
│ short[1920] │ audio_transcript.delta:│
│ (40 ms @ 24 kHz) │ append to log buf │
┌──────────┴───────────────┐ │ response.done → │
│ PortAudio input stream │ │ _noMoreDeltas = true │
│ 24 kHz mono Int16, │ │ downstreamChannel │
│ 1920 frames/cb │ │ .Writer.Complete │
│ callback: │ │ error → log + Cancel │
│ if !_micArmed: discard│ └────────────┬─────────────────┘
│ elif _stopSending: │ │
│ discard │ ▼
│ else: │ ┌──────────────────────────────┐
│ copy → short[] │ │ Channel │
│ TryWrite to │ │ (bounded cap 64, │
│ upstreamChannel │ │ FullMode.Wait) │
└──────────────────────────┘ └────────────┬─────────────────┘
│
▼
┌──────────────────────────────────┐
│ PortAudio output stream │
│ 24 kHz mono Int16, │
│ 1024 frames/cb │
│ callback: │
│ fill = 0 │
│ while fill < frameCount: │
│ if currentChunk == null: │
│ try pull short[] or break │
│ take = min(remaining, slack)│
│ memcpy chunk → output │
│ zero-fill remaining │
│ if currentChunk == null │
│ && Reader.Count == 0 │
│ && Completion.IsCompleted: │
│ doneFlag.Set() │
│ return Complete │
│ return Continue │
└────────────┬─────────────────────┘
│
▼
main thread: doneFlag.Wait
→ ws.CloseAsync(NormalClosure)
→ input/output stream.Stop
→ PortAudio.Terminate
→ exit 0
```
One process. Three managed threads (main + UpstreamSendLoop + ReceiveLoop) plus PortAudio's input and output audio threads (managed by the library).
**Shared mutable state — the full list, small on purpose:**
- `Channel upstreamChannel`, bounded 64, `FullMode.Wait`, single-writer (mic callback), single-reader (UpstreamSendLoop). Holds mic frames waiting to be base64'd and sent.
- `Channel downstreamChannel`, bounded 64, `FullMode.Wait`, **multi-writer** (main thread writes the startup beep; ReceiveLoop writes response audio chunks; temporally separated but Channel's `SingleWriter` flag is a contract not a runtime check, so keep it `false`), single-reader (output callback). Holds decoded audio chunks waiting to be played. Its `Writer.Complete()` (called by ReceiveLoop on `response.done`) is the WS-stream-end signal for the output callback.
- `volatile bool _micArmed`: starts `false`; set `true` after the startup beep has been queued and drained. Mic frames before this point are discarded so the beep doesn't get captured and shipped to OpenAI.
- `volatile bool _stopSending`: set by ReceiveLoop on `input_audio_buffer.speech_stopped`. Read by both the mic callback (which then discards) and UpstreamSendLoop (which then drains its queue and exits).
- `volatile bool _noMoreDeltas`: set by ReceiveLoop on `response.done`. Not strictly required because `downstreamChannel.Writer.Complete()` carries the same signal — but kept as a redundant fast path that the output callback can check without touching the channel.
- `ManualResetEventSlim doneFlag`: set by the output callback when it has drained the channel after completion. Awaited by main.
- `CancellationTokenSource cts`: wired to `Console.CancelKeyPress` + any WS error. Cancels both Task.Run loops; main's `doneFlag.Wait(cts.Token)` honours it.
No locks. No `Interlocked`. The bounded `Channel` handles all the cross-thread audio handoff with the right backpressure semantics.
## WebSocket lifecycle
Six discrete phases, all in `Program.cs` top-level statements + the two `Task.Run` loops:
1. **Load API key.** `LoadApiKey()` helper reads `~/.openai_key`, trims whitespace. Mode 600 verified — non-fatal warning if looser. Fatal error (exit 2) if the file is missing, empty, or doesn't start with `sk-`.
2. **Construct `ClientWebSocket`** with two request headers: `Authorization: Bearer ` and `OpenAI-Beta: realtime=v1`. `ConnectAsync` to `wss://api.openai.com/v1/realtime?model=gpt-realtime` with `cts.Token`. On failure (DNS, TLS, 401, 403, 429, 5xx), log the exception message + any HTTP status carried on `WebSocketException.WebSocketErrorCode` / inner `HttpRequestException.StatusCode`; exit 3.
3. **Launch ReceiveLoop** (`Task.Run`) before sending anything, so we don't miss `session.created`.
4. **Send `session.update`.** First client message; pins the session config (see "Client → server events" below). Don't strictly block on `session.updated` — the receive loop logs it asynchronously — but the spec expects it within ~200 ms.
5. **Open + start PortAudio streams.** Input stream first (24 kHz mono Int16, 1920 frames/cb), then output stream (24 kHz mono Int16, 1024 frames/cb). Both opened on `FindUsbDevice()`. Launch UpstreamSendLoop (`Task.Run`).
6. **Startup beep + arm mic.** Queue a 200 ms 880 Hz beep (`MakeBeep` lifted from Test 3, regenerated at 24 kHz) onto `downstreamChannel`. Sleep ~300 ms (beep + drain margin). Set `_micArmed = true`. Print `Speak now.` to stdout.
7. **Wait for `doneFlag`** on the main thread (with `cts.Token` for Ctrl-C). On wake-up: `ws.CloseAsync(WebSocketCloseStatus.NormalClosure, "done", …)` with a 2 s timeout token. Stop input stream, stop output stream, terminate PortAudio, exit 0.
## Client → server event payloads
Three send-once messages on the main thread, plus a stream of `input_audio_buffer.append` from UpstreamSendLoop. All written with `JsonSerializer.Serialize` over `Dictionary` (lightest thing that doesn't require typed records for a probe).
```json
// (1) Pinned session config — first thing sent after WS connect.
{
"type": "session.update",
"session": {
"modalities": ["audio", "text"],
"instructions": "Reply in one short sentence.",
"voice": "alloy",
"input_audio_format": "pcm16",
"output_audio_format": "pcm16",
"turn_detection": {
"type": "server_vad",
"threshold": 0.5,
"prefix_padding_ms": 300,
"silence_duration_ms": 500,
"create_response": true
}
}
}
// (2) Per mic frame (~40 ms cadence), from UpstreamSendLoop:
{ "type": "input_audio_buffer.append", "audio": "" }
```
No `conversation.item.create` and no `response.create` from the client — server VAD's `create_response: true` makes the server commit and create the response automatically on `speech_stopped`.
`instructions: "Reply in one short sentence."` keeps the response audio bounded (~2-4 s of speech) so a probe run is fast to verify by ear and cheap on quota.
`voice: "alloy"` is the long-standing default voice and is documented as universally available for raw WS sessions.
## Server → client event handlers
ReceiveLoop uses a 4 KB receive buffer + standard `ReceiveAsync` loop with `EndOfMessage` accumulation into a `MemoryStream`. On message end, `JsonDocument.Parse`. Switch on the top-level `type` property's string value:
| `type` | Action |
|-|-|
| `session.created` | log `[ws] session.created ` |
| `session.updated` | log `[ws] session.updated — session pinned` |
| `input_audio_buffer.speech_started` | log `[vad] speech started` |
| `input_audio_buffer.speech_stopped` | log `[vad] speech stopped`; set `_stopSending = true` |
| `input_audio_buffer.committed` | log only — server confirms it sealed the buffer |
| `response.created` | log only |
| `response.audio_transcript.delta` | append `delta` to an accumulating `StringBuilder` |
| `response.audio.delta` | base64-decode `delta` → bytes → reinterpret as `short[]` → `await downstreamChannel.Writer.WriteAsync(samples, cts.Token)`. The `await` is the backpressure path: if the channel is full, this blocks the receive loop, which blocks the WS receive buffer — correct behavior for a slow speaker. |
| `response.audio.done` | log only |
| `response.done` | set `_noMoreDeltas = true`; `downstreamChannel.Writer.Complete()`; print the accumulated transcript |
| `error` | log the full event JSON; `cts.Cancel()` |
| anything else | log `[ws] ignored type=<…>` once (or at debug verbosity), continue |
## Audio buffering — input
- Mic callback runs every 40 ms (1920 samples @ 24 kHz mono Int16, matches `framesPerBuffer = 1920`).
- Callback allocates `short[1920]`, `Buffer.MemoryCopy`s the input pointer, `upstreamChannel.Writer.TryWrite(frame)`.
- On TryWrite failure (channel full = UpstreamSendLoop is behind): log `[status] upstream behind, dropping mic frame`, continue.
- Discard frames when `_micArmed == false` (during startup beep) or `_stopSending == true` (after `speech_stopped`).
- On the `InputOverflow` status flag from PortAudio: log `[status] input overflow`, continue.
UpstreamSendLoop pseudocode:
```
await ReceiveLoop signals session.updated (or just optimistically loop from start)
while (!cts.IsCancellationRequested):
if _stopSending && upstreamChannel.Reader.Count == 0: break
frame = await upstreamChannel.Reader.ReadAsync(cts.Token)
if _stopSending: continue // drop frames captured after server VAD cutoff
bytes = MemoryMarshal.AsBytes(frame.AsSpan())
base64 = Convert.ToBase64String(bytes)
json = JsonSerializer.SerializeToUtf8Bytes(new {
type = "input_audio_buffer.append",
audio = base64
})
await ws.SendAsync(json, WebSocketMessageType.Text, endOfMessage: true, cts.Token)
```
## Audio buffering — output
`downstreamChannel = Channel.CreateBounded(new BoundedChannelOptions(64) { FullMode = BoundedChannelFullMode.Wait, SingleWriter = false, SingleReader = true })`. `SingleWriter = false` because the main thread queues the startup beep AND ReceiveLoop queues response audio chunks (temporally separated, but the flag is a hard contract).
Capacity rationale: 64 chunks × ~40-80 ms per `response.audio.delta` ≈ 2.5-5 s of buffered audio. Comfortable headroom against network jitter without holding silly amounts of memory.
ReceiveLoop writes `short[]` chunks of varying length (one per `response.audio.delta` event). On `response.done`, calls `downstreamChannel.Writer.Complete()`.
Output callback's job is to fill `frameCount` shorts every callback (1024 frames = ~43 ms @ 24 kHz). It maintains a `short[]? _currentChunk` + `int _chunkOffset` across calls. Pseudocode:
```
int fill = 0;
while (fill < frameCount) {
if (_currentChunk == null) {
if (!downstreamChannel.Reader.TryRead(out _currentChunk)) break; // drained
_chunkOffset = 0;
}
int slack = frameCount - fill;
int remaining = _currentChunk.Length - _chunkOffset;
int take = Math.Min(remaining, slack);
unsafe {
fixed (short* src = &_currentChunk[_chunkOffset])
Buffer.MemoryCopy(src, output + fill, take * 2, take * 2);
}
_chunkOffset += take;
fill += take;
if (_chunkOffset >= _currentChunk.Length) _currentChunk = null;
}
// Zero-fill the unfilled tail (underrun mid-response, or post-completion drain).
for (int i = fill; i < frameCount; i++) output[i] = 0;
// Completion check: drained AND writer completed AND no working chunk left.
if (_currentChunk == null
&& downstreamChannel.Reader.Count == 0
&& downstreamChannel.Reader.Completion.IsCompleted)
{
doneFlag.Set();
return StreamCallbackResult.Complete;
}
return StreamCallbackResult.Continue;
```
**Underrun behavior: zero-fill, return Continue.** If `response.done` hasn't arrived yet and the next `response.audio.delta` is just in flight, we play silence for one callback (~43 ms) instead of starving — the stream stays armed, no click on resume.
**This is the streaming-length cleanup discipline** that replaces Test 3's "scale `done.Wait` timeout to the buffer length" trick. Test 3 knew the playback wall-clock up front (`recordBuf.Length / SampleRate`); Test 4a does not, because the assistant audio length is unknown a priori. So the signaling moves from "wait this many seconds" to "wait until (writer-completed AND channel-empty AND working-chunk-empty)." The output callback is the single oracle for this triple condition.
**Startup beep wrinkle:** the beep is queued onto `downstreamChannel` BEFORE the receive loop starts producing. The output callback drains it just like any other audio. The completion check is safe because `Writer.Complete()` is only ever called on `response.done` — the beep getting drained does NOT cause an early completion firing.
## Components and files
```
tests/04a-realtime-oneshot-cs/
├── Probe.csproj
│ net9.0
│ linux-arm64
│ true
│ false
│ true
│ true
│ enable
│ enable
│ RootNamespace=RealtimeOneShotProbe, AssemblyName=Probe
│ PackageReference PortAudioSharp2 Version="1.0.6"
│ (no Microsoft.ML.OnnxRuntime — no wakeword in 4a)
│ System.Net.WebSockets + System.Text.Json come from the BCL; no package.
│
├── Program.cs ~300 lines, top-level statements:
│ 1. Libc.setenv("PA_ALSA_PLUGHW", "1", 1) + Environment mirror
│ (no ORT env vars — no onnxruntime in 4a)
│ 2. PortAudio.Initialize
│ 3. apiKey = LoadApiKey()
│ 4. var cts = new CancellationTokenSource();
│ Console.CancelKeyPress += (_, e) => { e.Cancel = true; cts.Cancel(); };
│ 5. var ws = new ClientWebSocket();
│ ws.Options.SetRequestHeader("Authorization", $"Bearer {apiKey}");
│ ws.Options.SetRequestHeader("OpenAI-Beta", "realtime=v1");
│ await ws.ConnectAsync(uri, cts.Token);
│ 6. Channel upstreamChannel, downstreamChannel (both bounded 64)
│ ManualResetEventSlim doneFlag = new(false);
│ 7. Task receiveTask = Task.Run(() => ReceiveLoop(ws, downstreamChannel,
│ cts, _stopSending, _noMoreDeltas, transcriptBuilder));
│ 8. Send session.update (JsonSerializer.SerializeToUtf8Bytes over a dict)
│ 9. int device = FindUsbDevice();
│ 10. Open + start input stream (24 kHz, 1920 frames/cb)
│ 11. Open + start output stream (24 kHz, 1024 frames/cb)
│ 12. Task sendTask = Task.Run(() => UpstreamSendLoop(ws, upstreamChannel,
│ cts, _stopSending));
│ 13. Queue MakeBeep(880, 0.2, 24000) onto downstreamChannel; sleep 300 ms;
│ _micArmed = true; Console.WriteLine("Speak now.");
│ 14. try { doneFlag.Wait(cts.Token); }
│ catch (OperationCanceledException) { /* Ctrl-C path */ }
│ 15. finally:
│ ws.CloseAsync(NormalClosure, "done", linkedCtsWith2sTimeout.Token)
│ inputStream.Stop; outputStream.Stop;
│ inputStream.Dispose; outputStream.Dispose;
│ PortAudio.Terminate;
│ Helpers (all in Program.cs):
│ - LoadApiKey() — reads ~/.openai_key, perms-warns, sanity
│ - MicCallback — discards unless _micArmed && !_stopSending
│ - OutputCallback — drain-or-zero-fill loop above
│ - MakeBeep(hz, sec, rate) — sin wave Int16 (lifted from Test 3, retuned 24 kHz)
│ - UpstreamSendLoop(...)
│ - ReceiveLoop(...) — JsonDocument switch on "type"
│ - FindUsbDevice() — lifted from Test 3 verbatim
│ - static class Libc { setenv }
│
└── (no models/ dir, no WakewordModel.cs)
bin/probe-cs-4a Copy of bin/probe-cs-3 with substitutions:
tests/03-full-cycle-cs → tests/04a-realtime-oneshot-cs
~/probe-cs-3 → ~/probe-cs-4a
PUBLISH_DIR /tmp/probe-cs-3-out → /tmp/probe-cs-4a-out
ssh -t retained (Ctrl-C propagation).
Wipe-and-replace publish dir retained (partial-deploy
footgun from Test 3).
Adds ONE new precondition step before publishing:
sshpass -p "$PI_PASS" ssh "$PI_USER@$PI_HOST" \
'test -f ~/.openai_key && test -r ~/.openai_key' \
|| { echo "Create ~/.openai_key on the Pi (chmod 600),
paste your sk-... key into it, then re-run."; \
exit 1; }
This catches the silent "key missing" failure mode at
the deploy step rather than 30 s into the publish + scp.
```
## Concurrency model
Five concurrent threads of execution in steady state:
1. **Main thread.** Setup, session.update send, beep + arm, `doneFlag.Wait`, teardown via `finally`.
2. **ReceiveLoop** (`Task.Run`). Pulls WS messages, dispatches on `type`. Single writer of `downstreamChannel`, `_stopSending`, `_noMoreDeltas`, transcript builder. Calls `cts.Cancel()` on error.
3. **UpstreamSendLoop** (`Task.Run`). Pulls mic frames from `upstreamChannel`, JSON-serializes `input_audio_buffer.append`, calls `ws.SendAsync`. Single writer of the WS send stream.
4. **PortAudio input audio thread** (managed by the library). Runs the mic callback every 40 ms. Allocates `short[1920]`, memcpy's the input, `upstreamChannel.Writer.TryWrite(frame)`. Never blocks.
5. **PortAudio output audio thread** (managed by the library). Runs the output callback every 43 ms. Drains `downstreamChannel` into the output buffer; zero-fills underruns; sets `doneFlag` on completion. Never blocks.
Note that `ws.SendAsync` and `ws.ReceiveAsync` can both be active concurrently — `ClientWebSocket` supports one concurrent send + one concurrent receive (but NOT multiple concurrent sends or multiple concurrent receives). Our design uses exactly one of each, so this is safe.
## Error handling
| Failure mode | Behavior | Exit |
|-|-|-|
| `~/.openai_key` missing on Pi | `bin/probe-cs-4a` precondition check exits before publishing. | 1 |
| `~/.openai_key` exists but empty / not `sk-`-shaped | `LoadApiKey()` logs `[error] API key file empty` / `[error] API key doesn't start with sk-`, exits. | 2 |
| `ws.ConnectAsync` throws (DNS, TLS, 401, 403, 429, 500, network down) | Log `[error] WS connect failed: {ex.Message}` plus inner `HttpRequestException.StatusCode` if present. | 3 |
| Server `error` event mid-session | ReceiveLoop logs the full event JSON, `cts.Cancel()`. Main wakes from `doneFlag.Wait(cts.Token)` with `OperationCanceledException`; `finally` runs CloseAsync (no-throw with timeout) + stream.Stop + PortAudio.Terminate. | 4 |
| WS drops mid-conversation (`WebSocketException` in ReceiveLoop) | Same as above — log + cancel + cleanup-via-finally. | 4 |
| PortAudio input stream `Start()` throws | Log + Dispose stream + cts.Cancel. | 5 |
| PortAudio output stream `Start()` throws | Log + Dispose stream + cts.Cancel. | 6 |
| Server takes forever and never sends `response.done` | No timeout in 4a. User's recourse is Ctrl-C. (If verification surfaces this, add a configurable timeout in 4b — out of scope for 4a.) | 0 (via Ctrl-C) |
| Mic callback can't `TryWrite` to `upstreamChannel` (UpstreamSendLoop falling behind) | Log `[status] upstream behind, dropping mic frame`, continue. (Same shape as Test 3's "consumer behind" log.) | 0 |
| Workstation Ctrl-C | `Console.CancelKeyPress` → `e.Cancel = true; cts.Cancel()`. Main wakes from `doneFlag.Wait(cts.Token)` with `OperationCanceledException`. `finally` runs full cleanup path. | 0 |
The `finally` block at the bottom of `Program.cs`'s outer `try` runs **once** and is the single point where WS is closed, streams are stopped, and PortAudio is terminated. Cleanup is idempotent (`stream.Stop` is a no-op if already stopped; `PortAudio.Terminate` is fine to call once even if some path didn't run).
## Gotchas carried forward (from findings.md)
1. **`Libc.setenv("PA_ALSA_PLUGHW", "1", 1)`** before any PortAudio init. `Environment.SetEnvironmentVariable` alone does NOT propagate to libc on Linux. Apply alongside the managed call.
2. **No OrtEnv work in 4a** — no `Microsoft.ML.OnnxRuntime` dependency. (4b reintroduces it.)
3. **`using Stream = PortAudioSharp.Stream;`** at the top of `Program.cs` to disambiguate from `System.IO.Stream` that `ImplicitUsings` pulls in.
4. **`true`** in `Probe.csproj` for the input/output callbacks' `Buffer.MemoryCopy` over `short*`.
5. **Fire-and-forget output stream lifetime.** `stream.Start()` runs synchronously under try/catch BEFORE any auxiliary cleanup task launches; on `Start()` failure, dispose + rethrow without ever leaking the stream. Test 4a uses a *persistent* output stream (opened once, drained for both the beep and the response audio, stopped on shutdown), so there's no per-cycle cleanup task — the `Start()` failure path is the only one that matters.
6. **Streaming-length cleanup discipline.** Test 3's "scale `done.Wait(timeout)` to recordBuf length" trick does NOT apply because the assistant audio length is unknown a priori. Replaced by the output callback's `_currentChunk == null && Reader.Count == 0 && Reader.Completion.IsCompleted` triple check. The `ManualResetEventSlim doneFlag` carries the signal to the main thread.
7. **`ssh -t` in `bin/probe-cs-4a`** for Ctrl-C propagation from workstation SIGINT.
8. **Always wipe-and-replace the whole publish directory.** A partial `scp Probe` after a rebuild leaves the old `.dll` on the target, which the new launcher loads. `bin/probe-cs-4a` does `ssh "rm -rf ~/probe-cs-4a && mkdir ~/probe-cs-4a"` BEFORE the `scp -r` (Test 3 pattern).
9. **Startup beep must not be captured by the mic and shipped to OpenAI.** The `_micArmed` flag stays `false` until ~300 ms after the beep is queued to `downstreamChannel` (200 ms beep + 100 ms drain margin). The mic callback discards frames while `!_micArmed`.
New for 4a (none carried forward from prior tests — these are surfaced here for the first time):
10. **`ClientWebSocket` concurrency rule:** one concurrent Send + one concurrent Receive is supported; multiple concurrent Sends (or Receives) is not. Our design has exactly one SendLoop and one ReceiveLoop, so this is safe by construction — but if a future change adds a second sender (e.g. a ping task), it must serialize with the existing one.
11. **Channel completion semantics:** `Writer.Complete()` makes `Reader.Completion.IsCompleted` true ONLY after all written items are consumed. The output callback's triple check accounts for this correctly. Don't change to a different completion sentinel without re-verifying the corner case.
12. **API key file ownership:** the precondition check in `bin/probe-cs-4a` verifies *presence* and *readability* by `pi`, not strict mode 600. The probe itself logs a non-fatal warning if the mode is looser than 600; this lets a slightly looser local setup proceed.
## Verification (hardware only, no automated tests)
One-time setup on the Pi (workstation tells you to do this; `bin/probe-cs-4a` precondition-checks it before publishing):
```sh
sshpass -p 'assistant' ssh pi@192.168.50.115 \
'umask 077 && printf "%s\n" "sk-PASTE-YOUR-KEY-HERE" > ~/.openai_key'
```
Run from workstation:
```sh
./bin/probe-cs-4a
```
Expected console flow (abridged):
```
>> precondition: ~/.openai_key present on Pi. OK.
>> dotnet publish (linux-arm64, self-contained)
>> scp to pi@192.168.50.115:~/probe-cs-4a/ (wipe-and-replace)
>> ssh + run on Pi (Ctrl-C from this terminal stops the probe)
[ws] connecting to wss://api.openai.com/v1/realtime?model=gpt-realtime
[ws] connected
[ws] session.created sess_xxxxxxxxxxxx
[ws] session.updated — session pinned
Using device 3 ('USB Speaker Phone (LISTENAI ...)')
[beep]
Speak now.
[vad] speech started
[vad] speech stopped
[ws] response.created
[audio] first delta received (t = … s since speech_stopped)
[audio] playback drained
[transcript] Hello! How can I help you today?
[ws] closing
bye
```
### Pass criteria
All must hold during a single run on the Pi:
1. **End-to-end round-trip works once.** Probe boots, beeps, you speak a short sentence, the assistant's spoken reply plays back recognisably through the same USB Speaker Phone, the probe exits 0.
2. **Response is intelligible.** The assistant's audio is clear speech, no clipping, no dropouts longer than a single 43 ms callback (one zero-fill is fine and expected on the first chunk arrival).
3. **Round-trip latency.** Time from `[vad] speech stopped` log to `[audio] first delta received` log (i.e. first response audio byte *arriving* at the probe — actual speaker emission adds a fixed ~43 ms output-callback period on top) is **under ~2 s**. Slower is acceptable if explicitly logged; **over ~5 s is a fail.**
4. **No `[status] upstream behind` warnings** during normal operation.
5. **No `[error] WS …` events** during normal operation.
6. **No PortAudio `InputOverflow` status flags** during normal operation.
7. **Workstation Ctrl-C** at any point produces a clean exit: WS closes with normal closure, no stack trace, terminal returns to prompt. Process exit code 0 on Ctrl-C during steady listen; non-zero on Ctrl-C during a failure path is fine.
8. **`response.done`-followed-cleanup signaling works.** Probe exits within ~500 ms of the last audio sample being heard. No long hang.
### What success teaches us for 4b
- Auth + WS handshake + headers + URL are correct.
- `session.update` with `server_vad` works; the server fires `speech_started` / `speech_stopped` correctly on real Pi mic input.
- Base64 decode of `response.audio.delta` produces playable PCM16 at 24 kHz on the USB Speaker Phone (PortAudio plug-resamples 48↔24 at ALSA without trouble).
- The streaming-length cleanup discipline (`_noMoreDeltas` + `Channel.Completion.IsCompleted` + working-chunk check) is correct.
- The `Channel` upstream/downstream pattern handles backpressure cleanly.
- `bin/probe-cs-4a`'s `~/.openai_key` precondition pattern is the right key-management story (it'll lift verbatim into 4b and the main assistant).
### What failure would tell us
- **401 / 403 on connect:** API key is wrong or the file's been mangled.
- **WS connects but `session.updated` never arrives:** the session config is malformed for the current Realtime API surface — `modalities`, `turn_detection`, or `voice` is using an outdated field name or value.
- **No `speech_stopped` event ever fires:** `turn_detection` config is wrong for the current API (field renamed?), or the mic is silent (gain too low, wrong device, ALSA plug-rate-conversion broken at 24 kHz). Cross-check by adding a periodic RMS log in the mic callback.
- **`[status] upstream behind` repeats:** UpstreamSendLoop is slower than the mic callback. Most likely `ws.SendAsync` is blocking on TCP. Cheap remediation: bump channel capacity. Real remediation: investigate the WS backpressure.
- **First delta takes >5 s:** server-side issue OR our `instructions` is producing a long thinking pause. Tighten instructions or report and move on.
- **Garbled / chipmunk-speed playback:** sample-rate mismatch. Verify the output stream actually opened at 24 kHz (PortAudio might have silently substituted a default).
- **Click at the start of playback:** zero-fill underrun ran before the first delta. Acceptable for a probe; if it bothers you, add a small pre-roll buffer.
- **Probe hangs on exit:** `Writer.Complete()` was never called (receive loop missed `response.done`), or the output callback's triple check is wrong. Investigate via logging.
## Reference paths
- Spec (this doc): `docs/superpowers/specs/2026-06-12-test-4a-openai-realtime-design.md`
- Plan (next, to write): `docs/superpowers/plans/2026-06-12-test-4a-openai-realtime.md`
- Code (to be written): `tests/04a-realtime-oneshot-cs/`
- Deploy (to be written): `bin/probe-cs-4a`
- Test 3 reference: `tests/03-full-cycle-cs/Program.cs`, `bin/probe-cs-3`, `docs/superpowers/specs/2026-06-12-test-3-full-cycle-csharp-design.md`
- Pi access: `pi@192.168.50.115`, password `assistant` (see `CLAUDE.md`)
- API key file location: `/home/pi/.openai_key` on the Pi, mode 600, single line of `sk-…`
- Canonical findings: `findings.md` § "C# probe outcome" through § "C# full-cycle outcome"
- OpenAI Realtime API surface as of this brainstorm:
- WebSocket URL: `wss://api.openai.com/v1/realtime?model=gpt-realtime`
- Headers: `Authorization: Bearer `, `OpenAI-Beta: realtime=v1`
- Audio formats: `pcm16` (default 24 kHz Int16), `g711_ulaw`, `g711_alaw`
- Session config keys used: `modalities`, `instructions`, `voice`, `input_audio_format`, `output_audio_format`, `turn_detection`
- `turn_detection.server_vad` subkeys used: `threshold`, `prefix_padding_ms`, `silence_duration_ms`, `create_response`
- Server events consumed: `session.created`, `session.updated`, `input_audio_buffer.speech_started`, `input_audio_buffer.speech_stopped`, `input_audio_buffer.committed`, `response.created`, `response.audio_transcript.delta`, `response.audio.delta`, `response.audio.done`, `response.done`, `error`