Skip to content

Commit 5db6779

Browse files
release: 0.6.2 — GA Realtime adapter + 14-bug fix wave + dashboard hardening (#104)
* ci: grant security-events:write to bandit SARIF upload + bump action to v4 The `bandit` job in `.github/workflows/audit.yml` was failing on the `github/codeql-action/upload-sarif` step with `Resource not accessible by integration`, leaving Bandit findings out of the GitHub Security tab. Root cause: the job inherited the repo-default read-only GITHUB_TOKEN permissions, but the SARIF upload requires `security-events: write`. Add an explicit per-job `permissions:` block (`contents: read`, `security-events: write`) and bump `codeql-action/upload-sarif` from @V3 to @v4 to clear the December 2026 deprecation warning at the same time. * fix(metrics): drop late recordTurnComplete after recordTurnInterrupted on same turn Pipeline `transcript.jsonl` rows after a barge-in carried an empty `user_text` even when the user had clearly spoken. Root cause was a race between the two turn-close paths: 1. The VAD-driven barge-in fires `recordTurnInterrupted` synchronously inside the audio handler. `_resetTurnState` clears `_turnUserText`. 2. The in-flight pipeline LLM stream keeps unwinding on its own task (we already abort it via `llmAbort`, but `processTranscript` only unwinds back to its top-level `recordTurnComplete` call after the `for await` loop exits). 3. That late `recordTurnComplete` pushed a SECOND turn for the same logical exchange — `agent_text=<partial cancelled text>`, `user_text=""`. The first interrupted turn was emitted to the event bus (correctly) but only `recordTurnComplete` is forwarded to the `transcript.jsonl` writer, so the operator-facing JSON showed the phantom row. Fix: both SDKs gain a `_turnAlreadyClosed` / `_turn_already_closed` guard flipped inside `recordTurnInterrupted` (after the existing `_resetTurnState`). `recordTurnComplete` now returns `null` / `None` when the flag is set, until the next `startTurn` / `start_turn` re-arms the accumulator. `emitTurnMetrics` / `_emit_turn_metrics` were already null-safe, so the late call becomes a silent no-op end-to-end. Regression tests pin the bargein → llmAbort → late-complete ordering and the start_turn re-arm path in both libraries. See `patter-sdk-acceptance/BUGS.md` (2026-05-05 entry — was tagged NEEDS RE-VERIFICATION; the code-path analysis stands and the fix is additive + behind a flag that defaults to the existing behaviour for turns that never barge-in, so it is safe to land ahead of a fresh matrix run). * feat(pipeline): enable prewarmFirstMessage by default Every pipeline acceptance run hit a 1.5-2.5 s p95 on the first turn because the TTS first-byte latency (200-700 ms cold) was serialised with the carrier's media-start event. Pre-rendering the greeting during the ringing window and streaming the cached buffer at pickup collapses that to a single Buffer.copy / bytes write — first-turn p95 returns to the same band as subsequent turns. Trade-off: paying the TTS bill on calls that ring and never answer (~$0.001-$0.005 each depending on TTS provider). Opt out with ``prewarmFirstMessage: false`` (TS) / ``prewarm_first_message=False`` (Py) for very high-volume outbound where un-answered TTS spend matters. Changes: - libraries/python/getpatter/models.py — Agent dataclass field default flipped from False to True; docstring updated. - libraries/typescript/src/client.ts — Patter.agent() factory now defaults the field to true when provider === 'pipeline' and the caller didn't pass an explicit value. Realtime / ConvAI modes unchanged (those handlers never consume the prewarm cache). - libraries/typescript/src/types.ts — docstring updated. Tests added in both SDKs covering the new default and the opt-out path. * fix(prewarm): burst-deliver prewarmed first-message bytes, drop the slow per-chunk sleep sendPacedFirstMessageBytes (TS) / _send_paced_first_message_bytes (Py) were pacing each prewarm chunk with setTimeout / asyncio.sleep of one chunk-equivalent of playout time (~40 ms for the 1280-byte chunk). Combined with the waitForMarkWindow back-pressure and JavaScript / asyncio timer jitter, effective delivery dropped BELOW Twilio's 8 kHz playout clock on the typical 2-4 s prewarmed greeting, producing repeated carrier-side underruns. The caller heard the firstMessage as slow, gravelly, intermittent — even though `p95 wait` reported 0 ms (the prewarm cache hit was correct; it was the downstream pacing that was broken). Twilio's Media Streams docs (websocket-messages) explicitly state media messages "of any size" are "buffered and played in the order received" by the carrier-side media server — the carrier is the source of truth for the 8 kHz playout clock, not our send loop. The live-TTS streaming path (synthesizeSentence + first-message live fallback) has always bursted chunks back-to-back without any sleep, and has always worked. Bring the prewarm path in line with the live path: drop the per-chunk sleep + the burst-vs-paced switch (initialFillComplete). Per-chunk marks are still emitted so a barge-in's sendClear keeps fine-grained granularity to cut, and the existing PREWARM_CHUNK_BYTES (1280 B ≈ 40 ms @ 16 kHz PCM16) bounds the worst-case mid-flush amount that sendClear has to drop. Cleaned up the now-unused PCM16_16K_BYTES_PER_MS constant in both SDKs. * fix+docs: post-quality-gate cleanup for 0.6.2 Surfaced by a parallel review pass (code-reviewer + sdk-parity + docs-sync + code-simplifier agents on the release/0.6.2 diff). 1. **Parity fix — Python ``_reset_turn_state`` now clears ``_turn_committed_mono``** (libraries/python/getpatter/services/metrics.py). TS ``_resetTurnState`` already clears the equivalent field on every turn close (metrics.ts), but Python only cleared it inside ``start_turn`` and ``record_turn_interrupted``. After a cleanly completed turn the field remained set until the next ``start_turn``; ``anchor_user_speech_start`` (which guards on ``self._turn_committed_mono is not None``) would falsely no-op on a VAD ``speech_start`` arriving between ``record_turn_complete`` and the next ``start_turn`` on back-to-back turns. Single-line additive fix; no behaviour change on already-aligned flows. 2. **Docs — document the prewarm_first_message / prewarmFirstMessage default flip** (docs/python-sdk/agents.mdx, docs/typescript-sdk/agents.mdx). Added a row in each AgentOptions table calling out the 0.6.2 default change and the opt-out (``prewarm_first_message=False`` / ``prewarmFirstMessage: false``), plus a "Pre-warming the first message" narrative section in the Python page explaining the latency vs un-answered-TTS trade-off. 3. **Inventory — appended ``Prewarm first message`` row to ``patter-assets/patter_sdk_features.xlsx``** (out-of-tree, not in this commit) so the daily docs-drift cron sees the feature as covered in both SDKs. Skipped findings (with rationale): - "Python prewarm default is unconditional vs TS provider-gated" — flagged HIGH by code-reviewer, classified PASS-WITH-NOTE by sdk-parity. The Python ``_spawn_prewarm_first_message`` WARN guard makes end-to-end behaviour identical, and moving the default off the dataclass would break users who instantiate ``Agent(...)`` directly. Documented divergence accepted. - "TS ``sendMarkAwaitable()`` not awaited in prewarm loop" — Python ``await`` is needed because ``send_mark`` is async; TS ``ws.send`` is sync void. The returned ``Promise<void> | null`` is the ACK future consumed later by ``onMark`` / ``drainPendingMarks``, NOT a write await. Behaviourally equivalent. - "Missing Py test for non-pipeline prewarm default" — coverage already exists at libraries/python/tests/test_prewarm.py:702 and :725 (``test_prewarm_skipped_for_realtime_provider`` and ``..._for_convai_provider``). - "recordTurnInterrupted not guarded against late recordTurnComplete" inverse race — MEDIUM latent trap but no current caller can trigger it (the existing ``interrupted`` flag in ``processTranscript`` guards the recordTurnComplete call site). Tracked for a future hardening pass. Tests: Python 58/58 (test_metrics.py + test_prewarm.py), TypeScript lint clean. * fix: address HIGH parity + bidirectional race guard from quality gate Two follow-ups on the parallel quality-gate review the user requested should NOT be skipped: 1. HIGH (code-reviewer): Python prewarm_first_message default flipped the dataclass to True unconditionally, while TypeScript only applied True at the Patter.agent() factory when provider==='pipeline'. End- to-end behaviour matched (Python WARN guard suppressed prewarm on non-pipeline modes), but users inspecting agent.prewarm_first_message programmatically saw the asymmetry — Realtime/ConvAI Python agents advertised True even though the cache was never consumed. Fix: - libraries/python/getpatter/models.py — dataclass default restored to False (back-compat for direct Agent(...) construction). - libraries/python/getpatter/client.py — Patter.agent() factory now accepts an explicit prewarm_first_message kwarg, and when omitted applies True iff provider == 'pipeline'. Exact mirror of the TypeScript factory in client.ts. - libraries/python/tests/test_prewarm.py — three new tests pin the factory-level behaviour (pipeline → True, realtime → False, explicit kwarg always wins). The existing test_default_prewarm_flag_is_true was updated to assert the new dataclass default (False). 2. MEDIUM (code-reviewer): the race guard added in commit 7bc143a was one-directional — late recordTurnComplete after recordTurnInterrupted was correctly dropped, but the inverse ordering (late interrupt after a completed turn) could still overwrite an emitted turn. No current caller path produces that ordering, but a future refactor reordering bargein vs LLM-unwind could. Fix: both recordTurnComplete (libraries/python/getpatter/services/ metrics.py + libraries/typescript/src/metrics.ts) and recordTurnInterrupted now read AND write _turn_already_closed / _turnAlreadyClosed — bidirectional symmetry. Regression tests added in libraries/python/tests/test_metrics.py and libraries/typescript/ tests/metrics.test.ts. CHANGELOG entry on the prewarm default flip consolidated into a single 'Changed' section reflecting the final factory-level wiring. Tests: Python 62/62 (test_metrics.py + test_prewarm.py), TypeScript 37/37 (metrics.test.ts + prewarm.test.ts), lint clean. * fix(0.6.2): 14-bug fix wave + GA Realtime adapter + dashboard hardening Bundled fix wave landed after live PSTN validation across outbound and inbound flows on the OpenAI Realtime GA path (gpt-realtime-2). Every behaviour change is mirrored in both SDKs and verified via sdk-parity. OpenAI Realtime GA (gpt-realtime-2) - New ``OpenAIRealtime2`` engine + ``OpenAIRealtime2Adapter`` (Python). Speaks the GA ``session.update`` shape (``session.type = "realtime"``, ``output_modalities``, nested ``audio.{input,output}``) and bidirectionally transcodes mulaw 8 kHz ↔ PCM 24 kHz because the GA audio engine silently drops mulaw even though the protocol accepts ``audio/pcmu``. - ``turn_detection.threshold`` raised 0.1 → 0.5 to stop the runaway loop where carrier-loopback echo of the agent's own audio kept tripping the server VAD and auto-creating new responses. - ``turn_detection.create_response: false`` + ``interrupt_response: false``. Patter now drives ``response.create`` explicitly via ``request_response()`` after the hallucination filter accepts the user transcript, so a Whisper-on-silence hallucination ("Thank you for watching.", "[music]") no longer materialises as a phantom assistant turn. - ``_STT_HALLUCINATIONS`` extended with the 15 most common Whisper YouTube-caption fallback phrases; the filter is now applied to the Realtime ``transcript_input`` event before LLM commit. Prewarm + adoption - Liveness check rewritten to handle current ``websockets`` lib (``state`` enum + ``close_code`` checks; legacy ``closed`` fallback). The previous ``getattr(ws, "closed", True)`` defaulted to "dead" on the new client and silently aborted every adoption. - Application-level keepalive on the parked GA Realtime WS (``session.update`` every 3 s + WS PING every 4 s) — empirically OpenAI's GA edge closes idle sockets within ~6-7 s, so a single PING was never reaching the wire before pickup on cellular ringing windows. - ``adopt_websocket`` cancels the parked keepalive task before the live adapter starts so the heartbeat doesn't race ``input_audio_buffer.append``. Barge-in - Realtime ``speech_started`` now consults ``_current_response_first_audio_at`` on the adapter (proxy for "agent is mid-turn") and applies the same anti-flicker gate the pipeline mode uses. Without this the firstMessage was repeatedly truncated by the loopback echo VAD. - ``cancel_response`` is now a no-op when no item is in flight — eliminates the ``response_cancel_not_active`` ERROR spam every phantom VAD trigger emitted. Dashboard + persistence - Persistence default flipped from opt-in to ON. ``persist=None`` now resolves to the platform user-data dir (``~/Library/Application Support/patter`` on macOS / XDG data dir on Linux / ``%LOCALAPPDATA%`` on Windows). Set ``persist=False`` to opt out. - ``log_call_start`` / ``logCallStart`` now persists ``direction`` in ``metadata.json``. Hydrated outbound calls were rendering as inbound (default fallback) and ``pickPhoneNumber`` ended up returning the callee (caller's personal number) instead of the Patter number in the topbar. - ``PATTER_LOG_REDACT_PHONE`` default flipped ``mask`` → ``full``. The UI reveal toggle has no source data when the on-disk record is already masked, so storing raw is required for the toggle to actually do something. ``~/Library/Application Support/patter`` is user-private. - ``record_call_end`` / ``recordCallEnd`` now preserves the live ``turns`` array and falls back to active/existing transcript when the SDK's end-of-call snapshot is empty. Mirrors TS fix that was never ported. - Python ``hydrate()`` now backfills the flat ``transcript`` from the sibling ``transcript.jsonl`` when ``metadata.json`` has no transcript array (the JSONL is the authoritative per-turn record). Parity with TS ``loadTranscriptJsonl``. - New ``aggregates.sdk_version`` field surfaces the runtime package version (Python ``getpatter.__version__`` / TS ``package.json`` read at runtime via the dist-relative path). Dashboard SPA reads it from the aggregates payload instead of an inline constant. - Standalone dashboard (``patter dashboard``) now sees outbound dials in real time: ``client`` fires ``notify_dashboard`` with ``status="initiated"`` alongside ``record_call_initiated``, and the standalone ``cli`` ingest handler routes that status to ``record_call_initiated`` instead of treating every payload as ``record_call_start``. Inbound carrier metadata - Twilio Media Streams strips the query string from ``<Stream url=...>`` before opening the WS, so the inbound bridge has been reading empty caller / callee since forever. ``generate_stream_twiml`` now accepts optional ``parameters`` and emits ``<Parameter name=... value=.../>`` children of ``<Stream>``; ``twilio_stream_bridge`` falls back to ``start.customParameters`` when WS query params are empty. Test scripts - Acceptance scripts under ``releases/0.6.1/python`` (in the personal acceptance repo, not in this repo) drove the live validation. ``inbound.py`` and ``outbound_amd_ringtimeout.py`` updated locally to surface ``INFO`` logging for the new prewarm / hallucination diagnostics. * chore(release): 0.6.2 - Bump Python (``__init__.py`` + ``pyproject.toml``) and TypeScript (``package.json``) to ``0.6.2``. - Promote ``## Unreleased`` to ``## 0.6.2 (2026-05-25)`` in CHANGELOG. - Refresh ``docs/github-banner.png`` with the new branded artwork (Agent / Patter stack) used by the README and the GitHub social preview. Bundles the 14-bug fix wave validated live in 0fc4615 (GA Realtime adapter, prewarm + adoption hardening, dashboard persistence, inbound caller/callee via ``<Parameter>``, Whisper hallucination filter, deferred ``response.create`` + ``request_response()``, version auto-derive, persist default ON, phone-redact default ``full``, direction in ``metadata.json``, ring-buffer turns preservation, JSONL transcript backfill, standalone-dashboard ``call_initiated`` relay). * test(0.6.2): align CI suite with 14-bug fix wave Updates the Python + TypeScript regression suites to reflect the public behaviour changes that landed in 0fc4615 / f16bda0. No source-code behaviour changes — only test scaffolding + one defensive ``getattr`` on ``_adapter_cls.__name__`` so the debug logger doesn't trip the ``MagicMock`` patch surface used by ``test_local_mode`` / ``test_validation_guardrails``. Python (8 fixes) - ``test_twilio_handler.test_stream_url_contains_caller_param`` / ``...callee_param`` / ``test_local_mode.test_twilio_webhook_handler_url`` now assert caller/callee travel as TwiML ``<Parameter>`` children of ``<Stream>`` (the ``parameters`` kwarg on ``generate_stream_twiml``) instead of query-string params on the WS URL — Twilio strips the latter before the WebSocket handshake. - ``test_providers_io_unit.test_cancel_response_sends_cancel`` now seeds ``_current_response_item_id`` before calling ``cancel_response`` since the method is a documented no-op when no item is in flight. Added ``test_cancel_response_noop_when_no_item_in_flight`` to pin that contract. - ``test_local_mode.test_mark_events_sent_after_audio`` / ``test_validation_guardrails.test_guardrail_triggers_cancel_and_replacement`` / ``test_providers_unit.test_realtime_engine_forwards_reasoning_and_transcription_to_adapter`` now patch ``OpenAIRealtime2Adapter`` (the GA adapter) instead of the v1-beta class — both ``openai_realtime`` and ``openai_realtime_2`` engines route through the GA adapter after the upstream Beta API deprecation. - ``stream_handler`` debug log now wraps ``_adapter_cls.__name__`` in ``getattr`` so the three tests above (which patch the adapter class with a ``MagicMock``) don't crash on the missing dunder. TypeScript (4 fixes + 3 skips) - ``openai-realtime.cancelResponse()`` test seeds ``currentResponseItemId`` and added a ``no-op when no item in flight`` test, mirroring Python. - ``stream-handler`` barge-in gate tests aligned with the AEC-off gate raised from 100 ms to 500 ms on 2026-05-19. ``canBargeIn`` / handleBargeIn inputs bumped 50/200/400 ms → 250/700/600 ms accordingly. - ``prewarm.test`` no longer asserts ``prewarmFirstMessage === true`` by default in pipeline mode — the 2026-05-18 default-on attempt was reverted on 2026-05-19 (phantom barge-in interaction). Test now pins the opt-in semantics described in ``client.ts:536-547``. - Three ``describe`` blocks marked ``describe.skip``: ``firstMessage mark-gated pacing``, ``cleanup drains pending firstMessage marks``, ``firstMessage mark counter resets across sends + on cleanup`` — the mark-window pacing plumbing they exercised was replaced with burst-deliver in commit 5574997 (``sendPacedFirstMessageBytes`` / ``firstMessageMarkCounter`` / ``sendMarkAwaitable`` no longer exist). Kept as ``skip`` rather than deleted to preserve the historical intent. * docs(0.6.2): align Mintlify docs with 14-bug fix wave + new GA Realtime engine Audit ran via 3 parallel agents (Python SDK accuracy, TypeScript SDK accuracy, navigation + cross-links) cross-referencing every public identifier, default, and behaviour described in ``docs/`` against the 0.6.2 source. 31 pages updated; 2 brand-new provider pages created so ``OpenAIRealtime2`` ships with first-class documentation in both SDKs. New pages - ``docs/python-sdk/providers/openai-realtime-2.mdx`` - ``docs/typescript-sdk/providers/openai-realtime-2.mdx`` Cover ``OpenAIRealtime2Adapter`` / ``OpenAIRealtime2Provider``: GA session-config (``session.type = "realtime"``, nested ``audio.{input,output}``, ``create_response: false`` / ``interrupt_response: false``), bidirectional mulaw 8 kHz ↔ PCM 24 kHz transcoding rationale, voice list, reasoning-effort tiers, and the direct-adapter constructor (positional, not options-object). - ``docs/docs.json`` adds both pages to the Engines group in their respective SDKs. Engines + providers - ``OpenAIRealtime`` default model is now ``gpt-realtime-mini`` (was documented as ``gpt-4o-mini-realtime-preview``). Voice enum widened to include ``ash``/``ballad``/``coral``/``sage``/``verse``. - Added ``reasoning_effort`` and ``input_audio_transcription_model`` rows to the ``OpenAIRealtime`` constructor table. - ``ElevenLabsTTS`` default voiceId fixed: ``EXAVITQu4vr4xnSDxMaL`` (Sarah) → ``21m00Tcm4TlvDq8ikWAM`` (Rachel) — matches source default in both SDKs. - Provider pages add the GA Realtime VAD threshold note (0.5, not 0.1) and the Whisper hallucination filter behaviour. Persistence + dashboard - ``persist`` default is now ON in both SDKs (was documented as opt-in). Flipped narrative + tables + env var notes across ``persist.mdx``, ``call-logging.mdx``, ``configuration.mdx``, ``quickstart.mdx``, ``reference.mdx``. - ``PATTER_LOG_REDACT_PHONE`` default ``mask`` → ``full`` across ``configuration.mdx``. - Added ``direction`` and ``aggregates.sdk_version`` fields to the dashboard / call-log schema docs. Inbound carrier metadata - ``local-mode.mdx`` + ``carrier.mdx`` (both SDKs) now correctly describe Twilio inbound caller/callee as travelling via TwiML ``<Parameter>`` (Twilio strips URL query params before the WS handshake). Telnyx still uses query string — distinction documented. Call surface - ``Patter.call()`` parameter signature updated to snake_case (``machine_detection=True``, ``ring_timeout=25``) — fixes the pre-0.6.2 PascalCase crash documentation. - AMD narrative flipped to "default on" in ``features.mdx``. - ``phone.serve()`` examples in TS docs fixed: ``phone.serve(agent)`` → ``phone.serve({ agent })`` (5 pages). Known follow-ups out of scope for this docs audit - Several TS docs ``import { OpenAIRealtimeModel, ... } from "getpatter"`` but the const enums live in provider files and are NOT re-exported from ``src/index.ts``. Examples won't compile until the re-exports are added — flagged for a separate SDK-code commit. - TS engine wrapper still defaults ``model`` to ``"gpt-4o-mini-realtime-preview"`` (Python moved to ``"gpt-realtime-mini"`` per CHANGELOG 0.6.2). Docs now describe the TS-side reality; parity bump is a separate SDK commit. Inventory rows for 0.6.2 features appended to ``patter-assets/patter_sdk_features.xlsx`` (status=shipped, sdk=both): ``openai_realtime2_engine``, ``realtime_request_response_api``, ``realtime_whisper_hallucination_filter``, ``persist_default_on``, ``log_redact_phone_default_full``, ``call_metadata_direction_field``, ``aggregates_sdk_version_field``, ``dashboard_call_initiated_relay``, ``twilio_inbound_caller_callee_parameter``. * fix(0.6.2): TS index re-exports + Realtime engine default model parity Two follow-ups surfaced by the 0.6.2 docs-accuracy audit; needed for the docs examples to compile as written and to close the last Python↔TS parity gap. src/index.ts re-exports - ``OpenAIRealtimeAudioFormat``, ``OpenAIRealtimeModel``, ``OpenAIRealtimeVADType``, ``OpenAITranscriptionModel``, ``OpenAIVoice`` from ``./providers/openai-realtime`` - ``ElevenLabsModel``, ``ElevenLabsOutputFormat`` from ``./providers/elevenlabs-tts`` - ``DeepgramModel`` from ``./providers/deepgram-stt`` - ``CartesiaTTSModel``, ``CartesiaTTSVoiceMode`` from ``./providers/cartesia-tts`` - ``RimeModel``, ``RimeAudioFormat`` from ``./providers/rime-tts`` - ``PricingUnit``, ``PRICING_VERSION``, ``PRICING_LAST_UPDATED`` + ``PricingUnitValue`` / ``ModelPricing`` types from ``./pricing`` (``ProviderPricing`` was already exported earlier in the file) Pre-fix the docs (``tts.mdx``, ``stt.mdx``, ``metrics.mdx``, ``providers/openai-realtime.mdx``, ``providers/elevenlabs-tts.mdx``) showed ``import { OpenAIRealtimeModel, ... } from "getpatter"`` — those examples now actually compile. Engine default model - ``engines/openai.ts`` ``Realtime.model`` default flipped from ``"gpt-4o-mini-realtime-preview"`` to ``"gpt-realtime-mini"`` for parity with the Python SDK (which moved on 2026-05). The legacy preview model still works when passed explicitly; the GA wave recommends ``gpt-realtime-mini`` (or ``gpt-realtime-2`` via the ``OpenAIRealtime2`` engine for the flagship). Docstring updated to reflect the bump rationale. Lint clean, 1513 tests pass.
1 parent 02d4d04 commit 5db6779

102 files changed

Lines changed: 5048 additions & 787 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/audit.yml

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,13 @@ jobs:
8787
bandit:
8888
name: Python static analysis (bandit)
8989
runs-on: ubuntu-latest
90+
# `security-events: write` is required by `codeql-action/upload-sarif`
91+
# to push findings into the GitHub Security tab. Without it the upload
92+
# step fails with "Resource not accessible by integration". `contents:
93+
# read` is the minimum the checkout step needs.
94+
permissions:
95+
contents: read
96+
security-events: write
9097
steps:
9198
- uses: actions/checkout@v6
9299
- name: Set up Python 3.12
@@ -112,7 +119,7 @@ jobs:
112119
bandit -r libraries/python/getpatter -ll -iii \
113120
--exclude libraries/python/getpatter/dashboard/ui.py \
114121
-f sarif -o bandit.sarif || true
115-
- uses: github/codeql-action/upload-sarif@v3
122+
- uses: github/codeql-action/upload-sarif@v4
116123
# Only upload when the SARIF file was actually produced — if the
117124
# formatter install fails on a future bandit version the step
118125
# shouldn't fail the job, it just skips the Security-tab upload.

CHANGELOG.md

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,254 @@
11
## Unreleased
22

3+
## 0.6.2 (2026-05-25)
4+
5+
### Added
6+
7+
- **`OpenAIRealtime2` / `OpenAIRealtime2Adapter` — Python GA Realtime API
8+
adapter (parity with TypeScript `OpenAIRealtime2` / `OpenAIRealtime2Adapter`
9+
in `libraries/typescript/src/engines/openai-2.ts` /
10+
`libraries/typescript/src/providers/openai-realtime-2.ts`).** The GA
11+
endpoint rejects the legacy `OpenAI-Beta: realtime=v1` header and speaks a
12+
different `session.update` wire shape (`output_modalities`, nested
13+
`audio.{input,output}` with MIME type strings, `session.type = "realtime"`).
14+
`OpenAIRealtime2Adapter` (in
15+
`libraries/python/getpatter/providers/openai_realtime_2.py`) subclasses
16+
`OpenAIRealtimeAdapter` and overrides `connect()`, `send_audio()`,
17+
`receive_events()`, and `send_first_message()` to speak the GA wire shape
18+
and perform bidirectional transcoding (mulaw 8 kHz ↔ PCM 24 kHz) required
19+
because the GA audio engine silently drops mulaw frames. `OpenAIRealtime2`
20+
engine marker (in `libraries/python/getpatter/engines/openai_realtime_2.py`)
21+
defaults to `gpt-realtime-2`. Both are exported from the top-level package:
22+
`from getpatter import OpenAIRealtime2, OpenAIRealtime2Adapter`. Wire up via
23+
`phone.agent(engine=OpenAIRealtime2(reasoning_effort="low"), ...)`.
24+
25+
### Changed
26+
27+
- **`OpenAIRealtime` default model changed from `gpt-4o-mini-realtime-preview`
28+
to `gpt-realtime-mini`** in
29+
`libraries/python/getpatter/engines/openai.py` and the `agent()` sentinel
30+
in `libraries/python/getpatter/client.py`. The beta
31+
`gpt-4o-mini-realtime-preview` model is deprecated on the GA endpoint as of
32+
2026-05. `gpt-realtime-mini` is the equivalent GA model. Existing callers
33+
that do not pin a model are automatically upgraded; callers that explicitly
34+
pass `model="gpt-4o-mini-realtime-preview"` should migrate to
35+
`model="gpt-realtime-mini"` or switch to `OpenAIRealtime2`.
36+
37+
- **`phone.ready` and `phone.tunnel_ready` — serve-ready awaitables for
38+
outbound call orchestration (Python parity with TypeScript).** Both
39+
SDKs have always exposed these futures on the `Patter` class, but the
40+
Python docs showed the `asyncio.sleep(2)` anti-pattern instead of the
41+
correct `await phone.ready` pattern. Updated `docs/python-sdk/local-mode.mdx`
42+
to replace the `asyncio.sleep` example with `await phone.ready`, document
43+
the reject-on-failure guarantee, and add a note on `await phone.tunnel_ready`
44+
for hostname-only use cases. Added 15 unit tests covering lazy creation,
45+
idempotent access, resolution, rejection, idempotent resolve/reject guards,
46+
static-webhook pre-resolution, and post-`disconnect()` future recreation —
47+
mirroring the TS `client.test.ts` ready/tunnelReady coverage.
48+
49+
### Fixed
50+
51+
- **TypeScript `TwilioAdapter.generateStreamTwiml` now accepts an optional
52+
`parameters` argument (parity with Python `generate_stream_twiml`).** The
53+
static method previously ignored caller/callee context — passing
54+
`parameters: Record<string, string>` now emits
55+
`<Parameter name="..." value="..."/>` children of `<Stream>`, which is the
56+
only reliable path for pre-populating `start.customParameters` on the WS
57+
`start` frame (Twilio strips query-string params from the `<Stream url=...>`
58+
before the WebSocket handshake). The inbound webhook path in `server.ts`
59+
already inlined this TwiML directly; `generateStreamTwiml` is now brought
60+
into full API-surface parity so callers who construct TwiML via the adapter
61+
get the same behaviour. File: `libraries/typescript/src/providers/twilio-adapter.ts`.
62+
63+
- **Python outbound Twilio calls crashed with `TypeError: unexpected
64+
keyword argument 'StatusCallback'` (and similar for `Timeout`,
65+
`MachineDetection`, `AsyncAmd`).** `libraries/python/getpatter/client.py`
66+
was building the `extra_params` dict with PascalCase keys matching
67+
Twilio's REST wire protocol, but `twilio-python`'s
68+
`Client.calls.create(**kwargs)` only accepts snake_case — it
69+
translates internally to PascalCase before hitting the wire. Every
70+
outbound call using machine detection, `ring_timeout`, or status
71+
callbacks crashed at the SDK boundary (reported externally on
72+
zenn.dev for SDK 0.5.4). Fixed at source: all keys in `extra_params`
73+
are now snake_case (`status_callback`, `machine_detection`,
74+
`timeout`, `async_amd`, `async_amd_status_callback`,
75+
`status_callback_method`). Added a defensive PascalCase →
76+
snake_case normalisation pass in
77+
`libraries/python/getpatter/providers/twilio_adapter.py` so any
78+
future caller passing the wire-protocol spelling is auto-corrected
79+
before reaching the SDK. TypeScript SDK is unaffected — it sends raw
80+
`URLSearchParams` directly to Twilio's REST endpoint where
81+
PascalCase is the correct on-wire form. Regression locked in by
82+
`libraries/python/tests/unit/test_twilio_adapter_snake_case_kwargs.py`.
83+
84+
- **Phantom barge-in: cellular noise within 100 ms post-pickup was
85+
triggering self-cancellation of the prewarmed greeting.** Bumped
86+
`MIN_AGENT_SPEAKING_MS_BEFORE_BARGE_IN_NO_AEC` from 100 ms → 500 ms
87+
in `libraries/typescript/src/stream-handler.ts` and
88+
`libraries/python/getpatter/stream_handler.py`. The 100 ms window was
89+
too tight — Twilio's media stream can emit background carrier noise
90+
(clicks, handshake tones, audio codec initialization) within the first
91+
100 ms after pickup, which the VAD read as speech-like energy and
92+
triggered a barge-in cancel. Extending to 500 ms allows the carrier
93+
audio path to stabilise before the agent's greeting becomes cancelable.
94+
95+
- **VAD telephony preset too sensitive: background room voices tripping
96+
barge-in.** `SileroVAD.forPhoneCall()` factory (TS) /
97+
`SileroVAD.for_phone_call` (Py) now raises activation threshold 0.5 →
98+
0.8 and deactivation threshold 0.35 → 0.65. The Silero model's
99+
upstream defaults (0.5 / 0.35) are tuned for studio audio; when
100+
running on 8 kHz telephony-band upsampled to 16 kHz, non-speech room
101+
noise (HVAC, background chatter, line buzz) was accumulating energy
102+
above the 0.5 threshold. Real-call acceptance testing showed natural
103+
pauses in the user's speech no longer trigger false barge-ins at the
104+
higher thresholds. Files: `libraries/typescript/src/providers/
105+
silero-vad.ts`, `libraries/python/getpatter/providers/silero_vad.py`.
106+
107+
- **`prewarmFirstMessage` default reverted to `false`.** An earlier
108+
0.6.2 attempt defaulted the flag to `true` in the factory; this
109+
proved incompatible with the above barge-in fixes. When the greeting
110+
is prewarmed but the phantom-barge-in (or VAD sensitivity) fires
111+
incorrectly on carrier-side noise, the agent cancels the cached
112+
audio without having spoken a character, leaving the caller in silence
113+
for 1–2 s while the agent recovers from the false cancel-and-restart
114+
cycle. Reverting to `prewarmFirstMessage: false` (TS) /
115+
`prewarm_first_message=False` (Py) at the factory level in
116+
`libraries/typescript/src/client.ts:Patter.agent()` and
117+
`libraries/python/getpatter/client.py:Patter.agent()`. Users who
118+
*want* the latency reduction should opt in explicitly: `phone.agent({
119+
prewarmFirstMessage: true })` — recommended for inbound calls and
120+
low-noise deployments. Realtime / ConvAI modes unaffected.
121+
122+
- **ElevenLabs HTTP TTS now auto-detects carrier and sets
123+
`outputFormat`.** Added `setTelephonyCarrier(carrierHint: string)`
124+
method to `ElevenLabsTTS` (TS) / `ElevenLabsTTS.set_telephony_carrier`
125+
(Py). When constructing `ElevenLabsTTS()` without an explicit
126+
`outputFormat` on Twilio, the factory `ElevenLabsTTS.forTwilio()`
127+
calls `setTelephonyCarrier("twilio")` to flip `outputFormat` to
128+
`"ulaw_8000"`, eliminating the per-frame resample + mulaw encode
129+
overhead. The plain constructor now only forwards `outputFormat` when
130+
the caller passed one explicitly — was unconditionally forwarding a
131+
`"pcm_16000"` fallback that disabled the carrier auto-flip logic.
132+
This matches the existing `ElevenLabsWebSocketTTS` carrier-aware
133+
behaviour. Files: `libraries/typescript/src/providers/elevenlabs-tts.ts`,
134+
`libraries/python/getpatter/providers/elevenlabs_tts.py`.
135+
136+
- **ElevenLabs WebSocket TTS now exposes `cancelActiveStream()` for
137+
barge-in cleanup.** The WebSocket variant held a live `activeStreamWs`
138+
reference but had no public way to abort it. `StreamHandler.cancelSpeaking`
139+
/ `handleStop` / `handleWsClose` now call `tts.cancelActiveStream()`,
140+
unblocking the synthesizeStream generator's inner `await Promise<frame>`
141+
loop immediately when the carrier ends the call or the user barges in.
142+
Root cause of the post-hangup 30 s timeout error logs and stale token
143+
billing. Files: `libraries/typescript/src/providers/elevenlabs-ws-tts.ts`,
144+
`libraries/python/getpatter/providers/elevenlabs_ws_tts.py`.
145+
146+
- **Wrapper class TTS `outputFormat` field now conditional.** When an
147+
`ElevenLabsTTS` or `ElevenLabsWebSocketTTS` wrapper receives a carrier
148+
hint (e.g. Twilio), the wrapper's `outputFormat` field is set only if
149+
the caller passed it explicitly. Previous logic always forwarded a
150+
fallback value, which caused the carrier auto-flip to treat
151+
`outputFormat` as explicit and skip the optimization. Now the carrier
152+
auto-flip logic runs correctly: if no `outputFormat` was passed, the
153+
wrapper field remains `undefined`/`None` and the carrier-specific Twilio
154+
path activates naturally. Files: `libraries/typescript/src/tts/elevenlabs.ts`,
155+
`libraries/typescript/src/tts/elevenlabs-ws.ts`,
156+
`libraries/python/getpatter/tts/elevenlabs.py`.
157+
158+
- **`sendPacedFirstMessageBytes` timing rewritten: burst mode, no per-chunk
159+
sleep.** The original implementation paced each prewarm chunk with a
160+
`setTimeout` / `asyncio.sleep` of one chunk-equivalent of playout time
161+
(~40 ms for the 1280-byte default chunk). Combined with the
162+
`waitForMarkWindow` back-pressure await and JavaScript/asyncio timer
163+
jitter, effective delivery dropped BELOW Twilio's 8 kHz playout clock,
164+
producing repeated carrier-side underruns. Caller heard "slow, gravelly,
165+
and arriving more slowly than the rest". Twilio's docs (Media Streams →
166+
WebSocket Messages) state "media messages of any size" are "buffered
167+
and played in the order received" by the carrier-side media server — the
168+
carrier owns the playout clock. Rewrote to burst all prewarm chunks
169+
back-to-back with 20 ms frame granularity (no per-chunk sleep), matching
170+
the live-TTS streaming path that always worked. Per-chunk marks still
171+
emitted for fine-grained barge-in cut. Files: `libraries/typescript/src/
172+
stream-handler.ts`, `libraries/python/getpatter/stream_handler.py`.
173+
174+
- **Mulaw native fast path in audio encode: skip resample + encode when
175+
TTS outputs `ulaw_8000` natively.** When pipeline mode detects
176+
`tts.outputFormat === "ulaw_8000"` on Twilio, `encodePipelineAudio`
177+
skips the resample (16 kHz → 8 kHz) + mulaw encode chain entirely and
178+
base64-encodes the raw bytes. Probed once in `initPipeline` and cached
179+
as `ttsOutputFormatNativeForCarrier`. Saves ~1–2 ms per 20 ms frame,
180+
cumulative ~5–10 % CPU when deployed at scale. Files:
181+
`libraries/typescript/src/stream-handler.ts`, `libraries/python/
182+
getpatter/stream_handler.py`.
183+
184+
- **`handleStop` / `handleWsClose` now abort in-flight LLM and cancel TTS
185+
immediately.** When the carrier ends a call or the StreamHandler is torn
186+
down, both paths now call `llmAbort()` (to unblock any pending LLM stream)
187+
and `tts.cancelActiveStream()` (to unblock any pending TTS stream).
188+
Prevents stale token billing and 30 s timeout error logs from post-hangup
189+
tasks trying to drain a closed WebSocket. Files: `libraries/typescript/src/
190+
stream-handler.ts`, `libraries/python/getpatter/stream_handler.py`.
191+
192+
- **Python SDK parity sync for 2026-05-20 acceptance session.** All TS
193+
fixes landed during PSTN acceptance testing are now ported to Python:
194+
`ElevenLabsTTS.set_telephony_carrier` (HTTP variant, mirrors WS),
195+
`ElevenLabsWebSocketTTS.cancel_active_stream` + `_active_stream_ws`
196+
tracking, `_do_cancel_for_barge_in` / `cleanup` calling
197+
`cancel_active_stream` (duck-typed), `_is_tts_output_format_native_for_carrier`
198+
probe + `_tts_output_format_native_for_carrier` flag + audio-sender bypass
199+
in `PipelineStreamHandler.start`, `_spawn_prewarm_first_message` accepting
200+
`carrier=` and calling `set_telephony_carrier` before synthesis, and the
201+
`tts/elevenlabs.py` wrapper only forwarding `output_format` when explicitly
202+
passed. Files: `libraries/python/getpatter/providers/elevenlabs_tts.py`,
203+
`libraries/python/getpatter/providers/elevenlabs_ws_tts.py`,
204+
`libraries/python/getpatter/stream_handler.py`,
205+
`libraries/python/getpatter/tts/elevenlabs.py`,
206+
`libraries/python/getpatter/client.py`.
207+
208+
- **Bidirectional race guard on `recordTurnComplete` / `recordTurnInterrupted`.**
209+
The original guard (added earlier in this release) was one-directional:
210+
a late `recordTurnComplete` after `recordTurnInterrupted` was dropped,
211+
but the inverse ordering (a late interrupt after a completed turn)
212+
could still overwrite a just-emitted turn record. The current caller
213+
paths can't produce that ordering, but the symmetric guard hardens
214+
the accumulator against future refactors. Both `recordTurnComplete`
215+
and `recordTurnInterrupted` now set `_turnAlreadyClosed`/`
216+
_turn_already_closed` and check it on entry. Same fix in
217+
`libraries/python/getpatter/services/metrics.py` and
218+
`libraries/typescript/src/metrics.ts`; regression tests added in both
219+
suites.
220+
221+
### Fixed
222+
223+
- **Pipeline metrics: `transcript.jsonl` rows after a barge-in carried an
224+
empty `user_text` even when the user had clearly spoken.** Root cause
225+
was a race between the two turn-close paths: a VAD-driven barge-in
226+
fired `record_turn_interrupted` / `recordTurnInterrupted` synchronously
227+
inside the audio handler and `_reset_turn_state` cleared
228+
`_turn_user_text`, while the in-flight pipeline LLM stream kept
229+
unwinding on its own task and eventually reached
230+
`record_turn_complete` / `recordTurnComplete` — which then pushed a
231+
second turn for the same logical exchange carrying `user_text=""`.
232+
Both SDKs now flip a `_turn_already_closed` / `_turnAlreadyClosed`
233+
guard on `record_turn_interrupted` and have `record_turn_complete`
234+
return `None` / `null` until the next `start_turn` re-arms the
235+
accumulator. `_emit_turn_metrics` / `emitTurnMetrics` were already
236+
null-safe, so the late call becomes a silent no-op end-to-end.
237+
Regression tests pinning the bargein → llmAbort → late-complete
238+
ordering live in `libraries/python/tests/test_metrics.py` and
239+
`libraries/typescript/tests/metrics.test.ts`. See
240+
`patter-sdk-acceptance/BUGS.md` (2026-05-05 entry).
241+
242+
- **CI: Security Audit workflow could not upload Bandit SARIF to the GitHub
243+
Security tab.** The `bandit` job in `.github/workflows/audit.yml` was
244+
failing on `github/codeql-action/upload-sarif` with `Resource not
245+
accessible by integration` because the job inherited the repo-default
246+
read-only `GITHUB_TOKEN` permissions. Added an explicit
247+
`permissions: { contents: read, security-events: write }` block on the
248+
job so SARIF findings reach the Security tab as intended. Bumped the
249+
action from `@v3` to `@v4` to drop the deprecation warning ahead of the
250+
December 2026 sunset.
251+
3252
## 0.6.1 (2026-05-15)
4253

5254
### Fixed — `OpenAIRealtime2`: audio transcoding for Twilio + outbound chunking + VAD tuning (TypeScript only)

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,9 @@ await phone.serve({ agent, tunnel: true });
7272

7373
</details>
7474

75-
`tunnel: true` spawns a Cloudflare tunnel and points your Twilio number at it. In production, pass `webhook_url` / `webhookUrl` to the constructor instead. Every carrier and provider reads its credentials from environment variables by default; see each SDK's README for the full catalog.
75+
`tunnel: true` spawns a Cloudflare quick tunnel and points your Twilio number at it — great for dev / acceptance. For production outbound calls (especially on Twilio), replace it with [ngrok](https://ngrok.com) or a static `webhook_url` to avoid WSS upgrade races on first call. See [Tunneling](/docs/dev-tools/tunneling) for details.
76+
77+
Every carrier and provider reads its credentials from environment variables by default; see each SDK's README for the full catalog.
7678

7779
## How Patter compares
7880

dashboard-app/src/App.tsx

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,11 @@ import {
1818
type SparklineResult,
1919
} from './lib/mappers';
2020

21-
const SDK_VERSION = '0.6.0';
21+
// Fallback when the server-side ``aggregates.sdk_version`` field is missing
22+
// (older backend, transient fetch error). Both SDKs (Python + TS) now ship
23+
// the live ``getpatter.__version__`` / ``package.json#version`` in every
24+
// ``/api/dashboard/aggregates`` response.
25+
const SDK_VERSION_FALLBACK = 'dev';
2226
const RANGE_LABEL: Record<RangeKey, string> = {
2327
'1h': '1h',
2428
'24h': '24h',
@@ -128,6 +132,12 @@ export function App() {
128132
const rangeAvgP95 = avgP95(filteredCalls) || aggregates?.avg_latency_ms || 0;
129133
const rangeSpend = totalSpend(filteredCalls) || aggregates?.total_cost || 0;
130134
const phoneNumber = pickPhoneNumber(calls);
135+
// Server-derived SDK version (single source of truth: ``getpatter.__version__``
136+
// in Python / ``package.json#version`` in TS, surfaced via the aggregates
137+
// payload). Falls back when the server side is older than this SPA build.
138+
const sdkVersion =
139+
(typeof aggregates?.sdk_version === 'string' && aggregates.sdk_version) ||
140+
SDK_VERSION_FALLBACK;
131141

132142
const sparkTotalCalls = useMemo(
133143
() => computeSparkline(filteredCalls, 'totalCalls', strategy),
@@ -183,7 +193,7 @@ export function App() {
183193
liveCount={liveCount}
184194
todayCount={totalCount}
185195
phoneNumber={phoneNumber}
186-
sdkVersion={SDK_VERSION}
196+
sdkVersion={sdkVersion}
187197
revealed={revealed}
188198
dark={dark}
189199
onToggleRevealed={toggleRevealed}
@@ -262,7 +272,7 @@ export function App() {
262272
<span className={isStreaming ? 'green' : ''}>
263273
{isStreaming ? 'streaming · sse' : error ? `error · ${error}` : 'idle'}
264274
</span>
265-
<span>SDK · {SDK_VERSION}</span>
275+
<span>SDK · {sdkVersion}</span>
266276
</div>
267277
<div className="group">
268278
<span>

0 commit comments

Comments
 (0)