fix(workflows): forward Twilio CallToken during warm transfers

[piyush-gambhir]

A warm transfer creates a new outbound call, so human recipients normally see the business number rather than the inbound customer's number. This adds per-call caller-ID preservation through Twilio's CallToken, while keeping the business number as the default and fallback. It lets applications use the SDK's consultation and cleanup flow instead of replacing Twilio call creation to preserve customer identity.

API and behavior

• twilio_from_number stays the agent/business Twilio number or verified caller ID.
• Optional original_caller_number carries the original inbound webhook's From.
• A nonempty twilio_call_token opts into preservation and requires the matching original caller number. No separate enable flag or fallback-number option is needed.
• Without a token (including an empty token), dial from the business number even if the original caller number is supplied.
• If the preservation request is explicitly rejected with HTTP 400 / Twilio error 21210, retry once from the business number without CallToken. This error means the original caller ID was not accepted; it was also the response observed for the invalid-token live control.
• Do not retry transport timeouts, unrelated rejection codes, authentication/rate-limit errors, or failures after a call was created. A failed fallback propagates normally. This avoids duplicate dialing after an ambiguous outcome.

Keep the number/token pair in server-side state keyed by the incoming CallSid, obtained from a validated Twilio webhook. Tokens are not included in connector requests, TwiML, prompts, or participant attributes. The token is not permission to choose arbitrary caller IDs.

This can help recipients identify the customer and support caller-ID-based lookup/callback records; the PR does not implement CRM matching. Caller-ID presentation remains subject to Twilio acceptance and downstream carrier behavior. The existing Calls API is used, with no conference or LiveKit server API change.

Twilio references

• Incoming TwiML request parameters: From, CallSid, and CallToken.
• Calls API: create a call: the endpoint used for the transfer leg.
• Trusted calling / forwarding: matching the inbound token and caller number.
• Error 21210: unverified From: the narrowly scoped fallback condition.
• Conference CallToken changelog and create a conference participant: related support for conference flows; this PR does not require a conference.

JavaScript counterpart: livekit/agents-js#2510.

Compatibility and validation

• Token use requires twilio-python 6.55.0+. The helper checks the installed client's calls.create signature before connector creation/dialing and gives an upgrade instruction if unsupported. Token-free transfers still work with older clients.
• 34 focused workflow tests passed, covering compatibility, forwarding/omission, original-caller validation, single fallback, failed fallback, unrelated errors, and cancellation of the fallback call.
• make fix, repository Ruff checks, and focused strict mypy passed. Full make check remains blocked at optional plugin typechecking (livekit.plugins.anam missing its type marker in this core-only environment).
• Prior compatibility checks used real 6.54.0 and 6.55.0 clients with mocked HTTP transport; the current live run uses 9.11.1.

Live QA — 2026-09-16, current fallback behavior

Ran both local SDK workers against UAT LiveKit Cloud and real Twilio calls. Two temporary phone numbers were used: a customer/receiver in a temporary subaccount and a business number in the SDK account. This makes the customer's number unverified in the dialing account. Each SDK passed all four cases below (8/8 total):

Input	Observed result
Valid token + original caller	Transfer completed; receiver saw the customer caller ID.
No token, no original caller	Transfer completed; receiver saw the business caller ID.
Original caller supplied, no token	Transfer completed; receiver saw the business caller ID.
Invalid token + original caller	Preservation rejected; fallback completed with the business caller ID.

Every successful case had one receiver webhook and published audio tracks for the customer and transferred human in the caller room. The invalid-token Python attempt also recorded the SDK's explicit caller-ID fallback warning. Unit tests verify both request bodies, the exact single-retry limit, and failure exclusions.

One Python harness attempt raced room initialization before dialing. After adding an explicit room connection in the harness, that case passed using the same temporary numbers; interrupted harness attempts are excluded from the final matrix. No SDK behavior was changed to work around that harness issue.

Independent API reads confirmed both purchased numbers released (404), the temporary subaccount closed, 38 associated call legs checked with zero active, and zero QA rooms remaining. Local QA workers and the webhook tunnel were stopped.

Earlier live QA on the pre-fallback revision also covered supervisor decline, no answer, busy/rejected calls, and caller hangup during ringing in both SDKs. Those lifecycle cases are historical validation, not a fresh full-matrix run of this revision. Current regression tests cover cancellation of an unanswered fallback call.

Limits: automated endpoints verify caller-ID signaling and published audio tracks, not physical handset display, subjective two-way audio quality, or every carrier. Telnyx is not included in these results.

Cancellation and cleanup deadlines

The synchronous Twilio HTTP request keeps running if its asyncio.to_thread await is cancelled. Initial and fallback creation retain the pending result so a late call SID can still be canceled. Cleanup has a five-second absolute wait deadline, which repeated cancellation cannot reset. After that deadline, transfer teardown proceeds and a strongly referenced background task continues best-effort cleanup while the worker is alive. Cancellation of an already-created, unanswered call uses the same bounded wait.

Twilio's HTTP client is configured with a ten-second socket timeout for creation and cancellation requests. This is separate from the teardown deadline; it is not a guarantee of total wall-clock request duration. Network/provider failure or worker termination can still prevent cleanup, and an ambiguous failed creation request is never redialed.

The focused suite passes 34 tests. Four creation-cancellation regression cases cover initial/fallback requests, late success/rejection, and repeated cancellation. Two additional real-thread barrier tests hold creation or cancellation stalled, verify teardown completes before releasing the blocked worker, then verify late cleanup completes and its background ownership is released. Ruff and focused strict mypy pass. Full repository typechecking remains blocked by the optional-plugin environment limitation described above.

These cancellation and deadline fixes were tested deterministically, without another live PSTN run. The live caller-ID matrix above applies to the preceding fallback revision. JavaScript does not use Python's thread-cancellation mechanism; this change is scoped to the Python review finding.

Cancellation propagation during failure cleanup

Cancellation received while handling an unanswered transfer's ToolError is now retained and re-raised after cleanup completes or the existing absolute deadline expires. Repeated cancellation does not reset that deadline. With no cancellation, the original transfer error still propagates. Late cleanup ownership is unchanged.

Corrected the prior stalled-cleanup test, which incorrectly expected ToolError despite shutdown cancellation. Regression checks cover both cleanup completion and deadline expiry; both cancellation cases failed on the preceding revision and pass now. The focused suite passes 34 tests, with Ruff and focused strict mypy passing. No new live calls were placed for this deterministic cancellation regression.