Replay and bundles

A customer complains about a call. You want to reproduce it exactly, attach it to a bug report, and make sure it never breaks again. That is what replay bundles are for.

What is in a bundle

A bundle is one self-contained JSON file:

Field	Content
`bundle_version`	Format version, currently `1`
`exported_at`	When the bundle was built
`call_id`	The original call
`org`	Your organization slug
`run`	The resolved run: everything the caller needed for this one call (target, persona, goal, instructions, test data, steps, assertions, limits), after merging the scenario and the variant
`runtime`	What really ran: caller agent version, speech-to-text, LLM and text-to-speech providers, models and parameters, turn detector, voice activity detection, seed, caller id
`events`	The full trace
`result`	The verdicts, metrics, score and recommendations (`null` if the call was not judged)
`notes`	The error message, when the call failed

A bundle contains your test data and the transcript: treat it like any other test artifact with customer-like data.

Export

curl -sS -H "Authorization: Bearer $CALLQA_API_KEY" \
  -o bundle.json "https://app.callqa.dev/api/v1/calls/$CALL/bundle"

# or with the CLI
callqa bundle export "$CALL" -o bundle.json

Scope read (members and admins in the dashboard, where the button is Export bundle). Every export is recorded. A number verification call, or a call whose content was deleted (retention period, or Data and privacy history deletion), cannot be exported nor replayed (404).

Replay

Replaying creates a new call with exactly the same resolved run:

# from a call of your organization
curl -sS -X POST https://app.callqa.dev/api/v1/replays \
  -H "Authorization: Bearer $CALLQA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"call_id\": \"$CALL\"}"

# from a bundle file
jq -n --slurpfile b bundle.json '{bundle: $b[0], same_runtime: true}' \
  | curl -sS -X POST https://app.callqa.dev/api/v1/replays \
      -H "Authorization: Bearer $CALLQA_API_KEY" \
      -H "Content-Type: application/json" --data-binary @-

# with the CLI, waiting for the verdict
callqa replay --bundle bundle.json --wait

Scope run. Answers 201 with the new call (replay_of points to the original call when it belongs to your organization).
same_runtime: true (default) reuses the bundle's models, voices and settings: the closest possible reproduction. false uses today's default runtime: useful to check whether a newer caller behaves differently.
The replay is a new run with trigger: "replay" and one call, in the environment of the original run (the default environment for a bundle file). In the dashboard: Replay this call.
The bundle's target number must be a registered, non-archived number of your organization (400 otherwise). A bundle can never be used to call an arbitrary number. The usual checks apply: verified number, credit, concurrency.
A bundle with a newer bundle_version than the server supports is refused (400).

A phone conversation is never perfectly deterministic: your agent, the network and the caller's speech recognition all vary a little. Same run and same runtime make the replay as close as possible, not bit for bit identical.

Turn a call into a regression test

Found a bug in a call? Add it to the scenario as a MANUAL variant (a test case), so every future run covers it. On the call page, the Add to regression suite card: pick the scenario, then Add test case. To also change a check, use Fix this test instead (see the assistant).

curl -sS -X POST "https://app.callqa.dev/api/v1/scenarios/$SCENARIO/variants/from-bundle" \
  -H "Authorization: Bearer $CALLQA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"call_id\": \"$CALL\", \"title\": \"Regression: hallucinated 3pm slot\"}"

You can pass bundle instead of call_id. The new variant copies the persona, goal, instructions and test data of the call. It does not copy assertions: it runs with the scenario's assertions, and you can add specific ones with PATCH /variants/{id}. Without a title, it is named Regression: <variant or scenario> (<first 8 characters of the call id>).

Scope write. Test data in the variant requires a plan with custom test data injection.

Export a failure for an LLM

Every call, run and scenario has an LLM export: a report made to be pasted into a chat assistant or a coding agent that will fix your voice agent. It holds the scenario, the results, every failed check with the judge's quote and timestamp, a readable transcript (AGENT is your agent, CALLER our test caller), the metrics, the judge's recommendations and a primer that asks for a fix.

In the dashboard: the Export menu of a call, a run report or a scenario: Copy for an LLM, or download the report as .md or .json.
From the API: GET /calls/{id}/llm-export, GET /runs/{id}/llm-export, GET /scenarios/{id}/llm-export. format=markdown (default, text/markdown) or format=json (the same document as compact JSON: format, version, level).

A run report details up to 5 failed calls; a scenario report covers its last 5 runs, the checks that fail most and its last 3 failed calls. Transcripts longer than 80 turns are cut in the middle.