Back to app

callqa CLI

One Python file, no dependency (Python 3.9+). Copy it where you need it.

cp examples/cli/callqa ./callqa && chmod +x callqa
export CALLQA_API_KEY="cqa_..."           # scopes: read, run (+ write to import scenarios)
./callqa me

Commands

# trigger a run and return its id
./callqa run --scenario 5f0f7c9e-...

# CI: trigger, wait, gate on the score, write a Markdown summary
./callqa run --scenario 5f0f7c9e-... --threshold 80 --code-version v2.3.0 --tag ci \
  --wait --timeout 1800 --summary summary.md

# only some variants
./callqa run --scenario 5f0f7c9e-... --variant 2b9d... --variant 8f41... --wait

# a run triggered earlier: show it, and gate on it if finished
./callqa status 7b0c1f8e-...

# replay bundles
./callqa bundle export 0d7e... -o bundle.json
./callqa replay --bundle bundle.json --wait
./callqa replay --call 0d7e... --fresh-runtime

# create or update a scenario from YAML (upsert by name)
./callqa scenario import ../scenarios/ivr-dtmf-navigation.yaml

Messages go to stderr, results to stdout: RUN=$(./callqa run --scenario ...) captures the run id. --json prints the run as JSON.

Option Env Default
--api-key CALLQA_API_KEY required
--api-url CALLQA_API_URL https://app.callqa.dev/api/v1

Exit codes

Code Meaning
0 Success. With --wait: the run finished and passed the gate
1 Quality gate failed: score under --threshold, verdict not pass without threshold, or a call in error or blocked
2 Usage error: missing key, bad argument, unreadable file
3 API or network error (401, 403, 404, 422, 5xx), details on stderr
4 Timed out while waiting. The run continues on the server
5 The run was canceled

Behaviour

  • Polls GET /runs/{id} every --poll seconds (10). Retries 429 (honouring Retry-After), 502, 503, 504 and network errors on reads. Never retries a trigger, except after 429, so it cannot start two runs by accident.
  • --summary FILE appends a Markdown report (verdict, score, counters, one line per test case, link to the dashboard). Pass $GITHUB_STEP_SUMMARY on GitHub Actions.
  • --outputs FILE appends run-id, run-url, verdict, score, passed as key=value lines. Pass $GITHUB_OUTPUT on GitHub Actions.

Tests: tools/tests/test_cli.py runs it against a simulated API.

Files in examples/cli/