# Writing scenarios

A scenario is a YAML (or JSON) document. This guide shows how to write, validate and manage
them. Every field is described in the [grammar reference](reference/scenario-grammar.md).

In the dashboard (**Scenarios**, **New scenario** or **Edit**), the editor shows the same
document in two synchronized views: a **Human view** in cards (persona, goal, IVR steps,
checks, test cases) and the **YAML**, validated as you type. While you edit an existing
scenario, a valid document is saved automatically; **Save** saves it at once.

## The short form

The format shown on callqa.dev is valid as is:

```yaml
name: "Book appointment - edge case"
phone: "+14155550100"
schedule: "every day at 09:00 UTC"
call_window: "09:00-18:00 America/New_York"
persona: "Impatient customer, speaks fast, has accent"
goal: "Book a haircut for tomorrow 2pm"
must_include:
  - "confirm date and time"
  - "provide booking reference"
must_not:
  - "hang up before confirmation"
  - "hallucinate availability"
max_latency_ms: 2000
```

## The full form

The full form gives you typed assertions (with ids and severities), IVR steps, limits,
variants and extension fields:

```yaml
schema_version: 1
name: "IVR - check account balance"
target:
  phone: "+14155550103"
  kind: ivr
  platform: genesys
persona:
  description: "Existing customer who knows the menu and presses keys quickly"
  patience: high
goal: "Reach the account balance message through the IVR menu"
steps:
  - type: expect
    text: "welcome message and main menu options"
  - type: dtmf
    digits: "2"
  - type: expect
    text: "asks for the account number followed by the pound key"
  - type: dtmf
    digits: "12345678#"
assertions:
  - id: balance-read
    type: transcript_regex
    pattern: "balance (is|of)"
  - id: reached-goal
    type: goal_reached
  - id: fast-tree
    type: max_duration_s
    value: 120
    severity: warn
limits:
  max_duration_s: 180
tags: ["ivr", "dtmf"]
x_owner: "payments-team"
```

More complete examples in [`examples/scenarios/`](../../examples/scenarios/).

## Writing good checks

- **Describe behaviour, not wording.** `must_include: "confirm date and time"` passes if the
  agent confirms in any words. Use `transcript_regex` only when the exact words matter (a legal
  disclaimer, a reference format).
- **One idea per assertion.** "Confirm the date and give a reference" is two assertions: you
  will know which one failed.
- **Give stable `id`s** to assertions you track over time (`id: confirm-date`): the dashboard
  can follow the same check across runs and edits.
- **Use `severity: warn`** for nice-to-haves (tone, empathy, speed): they lower the score and
  mark the call `warn`, without failing it.
- **Set a latency budget** with `max_latency_ms`. It is measured per turn, from the end of the
  caller's speech to the start of the target's answer, at `p95` by default.
- **Put test data in `test_data`**, not in the goal: the caller uses it when asked, and the
  judge can detect answers that contradict it.
- **Use variants for edge cases**: same scenario, different persona or goal, extra
  assertions. The base assertions apply to every variant.

## Validate before saving

`POST /scenarios/validate` checks a JSON document without saving it. It returns the normalized
document (short form folded, defaults filled in):

```bash
curl -sS -X POST https://app.callqa.dev/api/v1/scenarios/validate \
  -H "Authorization: Bearer $CALLQA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Smoke", "phone": "+14155550100", "persona": "Calm caller", "goal": "Ask the opening hours"}' \
  | jq '{valid, warnings, assertions: .normalized.assertions}'
```

Errors come back as `422` with one entry per problem:

```json
{
  "detail": "invalid scenario",
  "code": "invalid_scenario",
  "errors": [
    { "loc": "assertions.0.value", "msg": "Input should be greater than 0" },
    { "loc": "persona", "msg": "Value error, unknown field(s) ['mood']; custom fields must start with 'x_'" }
  ]
}
```

`loc` is the dotted path of the field (`variants.2.assertions.0.text`).

## What the server checks

On top of the grammar (types, required fields, unknown fields, E.164, DTMF characters, regex
syntax), the server refuses a scenario when:

| Check | `loc` |
|---|---|
| The number is not a valid, dialable number | `target.phone` |
| The number is emergency, premium rate, shared cost, personal, pager, voicemail or unclassified | `target.phone` |
| `schedule` is neither a cron expression nor an accepted English form | `schedule` |
| `timezone` is not an IANA time zone | `timezone` |
| `timezone` and the time zone written in `schedule` differ | `schedule` |
| `call_window` is not `HH:MM-HH:MM`, optionally followed by an IANA time zone | `schedule` |
| A schedule or call window in `environments.<slug>` is invalid | `environments.<slug>.schedule` |
| `environments` names an environment that does not exist | `environments` |
| Two variants have the same `title` | `variants` |
| Another non-archived scenario has the same `name` | `name` |

And answers `403` with `code: feature_not_in_plan` (or `country_not_in_plan`), the plan
that lifts the limit and an upgrade link, when:

- the scenario or a variant uses `test_data` and your plan does not include custom test data
  injection (Growth and above);
- a variant has `source: auto` and your plan does not include auto-derived coverage (Growth
  and above);
- the number is in a new country and your plan's country limit is reached.

Every new content of a scenario also goes through a [compliance review](compliance.md)
(acceptable use policy): it is saved either way, but only an allowed scenario can run. The
verdict and its reasons are in the `compliance` object of the API and MCP answers and on the
scenario page.

## Schedules and call windows

`schedule` accepts a 5-field cron expression, in UTC, optionally followed by a time zone, or a
plain English form. Set the time zone with the optional `timezone` field (an IANA name such as
`Europe/Paris`, `UTC` by default) or at the end of the `schedule` text; if you give both they
must be the same:

```yaml
schedule: "0 8 * * 1-5"
timezone: "Europe/Paris"
```

Times are local wall-clock times: the example runs at 08:00 in Paris all year, that is 06:00
UTC in summer and 07:00 UTC in winter. On the night the clocks go forward, a time that does not
exist that day (02:30 in Paris, for example) runs at the next valid minute; on the night they go
back, a time that happens twice runs once (an hourly schedule still runs every real hour).

| `schedule` | Runs |
|---|---|
| `"0 9 * * *"` | every day at 09:00 UTC |
| `"30 8 * * 1-5 Europe/Paris"` | weekdays at 08:30, Paris time |
| `"30 8 * * 1-5"` with `timezone: "Europe/Paris"` | the same |
| `"every day at 09:00 UTC"` | every day at 09:00 UTC |
| `"every weekday at 08:30 America/New_York"` | Monday to Friday at 08:30, New York time |
| `"every monday at 07:00 Europe/London"` | Mondays at 07:00, London time |
| `"every hour"` | every hour, on the hour |

`call_window: "09:00-18:00 America/New_York"` keeps **scheduled** calls inside business hours:
a scheduled call that comes due outside the window waits for the next opening. Windows can
cross midnight (`"22:00-06:00 UTC"`). Runs you trigger by hand, by API or from CI are not held
by the window.

Archiving a scenario stops its schedule.

### One schedule per environment

A scenario runs on its own schedule in each environment where it is activated (see
[Environments](environments.md#which-scenario-runs-where)). In the document, the top-level
`schedule`, `timezone` and `call_window` are the plan of your default environment, and
`environments` holds the plans of the other ones, keyed by slug. Each slug must be an
environment of your organization (the example needs `staging`: **Add development and
staging** on the **Versions and environments** board creates it); an unknown slug is refused
with the list of your environments, never created on the fly (see
[Environments](environments.md#which-scenario-runs-where)):

```yaml
name: "Opening hours - every environment"
phone: "+14155550100"
persona: "Calm caller"
goal: "Ask the opening hours"
schedule: "every weekday at 08:30 America/New_York"
environments:
  staging:
    schedule: "every hour"
    timezone: "America/New_York"
```

On the scenario page, each environment has its own schedule field. There you can also type a
schedule in plain words the parser does not know: CallQA proposes a reading (for example
"Every day at 04:00 UTC, cron 0 4 * * *") and saves it only once you confirm it. The API accepts only cron expressions and the forms in the table above.

## Create, import, update, export

| Task | Request | Scope |
|---|---|---|
| Create from JSON | `POST /scenarios` | `write` |
| Create or update from YAML (upsert by `name`) | `POST /scenarios/import` with `Content-Type: application/yaml` | `write` |
| Replace | `PUT /scenarios/{id}` | `write` |
| Read | `GET /scenarios/{id}` | `read` |
| Export | `GET /scenarios/{id}/export?format=yaml` (or `json`) | `read` |
| List | `GET /scenarios?tag=booking&q=appointment&include_archived=false` | `read` |
| Archive | `POST /scenarios/{id}/archive` | `write` |
| Delete | `DELETE /scenarios/{id}` | `admin` |

- Export and import round-trip: keep your scenarios in Git and sync them from CI with
  `callqa scenario import file.yaml` (see the [CLI](../../examples/cli/)).
- On update (`PUT` or import), variants are matched **by title**: a variant that keeps its
  title keeps its id and its history. A variant removed from the document is deleted.
- Prefer **archive** to delete. An archived scenario keeps its history, stops its schedule and
  frees its name. Deleting (admins only) is permanent: past runs and calls are kept, but they
  lose their link to the scenario and its variants.
- Unknown `x_*` fields are stored and returned as is by `GET` and export.

## Variants

| Task | Request | Scope |
|---|---|---|
| List | `GET /scenarios/{id}/variants` | `read` |
| Add a manual variant | `POST /scenarios/{id}/variants` | `write` |
| Enable, disable or replace | `PATCH /variants/{variant_id}` with `{"enabled": false}` or `{"spec": {...}}` | `write` |
| Delete | `DELETE /variants/{variant_id}` | `admin` |
| Auto-derive N variants | `POST /scenarios/{id}/variants/derive` with `{"count": 4}` | `write` |
| From a real call | `POST /scenarios/{id}/variants/from-bundle` | `write` |

```bash
curl -sS -X POST "https://app.callqa.dev/api/v1/scenarios/$SCENARIO/variants" \
  -H "Authorization: Bearer $CALLQA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "title": "Asks for Sunday 7am",
        "goal": "Ask for Sunday 7am (closed), then accept the first slot offered",
        "assertions": [{"type": "must_not", "text": "offer a slot outside opening hours"}]
      }'
```

Titles are unique within a scenario (`422` otherwise).

## Generate scenarios from a description

`POST /scenarios/generate` proposes scenario documents from a plain description of your
service. Nothing is saved: review the proposals, then create the ones you keep.

```bash
curl -sS -X POST https://app.callqa.dev/api/v1/scenarios/generate \
  -H "Authorization: Bearer $CALLQA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"description": "Hair salon booking line: book, move, cancel.", "phone": "+14155550100", "count": 5}' \
  | jq '.scenarios[].name, .rejected'
```

Proposals that do not pass validation are dropped and counted in `rejected`. When the
generator is not available the endpoint answers `503`.

## Edit a scenario in plain words (assistant)

In the dashboard, the scenario editor and the new scenario page have an **Assistant** box.
Type or dictate what you want ("I want this test to pass even if the salon offers 2 pm",
"add a variant where the customer is in a hurry"); the assistant writes the new scenario,
checks it against the grammar and your plan, and shows a diff with the saved version.
**Accept and save** saves it (audited as a normal edit), **Reject** drops it, and a follow-up
request refines the pending proposal.

- A session holds up to 15 requests; a dictation lasts 3 minutes at most. The audio is
  transcribed and not stored.
- On a call that failed, warned or ended in error, **Fix this test** opens the assistant with
  the failure (checks, quotes, transcript) and starts at once: it proposes to relax a check
  that was too strict, or to add a test case that reproduces the bug.
- **Done, back to the scenario** closes the session. Nothing is saved without **Accept**.
- Limits per organization: requests and dictations per hour, and a daily budget. The
  assistant is not billed per request.

## Build a matrix of test cases (Scale)

On a scenario page, **Build a matrix of test cases** creates many test cases at once: every
combination of personas, patience levels, test data sets and variations (extra instructions)
becomes one manual test case. **Preview** lists the combinations, then **Create** adds them.
At most 50 test cases per build and 20 values per field; titles that already exist are
skipped. Available on the Scale plan.