> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tryhamsa.com/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B tests: try a new version on a share of calls

> Send a share of prod calls to a candidate version, keep each caller on one version, compare how calls end and how long they last, then roll out or stop.

An A/B test sends a share of your live calls to a new version while the rest stay on the version callers hear now. You compare how the two do, then send all calls to the new version or stop the test. A/B tests run on prod only, so the agent needs deployments on.

## Before you start

* The agent has deployments on, and prod runs a version. "Deploy a version to prod first." See [Environments and releases](https://docs.tryhamsa.com/agents/release/environments.md).
* You have a second version to test against the live one.
* You have the Experiment permission. Owners and admins have it by default; members don't. Ending a test by sending all calls to the candidate also changes what prod runs, which needs the Deploy permission.

## Start an A/B test

**1. Open Releases**

Click the version chip in the editor header.

**2. Promote the candidate to prod**

Click **Promote \{version} to prod** on the staging lane, or pick a version from the prod lane's menu.

**3. Choose Start with a share**

Under **Who gets it**, pick **Start with a share** instead of **All calls**. "The rest stay on" the live version.

**4. Set the share**

Drag **Share of calls**. The candidate can get 5% to 50% of calls, in steps of 5. It starts at 10%.

![The Promote v8 to 10% of calls confirmation with Start with a share selected and the share slider at 10%](https://mintcdn.com/hamsa-452fa79a/2654KIaxnTY04VOj/images/agents/release/ab-test-start-light.webp?fit=max\&auto=format\&n=2654KIaxnTY04VOj\&q=85\&s=10e1c0d955cd90342d44800ee4b0029f)

**5. Start the test**

Click **Start test at \{percent}%**. "\{percent}% of live calls get \{version}. The rest stay on \{current}."

While the test runs, the editor's version chip says "Callers hear" both versions and their shares, and the prod lane says "Getting \{weight}% of prod calls".

## How callers are split

"Each caller keeps hearing the same version while the test runs: phone callers by their number, web and app callers by the id their page sends."

* **Raising the share** only adds callers. "Callers already on \{version} stay on it; new ones join until it reaches \{percent}%."
* **Lowering the share** moves only the callers above the new share back. "Only the callers above \{percent}% go back to \{current}; the rest stay on \{version}."
* **A new candidate is a new test.** Starting one with another version shuffles callers again.
* **A web session with no id** is assigned at random each time.
* "If \{version} cannot take a call, \{current} answers it."
* The share shows over many callers, not a few test calls.

## Compare the two versions

The comparison appears in **Releases** under the release path while prod splits. It counts "Calls this test sent to each version since it started. Test calls are left out."

| Chart | What it shows |
| - | - |
| Calls split | How many calls each version got, against the target share |
| **How calls ended** | **Completed** (ended normally), **Transferred** (handed to a person) and **Ended otherwise** (failed, unanswered or stopped), out of the calls that are over |
| **Call length** | Each version's recent calls as dots, with the median marked. Chats have no length. |

![The A/B test comparison: 354 calls split between v7 and v8, how calls ended for each, and call length dots with the medians](https://mintcdn.com/hamsa-452fa79a/2654KIaxnTY04VOj/images/agents/release/ab-test-compare-light.webp?fit=max\&auto=format\&n=2654KIaxnTY04VOj\&q=85\&s=8e9f0b343042723e4e9041b2d2a1e74a)

Until each version has 30 calls, it says "Too few calls to compare yet. Wait for 30 on each version." Click **See these calls** to open them.

To see A/B test calls anywhere, filter the [Conversation log](https://docs.tryhamsa.com/agents/monitor/conversation-log.md) by **A/B test** > **Only A/B test calls**, or use **A/B test calls** in the agent's **Calls**. A call's details say "The A/B test on prod chose this version."

## Change or end the test

From the prod lane:

| Action | What happens |
| - | - |
| **Change share** | Set a new share for the candidate. |
| **Send all calls to \{version}** | The candidate becomes what prod runs. "\{version} stops answering. Roll back returns to it." |
| **Stop test** | "All calls go back to" the live version. The candidate "stops answering. It stays in history." |

Deploying or promoting another version into prod also ends the test: "This ends the A/B test running there."

## Good to know

* A/B tests run on prod only: "A/B tests run on the live environment only".
* The candidate and the live version each read prod's [environment variables](https://docs.tryhamsa.com/agents/variables/environment-variables.md).
* Test calls never count toward the comparison.

## Related

> [card] [Environments and releases](https://docs.tryhamsa.com/agents/release/environments.md)
>
> Promote, switch and roll back versions.

> [card] [Drafts and versions](https://docs.tryhamsa.com/agents/release/drafts-and-versions.md)
>
> Publish the version you want to test.

> [card] [Conversation log](https://docs.tryhamsa.com/agents/monitor/conversation-log.md)
>
> Find and filter calls, including A/B test calls.

> [card] [Analytics](https://docs.tryhamsa.com/agents/monitor/analytics.md)
>
> Track how the agent does over time.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.