A/B tests management
This article explains how to manage a/b tests within the list and describes all status parameters and individual a/b test details.
Step-by-step instructions
Navigate to the "Outbound" section.
Click the "A/B tests" button.
Use of the filter button to select the channel (All, Email, WebPush, Telegram, Viber).
Filtering of A/B tests by status (Draft, Waiting, Ready, Running, Done, Stopped, Failed).
Sorting of A/B tests by name or test component for more efficient workspace organization.
Use the search bar to find specific A/B tests by their name or unique ID.
Access to campaign performance and key metrics via the "Analytics" button.
Review of A/B test configuration details through the "Info" button.
Access to the "More" menu for duplicating existing test settings or performing a permanent deletion.
Bulk removal of tests by using the selection boxes next to the rows and choosing the delete option.
Editing of A/B test settings by clicking directly on the name of the desired test.
-
Statuses
- Ready: The test is ready for sending, can be launched, and remains available for editing before the start. Launch types: Manual.
- Running: The test is currently sending (the stop button is active); editing is not possible during this phase. Launch types: Now, Manual, Schedule.
- Waiting: The test is scheduled and waiting for its time; stopping and editing are available. If restarted after the scheduled time has passed, a notification is displayed. Launch types: Schedule.
- Stopped: The test was halted before the sending process completed; it can be edited and relaunched. Launch types: Schedule.
- Done: The test is completed; editing and re-launching are unavailable. Launch types: Now, Manual, Schedule.
- Draft: The test is inactive; created after proceeding to the second setup step. Editing is available, but the test cannot be launched. Launch types: Not selected.
- Failed: The test was not sent; the specific reason for failure is available via the Info button. Editing is permitted. Launch types: Now, Manual, Schedule.
-
Details
- Channel - the type of communication channel used for the A/B test.
- Status - the current operational status of the campaign.
- Failure Reasons - detailed error logs that appear when a campaign cannot start due to:
- Templates: Missing, deleted, or modified Email, WebPush, Telegram, or Viber templates; Twig syntax errors.
- Sender Details: Deactivated, paused, or deleted Viber Sender Names; unavailable or deleted Telegram Bot tokens.
- Audience: Deleted contact lists; insufficient audience size to split into variants (less than 2 contacts); deleted variables in segment filters.
- Name - the name of the A/B test campaign along with its unique ID.
- Creation date - the timestamp of the test creation based on the account's time zone.
- Template (A/B/C) - the name and ID of the template(s) assigned to the campaign.
- Email subject (A/B/C) - the subject line(s) undergoing testing for the Email channel.
- From name (A/B/C) - the sender name(s) being tested or used for the campaign.
- Sender Name - the identified Sender Name displayed to contacts in Viber.
- Telegram bot - the name of the selected Telegram bot used for the test.
- Contact list - the name and ID of the targeted Contact List.
- TTL (Time to live) - the duration for which WebPush notifications or Viber messages remain active for delivery.
- Segments / Exclude segments - specific contact segments included in or omitted from the target group.
- Filtered by / Excluded by filter - temporary filter conditions used for specific audience segmentation in this test.
- Launch type - the method used to initiate the A/B test.
- Launch time (A/B/C) - the overall scheduled start time or variant-specific times for "Launch time" testing.
- Criterion & Duration - the defined winning metric and the timeframe for data collection to determine the winner.
- Winning message send time - the scheduled date and time for automatic dispatch of the winning variant.
- Tags - the list of organizational labels assigned to the A/B test.
