Our Advanced Testing feature is a multivariate testing tool that allows you to test all elements on email send in order to elicit the highest engagement from your audience. The process allows you to define your test segments and select the criteria used to determine the winning test segment. The elements from the winning test segment will then be applied to the winner segment of the Advanced Test.
An Advanced Test can be configured via multiple paths in the platform.
- Select Send Advanced Test from the Actions Menu for the email in the Email Manager
- Adding an Advanced Test activity to the Process Automation canvas
- Adding an Advanced Test activity to the Journey Automation canvas.
The configuration of an Advanced Test does vary between a Process Automation and Journey Automation.
Configuring an Advanced Test for a Process Automation
Advanced Test activities consist of four separate tabs that need to be configured in order to send the email. Many of the elements within these tabs can be found in a standard Email activity and AB Test sends.
Configuration Step 1
To begin configuring your Advanced Test, fill out the information in the initial tab. This tab consists of the basic information for the Advanced Test and allows you to define your audience.
Email Send Configuration
| Field Name | Description |
|---|---|
| Name | The name given to the automation and Advanced Test. This is editable and can be changed as needed. |
| Description | Enter a description for this Advanced Test. |
| Campaign Tag | Select a Campaign Tag for the Advanced Test. |
| Reports Folder | Select the folder for the Advanced Test to reside in. |
| Track Opens | Enable Open Tracking for the Advanced Test. This is set to Yes by default. |
| Track Links | Enable Click Tracking for the Advanced Test. This is set to Yes by default. |
| Web Tracking | Enable Web Tracking for the Advanced Test. If Web Tracking is configured for your client, this option will be defaulted to Yes. |
| Web Analytics | Enable Web Analytics for the Advanced Test. If Web Analytics is configured for your client, this option will be defaulted to Yes. |
| Send Classification | Select the Send Classification for the Advanced Test. |
Advanced Options
| Field Name | Description |
|---|---|
| Contact May be Sent This Activity Only Once |
If selected, any contacts who have passed through this automation previously will be suppressed. |
| Deduplicate Audience By Email Address | Deduplicates the audience by email address ensuring one email is sent to each recipient. This option is enabled by default. |
| Use Recipient's Preferred Email Frequency When Known | Allows you to send based on the recipient's email frequency if this feature is enabled on your account. |
| Email Frequency | Allows you to suppress sending to contacts who have received an email in the previous X number of days. |
| Opt-out Override | If yes, send messages regardless of a contact's opt-out status. The is configured to No by default. If the Send Classification is configured to Transactional, this will update to Yes. |
| Fail MX Rejections | If yes, do not attempt to retry delivery of messages because of a MX rejection, e.g, 554 Transaction Failed. This is set to No by default. |
| Fail DNS Timeouts | If yes, do not attempt to retry delivery of messages because of a DNS timeout, e.g, not able to reach host. This is set to No by default. |
| Never Retry Delivery | If yes, the system will bypass the email retry process. |
Audience Selection
| Field Name | Description |
|---|---|
| List(s) | Use this field to select Static or Dynamic Lists that will be used in the audience for the Advanced Test. |
| Sendable Data Table(s) | Use this field to select Sendable Data Tables that will be used in the audience for the Advanced Test. Data Tables must be configured in your client to use this option. |
| Coupon Table | Select the iPost Coupon Table the content is using for the Coupon iTL. |
| Suppression List(s) | Use this field if you would like to suppress contacts found on a list. |
| Suppression Data Table(s) | Use this field if you would like to suppress contacts found in a sendable data tables. |
| MD5 Suppression List(s) | Use this field if you would like to suppress records found on a MD5 Suppression List. |
Please note, a minimum of 100 contacts are required to configure an advanced test.
Once you have completed the Configuration Step, click Next to move to the Segments Step.
Segments Step
In the Segments Step tab, you can select the email content that will be used in the Advanced Test. You have the option of using the same email content for the test segments, or selecting multiple emails to be used in the Advanced Test.
If you started the Advanced Test from a specific email, the details of that email will populate in the Segment A field. If you started the Advanced Test from a Process automation, click the Email Name field in order to select the email content for the test. Once selected, the various fields will be populated form the email content.
The Duplicate Icon allows you to add more Test Segments to your Advanced Test. The Trash Can icon allows you to delete a Test Segments.
All of the fields for each Segment are configurable for testing. This allows you to not only test different content, but also any element of the email send itself.
Once the Segments Step has been completed, click Next to move to the Distribution Step.
Distribution Step
The Distribution tab allows you to determine the size of your test audience and the distribution of the test audience between test segments. You can show the distribution size in Numbers or Percentage by clicking the box next to the corresponding type. Use the slider to adjust the audience size in the Total Distribution or Segments Distribution sections.
Make sure you are testing an appropriate audience size in order to receive the best possible results.
Total Distribution is equal to the total size of your sending audience. In this section, you can determine how much of that audience will be used for the testing audience and how much will be used for the winner send. The total of the Total Distribution section will always equal to 100% of your total audience.
Segments Distribution is equal to the Testing total in the Total Distribution section. You can then split the total Testing audience among the various Segments configured in the Segments Step. The total of the Segments Distribution section will always equal to 100% of the Testing amount.
In the example above, 30.1% of the total audience will be used for testing. The remaining 69.9% will be used for the winner send. From the 30.1% allotted for Testing, each test Segment will be sent to 45 contacts.
Once you have completed the Segments Step, click Next to move to the Test Configuration Step.
Test Configuration
The Test Configuration tab allows you to configure your test criteria for the Advanced Test and determine how to proceed with the winner send.
- Test Duration allows you to determine the amount of time in which data is being collected in order to determine the winning Segment.
- Test Send Time allows you to determine when the test Segments will send.
- Winner Criteria allows you to determine what criteria will be used for determining the winning Segment.
Test Send Time
Test Send Time is an additional test element for the Advanced Test. You are able to schedule the Test Segments to send at the same time or at various times depending on what you are testing. The send times consist of two options: Send at the Same Time and Send at Different Times.
If Send at the Same Time is selected, you will then be able to determine when the winner audience will send.
Send immediately Upon Completion of Testing Duration, will send the winner send out one the test duration completes.
Send at scheduled date and time allows you to schedule a specific date and time for the winner send.
If Send at Different Times is selected, you will choose the specific dates and times for each test segment. The time associated with the winning test segment will be used to send the winner email at the end of the test duration. This may result in the email sending a day after the test duration ends.
Winner Criteria
Winner Criteria allows you to select one of the options that will be used to determine the winner send from the test Segments reporting metrics.
| Field Name | Description |
|---|---|
| Top Combinations | Selects the Subject Line, Preheader, and From based on the highest open rates and selects the contect with the highest click rate for the winner's send. |
| Click Thru Rate | Selects the winner send based on the highest click rate. |
| Open Rate | Selects the Subject Line, Preheader, and From with the highest open rate. |
| Click to Open Rate | Selects the winner send based on the highest click-to-open rate. |
| Conversion Rate: Top Items Sold | Selects the winner based on the highest number of conversions. |
| Conversion Rate: Top Revenue Performing | Selects the winner based on the highest revenue. |
| Manual Selection | Allows you to select the winner after the test duration has completed. |
If Manual Selection is selected for the Winner Criteria, you will receive a System Notice with the results of the Advanced Test. After viewing the results, click the radio next to the Test Segment you wish to send to the rest of the selected audience.
A Winner Segment is required when using the Advanced Test functionality. If you need to test without a winner segment, please split your audience and send separate emails.
Once the Winner Criteria tab has been configured, click Next to review your Advanced Test configuration. Once reviewed, click Confirm to complete the activity.
From this point, you can add any other activities that may be needed to your Process Automation, then click Continue to move to Schedule the automation for sending. You can also run the Automation immediately if needed.
Configuring an Advanced Test for a Journey Automation
An Advanced Test activity for a Journey Automation varies from a Process Automation because of the nature of the journeys. The same concept is applied in that you can test various parts of an email send within your Journey Automation, but the audience and how the test duration process works differs because of the nature of Journey sends themselves. Journeys are built for a continuous drip of contacts entering and flowing down the various paths in the journey. Because of this, there is not specific audience selected and the test Segment process continues over a longer period of time.
Configuration Step 1
To begin configuring your Advanced Test, fill out the information in the initial tab. This tab consists of the basic information for the email send.
Email Send Configuration
| Field Name | Description |
|---|---|
| Name | The name given to the automation and Advanced Test. This is editable and can be changed as needed. |
| Description | Enter a description for this Advanced Test. |
| Campaign Tag | Select a Campaign Tag for the Advanced Test. |
| Track Opens | Enable Open Tracking for the Advanced Test. This is set to Yes by default. |
| Track Links | Enable Click Tracking for the Advanced Test. This is set to Yes by default. |
| Web Tracking | Enable Web Tracking for the Advanced Test. If Web Tracking is configured for your client, this option will be defaulted to Yes. |
| Web Analytics |
Enable Web Analytics for the Advanced Test. If Web Analytics is configured for your client, this option will be defaulted to Yes. |
| Send Classification | Select the Send Classification for the Advanced Test. |
Advanced Options
| Field Name | Description |
|---|---|
| Contact May be Sent This Activity Only Once |
If selected, any contacts who have passed through this automation previously will be suppressed. |
| Deduplicate Audience By Email Address |
Deduplicates the audience by email address ensuring one email is sent to each recipient. This option is enabled by default. |
| Use Recipient's Preferred Email Frequency When Known |
Allows you to send based on the recipient's email frequency if this feature is enabled on your account. |
| Email Frequency |
Allows you to suppress sending to contacts who have received an email in the previous X number of days. |
| Opt-out Override |
If yes, send messages regardless of a contact's opt-out status. The is configured to No by default. If the Send Classification is configured to Transactional, this will update to Yes. |
| Fail MX Rejections |
If yes, do not attempt to retry delivery of messages because of a MX rejection, e.g, 554 Transaction Failed. This is set to No by default. |
| Fail DNS Timeouts |
If yes, do not attempt to retry delivery of messages because of a DNS timeout, e.g, not able to reach host. This is set to No by default. |
| Never Retry Delivery |
If yes, the system will bypass the email retry process. |
Audience Selection
| Field Name | Description |
|---|---|
| Coupon Table | Select the iPost Coupon Table the content is using the Coupon iTL. |
| Suppression List(s) | Use this field if you would like to suppress contacts found on a list. |
| Suppression Data Table(s) |
Use this field if you would like to suppress contacts found in a sendable data tables. |
| MD5 Suppression List(s) |
Use this field if you would like to suppress records found on a MD5 Suppression List. |
Once you have completed the Configuration Step, click Next to move to the Segments Step.
Segments Step
The Segments Step for a Journey Automation is the same as in an Process Automation. You will define the email content that will be used in the Journey itself. You have the option of using the same email content for the test segments, or selecting multiple emails to be used in the Advanced Test.
To select your email content, click on the Email field and select an email from the available content. Once selected, the other fields will populate form the email properties page for the selected email.
The Duplicate Icon allows you to add more Test Segments to your Advanced Test. The Trash Can allows you to delete a Test Segment.
All of the fields for each Segment are configurable for testing. This allows you to not only test different content, but also any element of the email send itself.
Once you have completed the Segments Step, click Next to move to the Test Configuration step.
Test Configuration
The Test Configuration tab allows you to configure your testing period and determine a winner send.
Test Configuration allows you to determine the minimum number of contacts that need to pass through each Test Segment in order to start the Test Duration period. The default quantity of contact is 500 but this can be adjusted to fit your email marketing program.
Test Duration is the amount of time in which data will be collected in order to determine the Winner Segment. The Test Duration time will commence once the minimum number of contacts has passed through each of the test Segments.
Winner Criteria allows you to select what criteria will be used for determining the winning Segment.
| Winner Criteria | Description |
|---|---|
| Top Combinations | Selects the subject line, preheader, and from based on the highest open rates and selects the content withthe highest click rate for the winner’s send. |
| Click Thru Rate | Selects the winner send based on the highest click rate. |
| Open Rate | Selects the subject line, preheader, and from with the highest open rate for the winner send. |
| Click to Open Rate | Selects the winner send based on the highest click-to-open rate. |
| Conversion Rate - Top Items Sold | Selects the winner based on the highest number of conversions. |
| Conversion Rate - Top Revenue Performing | Selects the winner based on the highest revenue. |
| Manual Selection | Allows you to select the winner after the test duration has completed. |
If Manual Selection is selected for the Winner Criteria, you will receive a System Notice with the results of the Advanced Test. After viewing the results, click the radio next to the Test Segment you wish to send to the rest of the selected audience.
A winner segment is required with the Advanced Test functionality. If no winner segment is required for your test, please use a Random Split activity to run your test.
Once you have entered the information in the Test Configuration step, click Next which will allow you to review your Advanced Test activity configuration. Once reviewed, click Confirm to close the activity.
From here, you can add other activities to your Journey Flow and save the automation. Once the Journey is activated, contacts will start to flow through the Advanced Test activity and you can view the report metrics in the Journey Report for that automation.
Winner - Manual Selection
To select the winning Segment with the Manual Selection option, follow these steps.
- Hover over the Report option in the left side navigation
- Select Advanced Test Sends from the Report menu
- Locate the report for your Advanced Test send
- Click on the Name of the Advanced Test
- Review the report metrics, and select the radio button next to the winning segement
The winner send will then send based on the options selected in your Advanced Test activity.