Knowledgebase

14.5-Stats of Split Tests

Split Test Stats

Split test helps to evaluate the best performing campaign by setting up the criteria. Once you create the split test campaign, you’ll be provided with the option to setup the kind of test to run. See create split test campaign for more information on setting up the split test campaigns. If you already have setup some split tests, view their stats by clicking “Split Test Stats” under “Statistics”.

Figure#14.5.1: Split Test Stats

Table#14.5.1: Statistical Details of Split Test Campaigns

Sr.	Serial Number of records in the table.
Schedule Label	Campaign schedule label is something which helps to identify the scheduled campaign, across the queue of the scheduled emails. This column also appears on the page “View Scheduled Campaigns”.
Campaign Name	The name refers to the split test campaign name. Name is the first mandatory field while creating the new split test.
List Name	Name of the subscriber list(s) you have scheduled the particular campaign for.
Schedule Time	Don’t get confused, it isn’t the time when the split test was first created or scheduled. It is the time on which the split test was scheduled to start the sending process.
Scheduled Status	The column shows the total number of subscribers which the split test is scheduled for. You will be able to see the message “Total Subscribers in List(s) on mouse hover.
Sent	Shows the number of subscribers which the split test campaign is being successfully sent to. So the schedule status column shows the total number of subscribers which the campaign is scheduled for, and this column shows the number of subscribers this split test campaign is being sent to.
Bounced/Unsubscribed	Number of bounced back emails and number of subscribers who opted to unsubscribe the mailing list, in result of receiving the email message.
Opened	The column displays the results of both, total opens and unique opens. The person who opens the campaign is known as opener, and the action is marked as open within the application. Unique opens point out the action of the subscriber who first time opens the split test campaign, and does not repeat the action. If the subscriber reopens the email by repeating the action, this repeated action will not be considered as unique open.
Clicked	Same goes for unique and total clicks as described above for the unique opens and total opens.

Actions

The action column provides the ability to go advance and view the detailed summary of statistics, perform particular actions such as filtration of the records and exporting the records to a file. After you click “View Detailed Stats” under the “Actions” column, you will be taken to the next page to view the detailed stats and perform related tasks with ease.Figure#14.5.2 shows how the detailed page looks like and what information it gives.

Figure#14.5.2: View Detailed Stats

a) Number of records per page

To select the number of records you want to view per page, the page puts a dropdown menu on view towards the top of the page. The dropdown menu starts from 10 records to display, and ends with all records on one page. The following figure indicates the position of the dropdown on the page.

Figure #14.5.3: Number of Records per Page

Search Split Test

The search box helps to search particular campaign from the list of sent. If you have sent a good number of split tests, this helps you to search and view result of particular test from the long list. Cut the process short, just type some unique identifiable information into the search box like, schedule label or name to search the split test. The search function works as it do in earlier described statistics pages like "Email Campaign Stats" , "Auto Responder Stats" or "Trigger Stats". You put a query forward, and application returns with relevant records according to the search query.

Detailed Stats Page

a) Campaign Summary

The first tab offers you the detailed summary of the campaign, which includes the information such as campaign type, name of selected campaign(s), selected subscriber list(s) to send this campaign to, status of campaign and brief of other related information.

b) Opens

Next to the campaign summary, a table opens along with the details of the subscribers who opened the split test. The table contains the following information of each opener.

Table#14.5.2: Details of Opens

Sr.	Serial Number of records in the table.
Email	Email address of the subscriber who opens the split test based campaigns campaign.
Opened Time	The exact time when the subscriber opens the email, he/she received.
Opener’s IP	The IP using which the email has been opened by the subscriber.
City	City in which the email is being opened by the recipient/subscriber.
Region	Regional information where the subscriber was located when opening the email.
Country	Country which the IP address of the opener belongs to.

Copy Opener Details

Use the copying options to copy the data in the columns of the table appears below. You can copy the table by using the “Copy” button and can paste it manually in a file, or can copy and save the complete table along with the columns to .CSV file, Excel or PDF format. Further you can get a print out of the complete table by clicking “Print”. See the following figure

Figure#14.5.4: Copy the Table

Filter Opens

Filter the records based on the opener’s country, select a country from the dropdown to view the details of the opens from the selected country. You can further copy the filtered records using the copying options as discussed above. . You can copy the table of filtered records by using the “Copy” button and can paste it manually in a file, or can copy and save the filtered table along with the columns to .CSV file, Excel or PDF format

a) Clicks

Table#14.5.3: Details of Clicks

Sr.	Sr is the abridged form of serial number and in almost all the view page of MUMARA, Sr would be the first column to start the view table with.
Email	Email address of the subscriber who clicks the link in the email he/she receives.
Click Time	The exact time when the subscriber clicked the link.
Clicker’s IP	The IP using which the link has been clicked by the subscriber.
Link	The link that was being clicked.
City	City in which the link is being clicked by the subscriber/ recipient.
Region	Regional information where the subscriber was located when clicking the link.
Country	Country which the IP address of the clicker belongs to.

Copy Clicker Details

See Figure# 14.5.4 Above

Filter Clicks

Filter the records based on the clicker’s country, select a country from the dropdown to view the details of the clicks from the selected country. You can further copy the filtered records using the copying options as discussed above. You can copy the table of filtered records by using the “Copy” button and can paste it manually in a file, or can copy and save the filtered table along with the columns to .CSV file, Excel or PDF format.

b) Bounced

Next to the clicks, the bounced tab opens up the table, which provides even the minor details of the bounced back emails. The table provides the following information.

Table#14.5.4: Details of Bounced Emails

Sr.	Serial Number of records in the table.
Email	The email which bounced back in result of sending the split test.
Bounced Time	The exact time when the system received this email as bounce.
Bounce type	The type of the bounce between hard and soft. Hard bounce shows the permanent delivery error, while the soft bounce is momentary delivery issue.
Bounce Code	Whenever an email is bounced, it bounces back with a reason, why these delivery problems occur. Each bounce reason has its own error code, and most of the general delivery issues and their codes are categorized as per the industry standards. The column shows the error/bounce code associated with the bounce reason.
Bounce Reason	As described in the above field for “Bounce Code”, there are reasons behind each email which bounced back. The column shows that reason.

Advance Bounce Stats (PowerMTA Bounce Processing)

Integration of the bounce account with the VMTA based application PowerMTA, helps to go forward to find out the advance details of the delivery issues. You will be able to view the advance details in the columns after “Bounce Reason”, only if the bounce account has been integrated with PowerMTA for processing and collection of the bounce stats. The following table describes what advance stats of the bounced email, the PMTA bounce processing can collect.

Extension- Table#14.5.4: Details of Bounced Emails (PowerMTA)

Bounce Details	Along with the bounce reason and the error code, the bounce which processed using the PMTA presents a delivery issue in detail. The column carries the appropriate detailed message, why the delivery issue occurred and why the email the sending of the email is being rejected? Is it due to the spam content, recipient address rejected it, rejected because the network is on block list or some temporary problem occurred? This detailed information further helps to troubleshoot and fix the delivery issues and improve the results.
DNS MTA	It refers to the MX host/ mail exchange host of the recipient, responsible of processing the transmission details received from the sender MTA within the server application infrastructure.
DLV Source
DNS Destination
VMTA	Virtual MTA or simply VMTA is the technology which PowerMTA uses to extend your control on your email stream, and enables to manage the IP address separately for even a single message which is being sent. This column shows information about the VMTA of the bounced back email.
VMTA Pool	The column refers to the pool of the VMTA using which the email is being sent, and eventually bounced back. As described above, to ensure standardized bounce categorization, details of non-delivery report along with the classified information of sending infrastructure is key.

Copy Bounced Details

See Figure#14.5.4 Above

Filter Bounced

Filter by Error Code

Error codes and Bounced reasons have already been elaborated while discussing “Setup Bounce Reasons”. The first dropdown offers the ability to filter the records with their bounce codes, an example of this is selecting 5.0.0 will filter out the email addresses which does not exist.

Filter by Bounced Reason

The second option is to filter the records with the bounce reasons. There are set industry standards for bounce reasons and their error codes. However, MUMARA allows setting up manual bounce reasons as well; reach “Setup Bounce Reasons” for more details. Select one of the bounce reasons from the dropdown to filter the records accordingly. For example, bad destination mailbox address (5.1.1) will show only the filtered records.

Filter by Type (Hard or Soft)

Keep the dropdown option at “All Bounces” if you don’t want to filter records based on bounce type. Select “Soft” to separate out the soft bounces only, and select “Hard” to filter and show the hard bounces of the split test based campaign.

Mark Filtered

Mark the filtered bounce records as soft by clicking the button “Mark as Soft”, and mark the filtered records as hard by clicking the button “Mark as Soft”. Permanent delivery error is called “Hard Bounce” and the momentary delivery issue is considered as the “Soft Bounce”. Application stops sending further campaigns to the emails marked as hard bounce within the system. But at times you may need to mark some hard bounce as “Soft” for further emailing; it is where these two buttons help.

Copy the Filtered Records

You can further copy the filtered records using the copying options as discussed earlier. You can copy the table of filtered records by using the “Copy” button and can paste it manually in a file, or can copy and save the filtered table along with the columns to .CSV file, Excel or PDF format.

Refer to Figure#14.5.4 Above

c) Unsubscribed

Numbers of subscribed contacts from the selected list(s) who request to stop receiving further emails are known as unsubscribed. When you send emails in bulk, it is likely to have less personal relation with each contact you have sent the split test to, therefore knowing the unsubscribed contacts is important and removing them from the list to stop further emailing to them is recommended. The table that follows provides required information about the contacts who unsubscribed.

Table#14.5.5: View Details of Unsubscribed Contacts

Sr.	Serial Number of records in the table.
Email	Email of the person who unsubscribed from the mailing list, recommended sending practice is to keep your list updated against the unsubscribed request.
Unsubscribe Time	The time when the system received the unsubscribe request from the subscriber.

Copy Unsubscribed

Figure#14.5.4 Above

d) Complaints (Abuse/Spam)

The table which follows contains the information of the subscriber who marked the received email as spam. The reasons of spam can be several, but to fix these reasons and improve deliverability, the key is obtaining and keeping the record of the spam/abuse complaints for further actions.

Table#14.5.6: Complainant Details

Sr.	Serial Number of records in the table.
Email	Email of the person who marked/hit the email as spam, as the spam directly effects the reputation of sending IP, and email stream. Therefore it is recommended to process the spam/abuse complaints as soon as you receive them, always implement Feedback loops.
Complain Time	This refers to the time when the system received the complaint.

Copy Complaint Details

Figure#14.5.4 Above

e) Logs

Beside the general ADL campaign information, the logs table contains some of the stats as follows.

Table#14.5.7: Logs Table

Sr.	Serial Number of records in the table.
Email	This refers to “From” email, the email using which the campaign is being sent.
Campaign Name	Name of the split test campaign which is being sent, name is the first mandatory field to create a new split test using “Create Split Test” page.
SMTP Label	MUMARA allows setting up as many SMTP accounts as you want. The column refers to the name of the SMTP account which is being used to send this split test campaign.
Sending Time	The time it took to complete the sending of the split test to the subscribers of the selected list(s)
Bounced	Numbers of emails bounced back in result of the split test.
Unsubscribed	Numbers of contacts/subscribers who requested to stop sending further emails by opting to unsubscribe from the mailing list.
Status	Current status of the campaign, is it in process of sending, completed the course of sending or paused?
DNS MTA	It refers to the MX host/ mail exchange host of the recipient, responsible of processing the transmission details received from the sender MTA within the server application infrastructure.
DLV Source
DNS Destination
VMTA	Virtual MTA or simply VMTA is the technology which PowerMTA uses to extend your control on your email stream, and enables to manage the IP address separately for even a single message which is being sent. This column shows information about the VMTA of the bounced back email.
VMTA Pool	The column refers to the pool of the VMTA using which the auto split test campaign is being sent.
Details