7 effective A / B tests with welcome letters

The growing popularity of email newsletters is a consequence of their effectiveness. To use this tool at 100%, you need to conduct serious training. For example, to figure out how to greet a potential customer. It turns out to be not so simple. Welcome letters to clients are the key to the success of all relationships. Text, design, dispatch time and structure affect whether customers become active, whether they buy more. Earlier, we examined tips on time, frequency, urgency, personalization, calls for targeted action, in this article we want to talk about how you can (and should!) Test greeting letters. The experience is shared by Peter Tanham, who has extensive practical testing experience that has yielded results.

When it comes to conversion optimization, we usually begin to optimize the top of the sales funnel and work with traffic - we change the time of posting on social networks, calls to action, headers and pictures on landings.
However, after we achieve maximum CRO (conversion rate optimization) in these areas, we should not stop there and work with the lower parts of the sales funnel.

Why?
People who took the first step and declared themselves your potential customers by signing up for the newsletter or registering for a free trial are very valuable to your business. And a series of timely and well-written letters can be a really powerful tool with which you will welcome new users and turn them into your regular customers.

A / B testing can help you optimize welcome emails - however, this involves more extensive research than just checking the line with the subject line.

Here are 7 effective A / B tests you should do to optimize your welcome emails.

1. Testing the time and frequency of sending

A new user has just signed up for a free 14-day trial. Now you have 14 days to use your service and e-mails to interest him and turn him into a paying customer.
• How many letters do you send?
• One per day? A bit like spam, right?
• One per week? Users will probably forget who you are after the second letter.
• Hmm, why not send five letters on the first day? Or three in the first three days, and then not a week?

As soon as you start asking such questions, you will understand that these are just assumptions about the optimal frequency of e-mails.

And very often your assumptions that you send letters too often or, conversely, too rarely, are quite far from the truth. This is where the good old A / B tests are good.

How to test sending frequency?

Thought: “Why don't we send 5 letters on the first day?” - this is the same as throwing a little spaghetti on the wall and see if they stick or not. Of course, you may notice a slight increase in conversion, but this is unlikely to teach you anything.

A much more effective approach is to start with a hypothesis!

Knowledge:users who register for our free trial want to solve a problem that is relevant to them. It is so relevant and urgent that it is item # 1 on their to-do list for this week. They will be able to evaluate our service and competitors' product over the next 48 hours, and for the first time they will try to work with the service in the next 72 hours.

Hypothesis: Active sending of welcome messages in the first 72 hours will be relevant and timely for users - this will increase conversion.

The hypothesis is knowledge-based and easy to test. Perfectly.

This is better than the option with spaghetti, as this approach will help you determine the ideal time to send an email, and it will also allow you to better know users. As an added bonus: the test results will put forward something new to learn, which can help in creating new hypotheses and conducting further tests and experiments.

Do not try to guess how many welcome messages should be in your conversation. Do the testing and let the subscribers solve it.

2. Testing the urgency element
Testing the time for sending messages is the first big step that you need to go through, however, after that you need to start testing the content that is contained in your letters.

Instead, let's look at the tactics that many companies use in their emails to increase conversion: adding an urgency element.

Mentioning time limits in your welcome letters may make users rush and prompt them to act right now.

How does Doggyloot use the effect of urgency?

See what happened when Doggyloot tested the effect of urgency in their emails, which were sent after the user left the cart abandoned - that is, interrupted the process of making a purchase. In the screenshot below you can see: the letter says that "the goods that you added to the basket are already almost all sold."



When they aroused a sense of urgency among their clients and used a clear call to action, the letters significantly increased Doggyloot's monthly income.

Every time I want to optimize a series of automatic emails, I often experiment with urgency. As a rule, in the early stages this leads to a significant increase in conversion.

How to test urgency?

If you are not sure where to start, then you can test the urgency element of your letters by sending e-mails with a reminder for different time periods.



Test the letter with the text "Your trial period is almost over" with an interval of 48, 24 and 12 hours.

A few points to know before you start:

• Always include a control group that does not receive the letter you are testing. It will let you know that any email reminder - no matter the time - is better than none. It will also allow you to isolate and measure the effect of each e-mail on the conversion rate.
• The number of test variations should depend on the number of new registrations. If few users register, start with two variations and use a statistical significance calculator. I usually conduct tests for two to three weeks, while I calculate the number of variations so that each of them gives me about 100 conversions (registrations).

3. Personalization testing

Doggyloot achieved great success through email marketing, partly because they conducted tests aimed at personalization and segmentation, which were based on one parameter that was very important for dog owners: the size of the toys (small, medium and large).
This is very important, because Chihuahua owners are unlikely to buy a St. Bernard toy for their pet, because it will be more than the dog itself!



These three chewy duck legs are unlikely to be suitable for a chihuahua.
Segmentation of emails based on this data yielded quite impressive results. It was found that emails that targeted large dog owners received a CTR of 410% above average.



These data turned out to be so useful that now the only question they ask a new user on the website’s home page (in addition to the email) is “What size is your dog?”

On the form on the left is written: "What size is your dog?"
Knowing and correctly using the data on the recipient of letters, you can significantly improve the conversion rate using e-mails.

How to test personalization?

What other personal data can you experiment with when creating your welcome email?
• You can start with details that are easy to learn — for example, name, date of birth and company name.
• You can also experiment with information that is not so easy to obtain — the location, type of device used, or level of use of your product.
• If you want to try something more advanced, try using personalization based on an advertising campaign that will attract users, the keyword they are looking for, or the landing page they registered with.

All these data can be very fruitfully tested. The main thing is to start. And carry out experiments.

Want to make welcome letters a little more welcome? Test personalization and segmentation!

4. Test your feedback request

In early 2012, I read an article by Derek Halpern, which allowed me to write the most effective letter of all that I have ever written.
In this article, Derek recommends attracting new users (or blog subscribers) using an automatic e-mail containing one simple question:

“What is your problem?”

Since his post was published, I have used this technique over and over again - and I realized that it is really effective.

How does Groove ask for feedback?

Customers who have already completed a targeted action usually have valuable information that they can share with you. In the case of my SaaS company, questions like: “What is your biggest problem as a marketer that you are facing right now?” Were justified. Answers play a huge role in shaping our product plan.

Groove, a SaaS customer support company, began asking a similar question in their confirmation email to those who signed up for the trial:

Groove asks new users, “Why did you sign up for Groove?”



This automatic writing has proven its worth:
"The response rate for this letter is 41%, and it allowed us to better understand our business than any of the emails we have ever sent."

Here are some of the responses they received: The





email replies that Groove received helped them form the messages that they use on their landing pages and home page.

They used these answers to refine their marketing messages, which ultimately doubled the conversion rate on their homepage.

They also have an interesting note about the timing of the response to the message they tested:

“Interestingly, we found that the answers to the question were significantly different for those customers who answered it immediately after registration, and those who used Groove for a week or longer. After a few days, the answers begin to focus more on specific application features.
For this reason, we considered it important to ask the question right away so that the decision is still fresh in the minds of the user. ”

Asking potential customers for feedback is great, but you will learn more about them if you first test when and what you ask.

Customer feedback is very important, however, first think and test what and when you ask.

5.Test unexpected benefits

In the past, when I worked in the field of marketing consulting, I had many clients among musicians and employees of record companies. Some artists had a large list of contacts for whom they made a monthly newsletter.

As a rule, everything went smoothly, but there was one problem: regardless of whether you were a fan of this musician for 10 days or 10 years, the same letter was sent to you. And, since most of the users have long been on the mailing list, the content, as a rule, was focused on old fans, which scared away new subscribers.
When we decided that new subscribers were important too and started sending them a welcome letter, the results were simply amazing.

Instead of receiving a monthly flow of newsletters, each new fan received a series of emails that introduced him to the artist, and ultimately formed an emotional connection with him. After several weeks, new subscribers had the opportunity to support the artist by buying a new disc or merch.



With one Irish performer, we started by testing various motivating factors that we could implement before asking for support. If we could send subscribers one letter 24-48 hours before asking to buy a disk or something else, what type of e-mail would be most effective and motivate users to perform a targeted action?

Something like social proof, a demonstration of how thousands of other fans are enjoying this music? Or a reminder that people who buy the disc are very supportive of the artist?
It turned out that the most effective motivator (which subsequently doubled the purchase rate) was an unexpected gift.

About four weeks after subscribing to the newsletter, the fan received an unexpected letter in which there were three free acoustic songs specially for subscribers and a personal letter from the artist.



It was a real gift - it did not depend on any user actions, it seems to me that this led to its success.
It was just nice.

And the people who received this gift were twice as likely to buy an album or T-shirt when asked about it two days later.
This is true for any business.

I saw how free e-books, useful reports were sent to subscribers, or simply “free” access to the old webinar was simply offered. Why this works can be explained from the point of view of psychology (gifts motivate to repay the same). However, it is very important to know how unexpected joy encourages a customer to buy something from you with a much greater probability.

6. Test the design of the letter.

We already know how important the subject of the e-mail is and the content contained there. However, what about the design of the letter?
Should welcome emails look like a catchy newsletter? Or should you prefer a less vivid design using corporate colors, a logo and several product images? What about a regular text email without branding and design per se?



Should the greeting letter be catchy and with many images, more relaxed and include several elements of your brand, or should it consist of text and only text? The only way to find out is to test.

Testing the design of the letter

The Obama team, which organized the 2012 campaign, found that the e-mails that you can use to make the most money can be done in an individual style in plain text.

Here is what Amelia Shawolter, director of analytics in the Obama team said about it: “It seems to me that this created the impression that these letters were written by real people, that they were not“ focused ”to death. These emails are more like an impromptu. ”

On the other hand, Wishpond recently wrote about testing, the results of which showed the opposite.
They studied the study, according to which "67% of users believe that clear detailed images are much more important than product information and customer ratings," so they decided to test this in their email.

As a result, they found that emails with CTR images were 60% higher than emails consisting only of text.
Keep in mind, I did not cite these two opposite cases as an example to confuse you!
I used them to illustrate that you cannot fully trust your intuition, industry standards, or what your competitors are doing. You can only conduct testing.

What emails do you send to customers? Maybe it's time to test something new ?!

7. Testing the opportunity to return the client

What to do when your free trial period of your potential client comes to an end? What if you sent him via timely, personalized and valuable welcome messages, but he still hasn’t completed the targeted action and has not become your client?
Just because he hasn’t done it yet does not mean that he is not interested in what you offer. Returning the user back is quite possible.

How Squarespace returns its users

Let's look at two letters that Squarespace sends to users who have not switched from trial to paid.

This is the first e-mail that is sent two days after the end of the free trial period, offers seven additional days to try again to attract the user - a good combination of urgency and an attempt to return the client.



The subject of the second letter, which comes a few days later, is the question “What could we improve?” Although there is a request for feedback in this letter, it also gives the user the opportunity to extend the trial period and return.



How Groove is returning its users

Groove is also experimenting with emails aimed at returning users, as part of tailoring potential customers. If the user has not yet completed the target action, they send emails 7, 21 and 90 days after the end of the free trial.

It seems that 90 days is too long (will users even remember the existence of Groove?) - however, as testing has shown, these emails are steadily increasing conversion by 2%.
Not a large-scale increase, but only 2%. But this is better than the 0% that you would get if you let your customers go forever.

At what point do you finally abandon your potential customers? Maybe it's time to test and return users via e-mail?

Is there a hypothesis? We will test!

You don’t need to completely copy someone’s successful tactics, it’s better to catch inspiration and conduct extensive testing of your greeting messages.

First you need to find out something about your customers - through support letters or simply by asking users what causes them difficulties. This understanding will help you hypothesize what needs to be added to your emails so that they motivate users, break down unnecessary barriers and encourage potential customers to act quickly.

If you combine the landing page, on the effectiveness of which you have worked for a long time, with the same competently created series of welcome letters, you will find that the conversion will increase significantly. However, the only way to make your welcome emails truly effective is to start testing today. You can use Mailchimp, AWeber or SparkPage to create your first series of welcome emails - each of these services has free trial versions.

So do it!