Our Guide to A/B Testing Email Drip Campaigns

A/B testing email drip campaigns means changing one variable at a time (subject line, send time, sender name, CTA, length) on a small sample of your list, measuring open rate, click-through, and reply rate, then rolling the winner to the rest. Test before you invest. We do it on every Overloop sequence.

Relying on gut feeling is good when you're leading a one-on-one interaction. If you know how to read people, it's an amazing thing. On the other hand, when you're sending emails to lots of people you've never met face to face or even talked to, your instinct may not be your best tool.

Speaking about tools, you don't need any to run A/B tests, so you've got no excuse here! I know it and everybody at Overloop knows it.

Bring in the analytics!

Know your KPIs!

There are traditionally three ways to measure the success of email campaign: [CNIL]

1. Open rate: Do prospects open my emails?
   The math: Total emails opened/Total emails sent * 100
2. Click-through rate: Do prospects click my links?
   The math: Total links clicked/Total emails opened * 100

or Total links clicked/Total emails opened * 100

3. Reply rate: Do prospects reply to my emails?
The math: Total emails answered/Total emails sent * 100

or Total emails answered/Total emails opened * 100

For the last two metrics, make sure you always use the same denominator.

A/B testing?

A/B testing is a method implemented to test and improve the efficiency of a process by modifying one of its features and determine what works best. [HBR]

The golden rule of A/B testing: only tweak ONE element at a time. It may seem obvious but if you go ahead and modify more than one feature of -in this case- your email, it'll be impossible to identify what caused the increase (or the drop) in opens, click-throughs...

See what works and then make a choice!

Test and then invest

You're excited, I know you are. But before launching a large scale campaign, you might want to test it on a sample of your target in order to maximize its effect and optimize your budget.

Let's look at an example.

Say you need to launch a drip campaign on 1000 prospects and you want to A/B test your subject line and your from field in order to maximize your open-rate.

Pick 100 of the prospects on your list and split them in two groups. Let's call them group A and group B. Split these groups in two again. Let's call them Aa and Ab, and Ba et Bb.

Now pick the two subject lines and the two from names you wish to test. Aa et Ab will receive the same from name but with a different subject line while Ba and Bb will receive the same subject line but with a different from name.

You can run both A/B campaigns at the same time on 10% of your target and learn a great deal in just a few days.

But those are not the only variables. Let's go deeper, shall we?

Goals and variables

Each KPI is linked to a certain number or settings and variables that you can influence.

1. Opens

Your first goal is, naturally, to have your email opened. 3 main criteria stand out when it comes to get your email opened:

The subject line

According to Convince&Convert, 35% of recipients open emails based on the subject line alone. Also, 69% of them report email as spam solely based on the subject line.

This stresses out the importance of a good subject line. Therefore, make sure to spend as much time as necessary to craft it.

Here's some advice on the subject!

The from field

A nice sender name influences greatly the dynamic of your relationship with your prospect.

Try variables such as: {your name}, {your name} at {your company}, {your company} Team

Delivery time

As already explained, delivery time is critical. Narrow down the most likely times for your prospect to be receptive to your emails and pick the best two options so you can test them.

2. Click-through

Congratulations, your recipient opened your email!

Although we don't recommend using links in sales email, especially at early stages, here's some advice in case you absolutely must redirect to a website.

There are 2 main criteria related to the click-through of a link within an email:

Trust in the link

Links are naturally not trusted when coming from unknown senders.

What you can do:

You should avoid URL shorteners as much a possible. It's understandable that you'd want to track results with bit.ly or other similar services, but if you're prospecting, shortened URL's are highly unrecommended. Use UTM's instead.

Only use a reputable domain, with something else than your landing page, so your prospect can research the website (e.g. through Google) and see for themselves that it's actually trustworthy.

The content of the email

If your prospect isn't interested at all in what you have to offer, there isn't much you can do. For all other instances, here's what you should be working on:

Keep it short

Time is scarce and emails are many. If your prospect gets bored before getting to the link, you're screwed. So keep it short and to the point.

Remove distractions

If your goal is to have you prospect click that link, make sure their eye won't be attracted by something else, like images, other links of... worst of all, multiple calls to action.

Include a P.S.

This is powerful, but to be used wisely. Make it super personal and not salesy, and just drop that PS that'll catch all their attention!

Hi Jeremy,

Are you struggling with sales automation?

My name is Forster and I work at Overloop, where we help businesses like [Company X], [Company Y] and [Company Z] automate their sales processes.

I'd like to know how you handle this at [Your Company], maybe I can give you a few tips!

Cheers,

Forster

P.S.: Check out this super actionable content I put together!

Mobile responsive

40% of all emails get opened on mobile devices first, so you need to make sure that your prospect will be able to actually read it. Either by using a mobile responsive format or -better yet- just use plain text, which is always better for one-on-one conversation. By the way, this is the reason why Overloop doesn't support or allow in-mail html.

3. Replies

Getting someone to reply to your email is tough, especially when it comes to cold emailing. Therefore, here are a few variables you can tweak to successfully get replies.

Adjust the length

Depending on your goal and on the content of your email, you may have to adjust its length. Do your prospects respond better to long form emails or to short and concise messages? Go with what you know and see what works.

Provide a reason

You have probably already heard of that crazy study that proves that giving even a nonsensical reason brings more results than giving no reason at all. If not, here it is in short: the researcher would spot someone waiting to use the copy machine and casually walk up to them with the intention of cutting them in line. He tried 3 versions of the request:

1. Version 1 (request only): "Excuse me, I have 5 pages. May I use the Xerox machine?" => Success: 60%
2. Version 2 (request with a real reason): "Excuse me, I have 5 pages. May I use the xerox machine, because I'm in a rush?" => Success: 94%
3. Version 3 (request with a fake reason): "Excuse me, I have 5 pages. May I use the xerox machine, because I have to make copies?" => Success: 93%

Version 3 clearly doesn't make much sense. He just said... because, and it worked. Maybe this can make the difference in your email?

Social proof

Dropping a few names is always a good way to prove you're a serious partner. How many names though? And which ones?

If you work with international prospects, make sure to mention customers from their country. If you work with prospects from various industries, make sure to mention customers from their industry.

Include a P.S.

Again? Well, the P.S. being a good spot to place your call to action, you might as well take advantage of it to ask for a reply!

Hi Jeremy,

Are you struggling with sales automation?

My name is Forster and I work at Overloop, where we help businesses like [Company X], [Company Y] and [Company Z] automate their sales processes.

I'd like to know how you handle this at [Your Company], maybe I can give you a few tips!

Cheers,

Forster

P.S.: Would a quick call on Tuesday morning work for you?

Your turn now, share this piece and get to work!