A/B Testing In ConvertKit

Engage Subscribers With Effective Emails

Knowing your subscribers, and their needs, is a key ingredient to effective email marketing, and your overall success as a blogger. We have just deployed a newer, more robust way of determining what your subscribers are positively responding to, giving you a better in-road to solving their pain points. Winning.

What Is A/B Testing?

A/B testing is a way to compare two versions of a single variable, (in this case, the subject line of an email), by testing a subscriber’s response to variable A against variable B. One variable (subject line) will come out on top, with the most clicks.

Here’s a more detailed look of how we do it at ConvertKit: a percentage of subscribers are randomly selected, and then assigned to one of two subjects. We then determine a winner, based on the open rate of the subject. The remaining recipients (not including that first percentage), will be sent the ‘winning’ variant, (subject line), yielding the highest open rate overall.

Think of it as Group A, and Group B. Each group receives a version, or variant, of the subject line, and the subject line with the most clicks is sent to the remaining subscribers.

This testing period occurs over a four hour period of time, so you can zero in on better  open rates, in less than a day.

Who Is A/B Testing For? 

It is best to only run A/B testing for a recipient count of 1k or more. A test run on a smaller number will not give actionable data, as the percentages and numbers will not be statistically significant. Here is an example of why this is important: 

If you send your broadcast to 100 recipients, then each variation would go out to 15 people. With a less-than 50% open rate, only 7 recipients would likely open. This means that the winning subject line would be determined by the whims of a single person. One individual's decision of whether or not to open an email will not provide the best results. 

Turn On A/B Testing In Your ConvertKit Account

You can find the A/B Testing option in your account, by clicking the Broadcast tab, and then New Broadcast. You’ll notice the A/B option to the right of your subject line:

No more wondering what your subscribers are looking for, because now you can know exactly what they need. (Because they’ll tell you!) Whether you’re testing out new content for your next sequence, or you would simply like to know more about what makes your subscribers click, this tool will go to work for you.

Have you joined our Slack Community yet? Jump in today and don't miss out on live updates!

Enjoy this article? You may also like How to write more effective email content

Still need help? Contact Us Contact Us