Don’t Cross the Border
June 12th, 2008 Chris Posted in E-mail | 1 Comment »
Every once and a while (not as often as I should), I do a little split testing of our solicitation (i.e. sales) emails. I make two versions and send one to respective halves of our database. Because of the marketing tool we use, we can track open and click rates. Ideally, this shows us what works better.
I’m always amazed at how little tweaks can make significant differences in open and click rates. One tweak that we’ve had a debate on was the use of borders in email communication. That is, whether to send emails with free-flow text (text-wrap taken care of by the email client) versus setting a specific width to the text.
The former tends to look more personal – something that we’d do if we were writing one-to-one. The later looks more professional but is more obviously a mass mailing. Colleen was leaning to the former as she is fantastic at adding personality to communication. I was leaning to the border version as I am more anal about look and feel. So we put it to the test!
We drafted two versions of a sales email: one had free flow HTML text and one had a nice border to constrain the width to about 650 pixels. The text itself, including all formatting (line spacing, para spacing, bolds, italic, etc…) was identical. We sent each one to 3500+ recipients and waited for the results.
The difference was dramatic. Click through rates – that is, the number of people that opened and clicked through to look at our web page where our product could be ordered – varyied significantly with one being 240% of the other.
And the winner? Borders rule. Keeping our email text within a border resulted in 140% more clicks to our web page than without.
I spent the next two hours redoing all our campaign emails to make sure they had borders. I also savoured one of the very few times where I’ve been right versus Colleen.
I typically reserve the “Geek” piece for technology issues but I have to expand on a math point. Every since my former CFO Dave Wagner corrected me, I’ve become a bit of a zealot on this point. It’s about calculating how much “more” something is versus something else.
In the click-through-rate calculations above, I stated that the rates with borders were 240% of those emails without. In other words, the number of click-throughs with borders, divided by the number of click-throughs without borders was equal to 2.4, or 240%.
This does not mean that the border-equipped emails performed 240% better or got 240% more clicks – it means that they were 240% of the non-border email clicks.
For a calculation of how much better or how many more there were, you need to take off 100%. So for example, if you have 100 apples and 200 oranges, you have 200% (200/100) as many oranges as you do have apples. But you only have 100% (200% – 100%) more oranges than apples.
Don’t feel bad if you’ve been doing this, it’s a common gotcha. Dave mentioned that even some CEOs have been known to do it…
C.
June 12th, 2008 at 8:57 pm
Chris, you hit on a great point which is that even subtle changes on an email or landing page can have a huge impact on response. I just attended a seminar by Interwoven which makes very cool software for multivariable A/B testing that is used on large consumer sites like American Airlines. They have been able to drive big increases in revenue simply by changing the color of the buy button (green to red) and changing its place on the page. While smaller companies like ours can’t afford sophisticated testing software we can make big improvements like you achieved by simple trial and error. For other tips on marketing emails you should check out the MarketingSherpa report “Dirty Dozen: Email Newsletter Mistakes Nearly Everyone Makes.”