Category Archives: SEM Stats

SEM Stats

Running a split campaign

How to Split Test Offers to Your House List (Without Customer Service
Are you convinced a particular offer would increase your ROI dramatically,
but management thinks it’s chancy? Check out this new Case Study on Pete’s
Coffee & Tea. Last December they tested three different email discount
offers – with surprising results.
Our favorite part: see data on how adding the offer to every page of the
site that email clickthoughs see on their path (not just their initial
landing page) helped improve results. Yes, includes creative samples:

“We’re a highly branded site. The people that visit us for the most part
already know us,” says Brian Platter, General Manager Home Delivery, Peet’s
Coffee & Tea. “They look forward to our emails.”

Because coffee isn’t a “commodity business” — that is, consumers mainly
purchase based on brand and taste preferences rather than price — Peet’s
doesn’t often do discounting promos. “We only do it when we’re trying to
encourage people to do something different.”

But when Peet’s wanted to encourage customers to give Peet’s products as
gifts last December, the team decided to run a free shipping promotion.

In the past, the team had offered free shipping on orders of $50 or more
which is slightly higher than the site’s average order value of $40-$45.
“Fifty dollars is conservative,” says Platter. “The ‘powers that be’ worry
about giving away something and not getting enough in return.” The
ecommerce team suspected that if they dropped the required purchase minimum
conversions would rise high enough to make up for the potential drop in
average order per cart.

Platter talked fast and although he agreed to stick with the safer $50
minimum for all regular site visitors, he won management’s approval to split
the house email list into a series of test cells each with a different
discount minimum for the December promo campaign.

However, this could cause a customer service nightmare. What if customers
got one offer via email and then a completely different one when they
clicked to the site? How could Peet’s run three different test discounts via
email and a single discount offer online without confusing people?

The team split the house list into three parts (link to sample creative

#1. The bulk of the list was sent a control offer featuring the conservative
$50 threshold.

#2. 20,000 names were sent a $30 minimum offer (the team had considered $25
but they tested $30 instead to see if they could get consumers to purchase a
minimum of three pounds of coffee).

#3. 20,000 names were sent a zero minimum offer — ie. free shipping no
matter how low the purchase was.

To keep the test as clean as possible, the subject line for all three email
creatives read the same:
Peet’s Gifts for All, Free Shipping for You

The creative featured a photo on the top left of a package of coffee and a
package of chocolate covered cherries. Beside that, on the right, was a box
that included the text, “Give the gift of Peet’s and Get Free Shipping.”
Running through the middle of the page was a red banner that said, depending
on the offer received: “Free Shipping on Orders of $50 or more.,” “Free
Shipping on Orders of $30 or more” or “Free Shipping on all online orders.”

Below that was a series of gift ideas.

Key — because these were campaigns to current customers who already knew
the site, the team knew there was a good chance email respondents would surf
a bit instead of buying directly from a single landing page. So every page
of each customer’s experience had to repeat the exact same offer the
customer had received in their email.

Anyone who received a particular offer via email saw that same offer on both
the home page and on all subsequent product pages they surfed. “We were able
to have the home page and coffee page ‘magically’ match the offer on the
email,” explains Platter. This worked by designating a certain area of the
site as a “content slot.” The content in that slot could be changed
depending upon the source of the visitor.

While developing this landing page system for the email campaign, the team
began to wonder if the fact of the continual reinforcement of the offer on
every page of a site would itself result in higher conversion rates. So they
decided just for fun, to split regular (non-email-driven) site traffic in
half for a week. All home page visitors saw the $50 shipping offer but of
clickthroughs, half were shown a page that included the offer again, while
the other half were shown a page that did not include the offer.

In the meantime, the first email campaign with the split test was sent on a
Monday. By Wednesday the team had enough response data to determine which
offer was the winner. They used that data to send the entire house file the
winning offer as a “reminder” on Thursday.

The winning promotion was the offer of free shipping with orders of $30 or
more. “I was surprised, because I was assuming that either the no-limit
would raise conversion so high that it would win, or that just keeping the
threshold high, which would keep the average order value high, would win. I
didn’t expect the middle,” says Platter.

–The control (the $50 threshold) converted at a rate of about 0.5%.

–The $30 threshold converted 14% better than the control, and the average
order value (AOV) of those who purchased at the $30 threshold was 94% of the
control’s AOV. “So the conversion lift more than outweighed the slight
reduction in AOV,” says Platter.

–The promotion with a zero dollar threshold had a conversion rate of 1%,
twice as high as the control, but the AOV was only 72% of the control.

Generally, Peet’s emails convert at a rate of between 0.2% and 1%. A
Father’s Day offer last year had a conversion rate of 0.5% on an offer of
free shipping with a $50 minimum purchase.

Platter was also surprised by the results from the split test for
non-email-traffic. Turns out visitors who saw the web offer repeated through
the site did *not* convert more than those who saw the offer only on the
home page. However, the former group’s AOV was 9.5% higher than the latter.
So, although folks didn’t convert from shoppers to buyers at a higher rate,
they did convert for a slightly higher dollar purchase.

“The key was how little effort it was to actually learn something and be
able to apply that learning on the fly,” says Platter. “We applied the
learning later that week for a nice surplus. And it took all of a couple
hours work.”

Useful links related to this story:
Creative samples from Peet’s

Offermatica, the testing and optimization company Peet’s uses:

Peet’s Coffee & Tea

Win a BlackBerry device from O2 with Yahoo!. Enter now.

Email reading behaviour

Great info from those guys at MarketingSherpa:

Anne’s Blog:

I’m going to go out on a limb here and call it a trend:

Super-short email copy for offers.

I think we all started out thinking that email copywriting was like writing a letter. And for a very few, very special brands (mainly based around an individual) that’s still the case. But based on reams of anecdotal evidence across multiple industries, I’ve gotta say — how short can you cut it?

Fact is people don’t spend more than a few seconds when they open and review email prior to making a clickthrough-or-delete decision. From what I can tell, that clickthrough decision sure isn’t being made by anyone reading
complete sentences, sentence after sentence, in order.

Our eyetracking studies show the eye skips around more than that. Flick, flick, flick.

Instead of plodding along reading word after word in order, the eye is seeking clues that will allow it to cut-to-the-chase quickly. People want to make that clickthrough-or-delete decision and get on with life.

Alain Tremblay, who’s been heading the email team at Bell Canada since before there was even an email team to speak of back in the mid-90s, told me his copywriters have changed the way they write messages.

“Before, we would have a descriptive before the link. An example, ‘You will find all sorts of good information about your service call answers here; and, remember if you want to get your messages dial *98.’ We found that was just too much information. They just wanted to know where to go and click. Now we just say ‘here is the info on message retrieval’ and the content is
the link.”

He continues, “It’s obvious the more stuff you put in the email the less people are going to read it. I sound like a broken record when I say this, but: keep it short, keep it short, keep it short.”

“Back in the old days we thought more than 300 characters sounded like a lot, and right now, we’re A LOT less than that.”

The idea behind Alain’s newer copy (which by the way gets significantly higher click rates), is copy as navigation.

That’s right, you’re not explaining and informing and detailing and convincing and half-way converting. You’re just getting the frigging click and letting the landing page take over from there with the more detailed

And now, of course, I’ve taken up plenty of words to tell you to just use a few. Ah, the little ironies of editorial life!

Anne Holland – Publisher Marketing Sherpa

Broadband usage rising sharply

According to Nielsen//NetRatings, the number of active US broadband users
from home increased by 28% last year, rising from 74.3 million in February
2005 to 95.5 million in February 2006.

Yahoo! Photos – NEW, now offering a quality print service from just 8p a photo

Best open rates on a Friday

Contrary to conventional wisdom, Fridays had the highest open rates for
email messages in 2005, according to a new report from ExactTarget, writes
Internet Retailer. The average open rate for Fridays was 39.6 percent, with
Thursday right behind at 39.0 percent. Sundays, however, had the highest
click-through rate: 6.9 percent. Open rates by day of week were as follows:
Friday, 39.6 percent; Thursday, 39 percent; Wednesday, 38.4 percent;
Tuesday, 38.4 percent; Monday, 37.9 percent; Sunday, 33.5 percent; Saturday,
31.5 percent.

Win a BlackBerry device from O2 with Yahoo!. Enter now.

Search marketing stats 2005

” SEM spending is forecast to increase an average of 41% in 2005. [SEMPO, 2004] ” About 10 million people in the UK used a search engine in January 2005, compared with about 13 million Germans and 6.7 million in France. [Nielsen//NetRatings, 2005] ” 84.8% of users found sites through search. [GVU, 10th User Survey, 2004, US data] – 85.7% searched with intent to buy.
” The four leading search engines powered 81% of UK internet searches in September 2005. When the UK and .com properties of Google, MSN Search, Yahoo! Search and Ask are combined, that figure reaches 94%. [Hitwise UK Search Report, October 2005]

figures from…