« March 2007 | Main | May 2007 »

How Experimentation Boosted Restoration Hardware's ROI 267%

We're in the third month of what we have been calling "The Year of Optimization." (We're calling it that because it has become apparent to us at Offermatica that the companies that don't focus on optimizing their sites, in order to give their customers exactly what they are looking for online, will die... In the same way, the companies that do consider optimization an ongoing part of their marketing mix will thrive and speed past their competitors.)

As I wrote recently, the first and easiest phase of optimization is simple experimentation, and almost every company already does it to some extent. Any marketer who has tweaked copy, changed images, or tried something new has experimented with optimization.

But such simple tweaking is merely a baby step. As marketers become skilled at testing elements on a page, watching to see what works, and implementing winning elements, they become able to test and optimize for more intricate needs.

For example, a couple of years ago, marketing executives at Restoration Hardware wanted to improve usability on their website. They started with simple tests, experimenting with where they placed certain details, such as product copy, on the product detail page.

Then they tested the placement of key gift items on a page, during the holiday season.

Finally, they tested different cross-sell products on the checkout page.

Overall, by experimenting with various elements, the company was able to increase ROI by 267% over the duration of the test.

But perhaps more important, they learned how to test quickly and often, and they learned that experimentation can lead to valuable knowledge about their consumers.

For an article on the lessons Restoration Hardware learned from testing, click here.

Targeting Made Simple - Optimization Approach #2

You want to sell more?  Be more relevant.  Of the four types of optimization technology that I have been writing about recently, targeting is the simplest way of exploiting differences among your visitors.

For example, if you find that different groups of people - say, visitors who arrive from Yahoo vs. visitors who arrive from Google - respond differently to different layout or copy, for example, then you should make sure you can show those groups different content to make the most of their preferences.

Like testing and experimentation, targeting is a white box approach that allows you to actually watch what is taking place and how different groups of visitors act. Whereas black box approaches use technology that takes place behind the scenes, a white box approach lets marketers set up tests, with a control, learn what visitors are responding to, and make decisions about future interactions based on those actions.

What is it?

Most marketers are already doing targeting to some extent. The bulk of landing page optimization is a combination of experimentation and targeting. When a marketer makes a decision about what page he wants his search traffic to land on (interior site page vs. home page), that's targeting. Rules-based targeting is simply doing it faster, and under more defined circumstances, to get more yield.

Adwords is a perfect targeting example.  You specify which words you want to buy and other rules like type of match and bid and, "voila", you are defining rules-based targeting for your prospects.  Who knew you were already in so deep:

Targeting_keyword

Targeting and rules-based targeting are probably the most prevalent form of optimization. In its simplest form, targeting can be used for landing page optimization by showing specific content to visitors based on the keyword they typed.

And it doesn't end there.  Where do we go from the keyword ad.  Why, to a landing page, which has been targeted based on the keyword.  Here is one we use for the ad above:

Targeting_lp

But targeting is by no means trivial. Behavioral targeting can be very sophisticated, from profiles or persona-based targeting to scenario-based approaches that can model a complete customer purchase cycle and target content by stage or maturity.

Targeting's strength is its simplicity and transparency. That is why it is likely the most prevalent form of  optimization or personalization in use today.

At its most basic, targeting is about searching for groups of people that respond similarly. First time buyers may respond in one way, while those who buy several times a year respond differently, and those who have yet to make a purchase respond in a different way still. Successful targeting exploits the differences between those groups by showing different things to the different groups.

Keys to successful targeting

In order to perform rules-based targeting successfully, you must be able to do several things:

1. Identify different groups of visitors that will behave differently

Different groups might be those who come via natural search vs. those who come from paid search, email recipients who tend to respond vs. those who don't respond, regular buyers vs. non-buyers, etc.

Of course, you can't target based on information that you don't have. If you don't have access to buying history, then targeting to first-time buyers vs. long-time buyers is not a good choice.

Note that every company likely has at least four different groups of visitors. These groups may be divided according to your offerings (people who search for home mortgages vs. people who shop for car loans) or according to gender, geographical area, or other attributes that are ripe for exploiting.

2. Identify the elements that "matter"

While you may have identified several different groups that have the potential to behave differently, they may only behave differently about certain things. Weekend visitors may act differently than weekday visitors in that weekend visitors respond better to longer-form content while weekday visitors respond better to bullet points. On the other hand, the two groups may respond identically to what image you show - product photograph vs. lifestyle shot.
< 
This step is a matter of choosing, among the hundreds of elements within a site or page, what will make a difference.
 
3. Have the ability to run tests, quickly and easily

Of course, you may isolate several elements that you think will matter among different groups, but you won't know if they do until you run tests to see what works. If you believe that people coming from Google will behave differently than people coming from MSN, you need the ability to have a control, to show the control to a certain number of both groups, and to test your variations to both groups in order to learn whether your hypothesis was correct.
 
4. Have the ability to show different things to different groups of people on an ongoing basis
 
Once you learn that various groups do indeed behave differently, you need the ability to continue to serve those groups different content based on their needs. You must, in other words, be able to set the "rules", to say, "In x circumstance, when a visitor behaves like y, I want to serve z content."

The key here is that you must be able to do this without having to go through changing routers, reconfiguring the page, or any other action that requires software development any time it needs to happen. You must be able to do this on the fly, quickly, in order to be able to use it all the time.

If you can do this quickly, it's yours - it's a marketing decision. If it takes too long, or if it requires IT involvement, it's an IT decision and is no longer useful to marketers.

Luckily, there are tools like Offermatica that allow marketers to lay out their guidelines, saying, "On this spot, on this page, show this content only under these circumstances…"

The good, the bad and the ugly

Targeting doesn't happen by magic. You have to set it up and define the rules. You have to be able to test, and then serve different content on an ongoing basis.

But because almost every company has at least four significant customer types that have measurably different buying behaviors, targeting is invaluable in learning how to recognize them and exploit their different buying types.

Google's Website Optimizer - Let the Games Begin, again.

Last fall, Google announced that it was beta testing its Google Website Optimizer. Beginning today, the optimizer is in general release and available to everybody.

Tom Leung, the product manager for Google's Website Optimizer, says in a ClickZ article: "The main problem we're trying to solve is to get people out of the dark ages in terms of how they develop pages. All too often, they'll just put a page together and maybe the designer will do a few mock-ups, and they'll point to the one they feel is going to be the best one."

Sound familiar? It's the "Test, don't guess" philosophy we've been championing for three years. Google should be telling telling advertisers that optimizing is a smart thing to do, because even though most advertisers realize that testing and optimizing landing pages can improve results substantially, having Google say Do it amounts to a marching order for many. There is virtually no amount of marketing budget that I could spend that would be as validating.

On the other hand, Leung's implication that Google's solution can replace large-scale platforms like ours is misguided. "One of those guys can theoretically handle 100,000 page variations. We go up to 10,000," he says in the ClickZ article. He adds that a marketer testing that many variations would be in the .001 percentile. But (perhaps intentionally) he misses the point. He's right that marketers for the most part don't need to test 100,000 variations.

What marketers do need, however, is the ability to test things beyond the landing page. A marketer needs to be able to test an offer on a landing page and then draw that offer through the user's next step, and his next, to include it on internal pages within the site. They need the ability to test the offer within the site, and to continue to optimize the site's internal pages based on the test's results.

So am I concerned about Google's optimizer? When a company with $10B in cash enters your market, things are going to change, and we over here at Offermatica may be delusional, but we are not stupid.

But I trust that the market is (mostly) rational, and so I am very optimistic that this will be continue the surge in interest for Offermatica's testing and targeting service.

Why? Because quite simply, more people need to do testing, and a free tool for a small business is a great start.  For the brands we serve,

1. Offermatica integrates well with the leaders in analytics, WebTrends, Omniture, and CoreMetrics, and can even deliver content to segments derived from these systems.

2. Google's freebie tool is for Google Adwords. Offermatica is for Google, Yahoo, and MSN  -- not to mention display ads, email, affiliates and any other source of traffic, and can be used both externally and internally within your site.

3. As we have covered in What is Optimization, discrete testing is important, but ongoing content targeting and personalization must follow quickly behind.

4. And, as Bryan Eisenburg pointed out during the eMetrics show, "The Page is Dead". If you want better advertising, you have do more that just single page content-switching.

We continue to be passionate about not only providing the highest-performing testing and targeting application for companies like Intuit and C*net, but also providing the kind of marketing guidance and service through our own team and our partners like Camelot, Zaaz, Range Online Media, Wunderman, Digital Grit and others that are leaders in interactive and search marketing.

I want to be clear. Hats off to the team. Truly. We are passionate about better online marketing, we appreciate Google's participation, and we toast their success. Marketers must get into the mold of testing more often, and it cannot happen fast enough.

There is no looking back now, marketing is now officially beyond the site.

Continue reading "Google's Website Optimizer - Let the Games Begin, again. " »