I haven’t found much information about A/B and Multivariate testing specifically for Miva Merchant, yet testing is so important for improving sales/lead conversions it cannot be ignored.

If you want to do A/B testing on e-commerce template pages, this post is for you. If you’re using Miva Merchant, that’s even better. If you’re using the SEO Friendly URLs feature in Miva Merchant, then this method was written exactly for you.

Let’s get started!

The Test

Recently, I designed a Product Page template that I felt was superior to our existing Product Page template for a number of reasons. I wanted to test and see if we would see marked improvement for the following:

  • Increased Number of Basket Clicks
  • Increase in overall average revenue per visitor

Here is how to set up this test.

Create a Variable Product Page

To perform an A/B Test for the Miva product pages, two different pages are required.

First Page: Control (Miva’s PROD page)
http://www.domain.com/mm5/merchant.mvc?Screen=PROD&Product_Code=XXXXXX

Second Page: Variable(s)
To create a variable page, duplicate the PROD page entirely (ALL Code, assigned Items and applied Settings) and name it something else. I named mine VARTST:
http://www.domain.com/mm5/merchant.mvc?Screen=VARTST&Product_Code=XXXXXX

Alter the code, change styles, add scripts as you see fit on this duplicated version. This will be the “B” of your A/B test, but you can create as many variables as you would like to test at this point.

(SEO Tip: Make sure you add a meta Robots Noindex,nofollow in the HEAD tags of the Variable page so the search engines do not think you have two versions of the same page when they crawl your site!)

The Challenge – URL Rewrites

We’re off to a good start but if you are using search engine friendly URLs, we need a tool that can adapt to this and ensure users are seamlessly directed to your TEST version as applicable.

For us, our product pages look like this:

http://www.domain.com/store/CATEGORY-NAME/PRODUCT-CODE.html

or just simply:

http://www.domain.com/store/PRODUCT-CODE.html

In the past, I would use Google Website Optimizer to run Multivariate tests on my pages. Google has since gotten rid of Website Optimizer and now has an (inferior) A/B testing feature in Analytics called “Content Experiments”.

At the time of my test, Google’s new tool has no Multivariate capability. This may have changed. The point is that Miva URL Rewrites for template pages make testing impossible with Google’s tool. Just try and you’ll go bald from pulling your hair out.

(On a side note, if you DO know a method to make Content Experiments work to test any MIVA template page like CTGY or PROD where URLs are rewritten, then please share it!)

Finding The Right Tool – Optimizely

After doing research into A/B Testing Tools, I found Optimizely. It is reasonably priced, comes with a 30 day trial period, responsive support and an intuitive interface that runs almost any kind of test.

A/B Testing Your Website with Optimizely - 1

The trial is perfect because you can ensure the tool is working correctly before you ever pay for it.

Because we chose Optimizely, I’m going to walk through the steps we took to run our test while respecting our rewritten URLs.

Setting Up The Experiment


Create an Optimizely trial account.

Sign in and select “Set up a New Experiment”

A/B Testing Your Website with Optimizely - 2

When asked to insert the Experiment URL, you will want to enter the Miva dynamic URL for your PROD page:

http://www.domain.com/mm5/merchant.mvc?Screen=PROD

A/B Testing Your Website with Optimizely - 3

You should see the PROD Page template appear in the Preview Frame.

Next, click the Variation #1 button. Click the small dropdown arrow. There is an option that says “Redirect to a New Page”. Click this option and enter your Redirection URL:
http://www.domain.com/mm5/merchant.mvc?Screen=VARTST

A/B Testing Your Website with Optimizely - 3-1

*Note: Your variation page should not have ‘PROD’ in its name, i.e., do not name your variation something like “PROD2” or “PRODTEST”. The test will not work with this method!*

Redirect Fixes – Select Targeting


In the Edit Mode for the Experiment, click “Targeting” in the Options dropdown.

A/B Testing Your Website with Optimizely - 4

Tell it to apply this test to everyone who visits your rewritten site URLS. Add everything up to the dynamic content, such as Category Codes or Product Codes, for instance:

http://www.domain.com/store/

In the dropdown, select “Substring Match” and then click “APPLY”. This ensures the test goes to all visitors. (More on this later)

A/B Testing Your Website with Optimizely - 5

Redirect Fixes – A Script for Redirects

Now, Optimizely needs to do a little extra code jiujitsu to get users to the correct variation when users visit your PROD Page.

First, click the “Variation #1” Tab at the top. You will probably see a popup that says “Redirection Notification” and your Variable page. If you DO NOT see this, you need to click the Dropdown arrow next to Variation #1 and click “Redirect to a New Page”.

In “Editing Mode”, click the button in the bottom-right that says “Edit Code” A menu appears.

A/B Testing Your Website with Optimizely - 6

What you will need to place here is logic to strip the Product Code from your rewrites and stick it at the end of the dynamic Variable URL:

/* _optimizely_redirect=http://www.domain.com/mm5/merchant.mvc?Screen=VARTST */
var url_no_slashes = window.location.href.split(‘/’);
var url_html = url_no_slashes[url_no_slashes.length – 1];
var url_product_code = url_html.split(‘.html’)[0];
window.new_url = “http://www.domain.com/mm5/merchant.mvc?Screen=VARTST&Product_Code=” + url_product_code;
window.location = window.new_url;

Obviously the code you will use depends on how you are rewriting your URLS and the name of your Test page.

Change as needed.

Setting up Goals


Goals can track almost any kind of engagement or conversion; in our case, we want to track Basket Clicks and Revenue.

At the top of the Optimizely Edit screen for this test there is a “Set Up Goals” button.

A/B Testing Your Website with Optimizely - 7

Click this, then click “Add A Goal”.

A/B Testing Your Website with Optimizely - 8

Name it “Basket Clicks”. Under “What to Track” select “Clicks”.

In the view that displays the PROD page, click the “Add to Basket” button by clicking it.

Now, click the “Advanced” Tab. Change the Custom Event Name and ensure the Selector is correct (in our case it is the INPUT for the Basket button)

Under Goal targeting, make sure you select “Specify which customer pages on which to track this goal” and add: www.domain.com/mm5/merchant.mvc and www.domain.com/any-rewrite-categories/ For the URL Match Type, you will select “Substring Match”.

Now, save this goal! Here’s an example of these settings at work:

A/B Testing Your Website with Optimizely - 9

Next, we need a revenue goal. Click the Set up Goals button again and add “Revenue”.

The Optimizely snippet will do its work by using a Tracking Event on your Invoice (INVC) page. We will add these snippets in the next step.

Adding Optimizely Snippets

In this step, you will add the Optimizely snippet to the portion of your PROD page, the VARTST page and your INVC Page.

VERY IMPORTANT – Remember earlier in “Select Targeting” we told this test to apply to ALL pages of the website? Make sure you ONLY install the tracking script on the PROD page, its Variation page and the INVC page! DO NOT install this script sitewide! Doing so will cause your urls to redirect incorrectly to your variable page!

One More Heads Up – Since Optimizely uses a single script tag to run your tests, you have to Archive this test before you create a new one for a different page. Otherwise, the JavaScript additions we’ve added will apply to those pages and potentially break something.

I hope this tutorial has been helpful!

Huge thanks goes out to the fine staff at Optimizely for their patience and assistance to make this work. If interested in Optimizely testing for your business, please click this link as the referral helps me out as well: http://optimize.ly/Ou7FQ2