Optimization tests

  Written by The Jahia Team
 
Marketers
   Estimated reading time:

Making their website perform better, improving it to increase visitors satisfaction and, ultimately, obtain higher conversion rates are certainly top priorities for all marketers. It is, unfortunately, a tedious and complex task, and can be nearly impossible if you do not have the proper tools, to measure the effect - positive or negative - of the changes to the Web site you have decided to update. Marketing Factory integrates the support tools that enable this kind of testing easily. And with precisely measured results and statistical data you can make data-driven decisions versus those based only on assumptions.

Optimization testing is, at its heart, the process of presenting different versions of a page (or a part of a page) to visitors in order to evaluate statistically which version provides the best results (or conversion rate) and user acceptance. Once the test is completed, the marketer can decide, based on precisely collected and measured data, which version provides the best results and then to use it.

1. Creating a test on a Content

To create an optimization test, follow this procedure:

  1. Go into any page of the Web site (powered by Digital Experience Manager).
  2. Right click on the page on which you want to create an optimization test.

  3. Click on Add optimization test.
  4. Create or insert new content in the list.
    Create new content:

    Paste existing content:

    Paste existing content as a reference:
  5. Open the settings dialog by clicking on the main button:
  6. In the dialog box, complete the following fields
     
    Field Usage
    System name Name of the test in the repository
    Active Checked by default. Uncheck to stop / pause the optimization test.
    Ajax rendering If youre experiencing troubles to display some of the variants it may be because of a JavaScript or css conflict. That situation can happen in particular with content coming from an external service and embedded in your page. Check the box to try an alternate type of rendering and see if it solves the issue.
    Starts on The personalization will start on that date. If the field remains empty, the personalization will start immediately (as soon as the page is published)
    Ends on The personalization will stop automatically at that date. If the field remains empty, the personalization will never, until a manual action.
    Maximum hits Maximum number of times the optimization test will be served to users before it stops. If you have two variants and 4,000 hits, then statistically each variant will be displayed 2,000 times (whatever the number of conversions) and will then stop the test.
    End strategy Lets you define what must happen when the test ends. There are two possibilities. First, Digital Experience Manager will automatically use the winning variant (the one with the highest conversion rate). Second, Digital Experience Manager will continue to serve the current control variant. Then it is up to you to decide manually to promote another variant.
    Control variant Lets you choose between all the variants as to which one should be the control
    Define the goal to measure your success rate. As soon as a test has started, it is not possible to change the goal anymore; this ensures your results are valid. (Otherwise, the results could be completely false.)
  7. Save.
  8. Publish the area

3. Creating a test on a Page

To create an optimization test, follow this procedure:

  1. Go into any page of the Web site (powered by Digital Experience Manager).
  2. Right click on the page on which you want to create an optimization test.
  3. Click on Add optimization test.

    Add optimization test on a whole page
  4. Create a new page from the dedicated toolbar that will appear top right :

    Please note that by default, the existing page is copied as the next variant so you can modify it instead of starting from scratch.
  5. If you want to edit the current variant, you can use the toolbar:
  6. To open the settings dialog, go to the main page edit function, either through the left panel site tree or with the edit button that you can find top right :
     

The settings dialog is the same as the one available for content level optimization tests.

4. Test results

Once an optimization test has started, you can follow the results in-context.

  1. Go into the page containing your test
  2. Open the settings dialog by clicking on the main button:
  3. Click on the Goal tab

This tab does not provide the GUI to choose the goal of the test anymore (changing this goal in the middle of the test would lead to incoherent data) but displays the measured results instead.

Results are displayed as a table

  1. Number of views: number of times the variant has been displayed
  2. Success: number of visitors having seen this variant who converted (achieved the goal)
  3. Conversion rate
  4. Lift: the % difference between your control variant and other test variants.

This first level of reporting allows you to easily identify which variant has the higher conversion rate and may increase performances if promoted as default variant when the test ends.

But in fact, reports work like all the other reports in Marketing Factory and allows marketers to break-down the results using the filtering options.

Thanks to that capability, marketers are not only able to know that a variant works better for the overall audience, but can see if another variant works perhaps much better for a subset of the audience. In that case, promoting the winner will improve the conversion rate, but creating a permanent personalization to serve the best variant to each segment of the audience for which it works the best can improve the results much higher!