Creating AB tests in Page Composer

November 14, 2023

In Page Composer, you can create and apply A/B tests to content on a page or to an entire page. You create an A/B test by adding multiple content items to an area in a page or by adding new pages. The additional content items and pages are called variants. Then you specify conditions under which variants display to site visitors.

When a test starts, you can also monitor the results of the test in Page Composer.

Creating an A/B test for content on a page

To create an A/B test for content on a page:

  1. Navigate to a website page that is powered by Jahia.
  2. On the page, right-click on the content where you want to add the A/B test. Select Add A/B test.
    adding_optimization_to_content.png

    A toolbar displays at the top of the content item.
    adding_optimization_toolbar.png
  3. To add new content to use in the A/B test, click the button at the top-right of the content item. In this example, the News entry button displays.
  4. In the New Content page, enter information for the new page. 
  5. Open the Edit Content page by clicking on the content's name:
    optimization_toolbar.png
  6. Click Save.

Next, specify settings for the A/B test.

Creating an A/B test for a page

To create an A/B test for a page:

  1. Navigate to a website page that is powered by Jahia.
  2. In the navigation menu on the left, right-click on the page that you want to create an A/B test for. Select Add A/B test.
    make-ab-test.png
  3. Click Add on the toolbar that displays at the top-right of the page.
    optimization_toolbar_page_01.png

    Note: By default, the existing page is copied as the next variant so that you can modify it instead of creating a new page.
  4. To edit the new variant page, select Edit page name in the toolbar.
    optimization_toolbar_page_02.png
  5. In the Edit Content page, modify the variant page as needed. Then, choose Save.

Next, specify settings for the A/B test.

Specifying settings for an A/B test

To specify settings for an A/B test:

  1. Navigate to the content or page and open it for editing.
  2. On the Edit Content page, select Content and complete the following fields.
    edit-AB-test.png
      
    Field Usage
    System name Name of the test in the repository
    Active Whether the test is active. Selected by default. Deselect to stop or pause the A/B test.
    Ajax rendering If you experience issues displaying variants, it may be because of a JavaScript or CSS conflict. This issue may occur with content coming from an external service that is embedded in your page. Select the checkbox to try an alternate type of rendering and see if that resolves the issue.
    Starts on The date and time that the test will start. If the field is empty, the test will start immediately (as soon as the page is published).
    Ends on The date and time that the test will stop. If the field is empty, the test continues until it is stopped by a manual action.
    Maximum hits Maximum number of times the A/B test is served to users before it stops. For example, if you have two variants and specify 4,000 hits, then each variant will display 2,000 times (whatever the number of conversions) and then the test will stop.
    End strategy Defines what happens when the test ends. You can specify that Jahia automatically uses the winning variant (the one with the highest conversion rate) or continues to serve the current control variant. If you choose to keep the current control variant, then you can decide whether to promote another variant when the test ends.
    Control variant Defines which variant is the control.
  3. In the navigation pane, select Goal and define a goal to measure your success rate.
    Note: As soon as a test has started, you cannot change the goal. This ensures your results are valid.
  4. Click Save.
  5. Publish the content or page.

When the test starts, you can monitor test results.

Interpreting test results

Once an A/B test has started, you can follow results as they occur.

  1. Navigate to the page containing your test.
  2. Open the Edit Content page by clicking on the content's name.
    optimization_toolbar.png
  3. Click on the Goal tab. Test results display in a table. This level of reporting enables you to easily identify which variant has the highest conversion rate for the overall audience.
    Note: When the test is running, you can no longer modify goals for the test. Changing the goal during the test would lead to incoherent data. 

    Columns in the table display the following:
    • Views
      Number of times the variant has been displayed
    • Success
      Number of visitors who saw this variant and converted (achieved the goal)
    • Conversion
      Conversion rate
    • Lift
      The percentage difference between your control variant and other test variants
  4. To break down test results further, select a filter from the Variants for current A/B test list. You can discover if another variant works better for a subset of the audience.
    In this example, promoting the winner will improve the conversion rate. You could also further improve results by creating a permanent personalization to serve the best variant for each segment of the audience.