Release notes

November 14, 2023

For previous releases of Augmented Search, please consult the "View older versions" on the Jahia store page

3.4.0

What's new?

Augmented Search 3.4.0 is a maintenance release, with a focus on error management:

  • if an issue prevented the first indexing of a page, the page indexing will be triggered upon the next modification/publication of the page
  • a GraphQL mutation has been added to trigger the indexing of specific nodes

Changelog

  • Added recording of Elasticsearch errors per site in JCR (only the last 100)
  • Added error management to automatically reindex the full document if parent document is missing
  • Added graphql endpoint to be able to index a specific node
  • Added the possibility to disable the indexation of a folder
  • Fixed issue with node type mapping for files not taken into account when searching in all content
  • Fixed issue with deleted page not removed from the default elasticsearch index (used in edit / preview)
  • Fixed healthcheck error when there's no site deployed
  • Fixed exceptions during indexation due to BulkProcessor objects still in memory
  • Fixed issue with the page title having less weight than the page content by updating the Elasticsearch main field
  • Fixed issue with lastModification and lastPublication info not indexed for files and jmix:mainResource
  • Fixed issue with weakreferences not well indexed
It is required to be using AS 3.3.0+ to upgrade to AS 3.4.0. In the eventuality that you are using a version prior to AS 3.3.0, please first upgrade to AS 3.3.0 (see below) before upgrading to AS 3.4.0
  1. Deploy and start AS 3.4.0
  2. Undeploy the previous Augmented Search version(s)
  3. Start a reindexing of the platform
    • Several indexing issues are fixed in AS 3.4.0, so we do recommend a reindexing to benefit from them and to have a consistent index. Please note that the search will remain available during this operation, using the previous index. 

What's new?

Augmented Search 3.3.0 comes with the following noticeable improvements:

  • More possibility to configure boosts, now including file dedicated boosts
  • The possibility tu use Function score in your queries
  • A new way to index files, resulting in a reduction of the disk space use and indexation time in multilingual context
    • This change also reduces the weight of files in the search results, compared to page contents, solving an issue where files were appearing too high in the search results. If the impact on the file releavance is too important,  the file boost configuration can be adjusted.
  • The best number of shards is now automatically calculated before reindexing (this behavior is configurable)

Changelog

  • Added a new index for files only
  • Added a language detection plugin
  • Added a new property "org.jahia.modules.augmentedsearch.extracted.text.maximum" for the file content indexation (from 100Ko to 1Mo)
  • Added search-in functionality for the new file index
  • Added default index and component templates (more details here)
  • Added new boost values in configuration file for fine tuning search results
  • Added possibility to boost specific content with a json file
  • Added the possibility to get score functions from modules
  • Improved indexation by avoiding files references
  • Added property "org.jahia.modules.augmentedsearch.file.mappedNodeTypes"
  • Added upgrade scripts to add the new properties in the cfg file
    org.jahia.modules.augmentedsearch.numberOfShards (-1 by default - meaning automatically calculated)
    org.jahia.modules.augmentedsearch.numberOfReplicas (-1 by default - meaning automatically calculated)
    org.jahia.modules.augmentedsearch.maxShardSizeInGb (20 by default)
    org.jahia.modules.augmentedsearch.minHeapPerShardInMb (50 by default)
    org.jahia.modules.augmentedsearch.file.mappedNodeTypes
  • Added usage of the prefix for ElasticSearch index name if available (see elasticsearch-connector-3.2.0)
  • Added a new GraphQL entry point to estimate the best number of shards to use
  • Added SAM probe to compare shards number with module recommendation
    Verifies the number of shards. This probe has a default severity of HIGH and can return a GREEN, YELLOW, or RED.
    The following messages are available:
    1 - AS status is OK (GREEN - everything is OK)
    2 - Number of shards can be improved, should be between min and max, is now nb (YELLOW - indicates it differs by 1)
    3 - Number of shards can be improved, should be between min and max, is now nb (RED - indicates it differs by more than 1)
    4 - Cannot get information from AS (RED- the server produced an Exception)
  • Added a step to update the shards number before full indexation
  • Added indexation of extension nodes in edit and live
  • Added a force re-indexation option in GraphQL query and UI
  • Optimized the query used to retrieve indexed sites
  • Improved memory usage by refreshing the JCR session more frequently
  • Improved license handling by adding violation message in Augmented Search UI
  • Fixed query for finding document impacted by category change
  • Fixed issue with force indexation when indexationEnd doesn't exist
  • Fixed performance issue by avoiding heavy query inside the JCR listener
  • Fixed issue with event replay after stopping / restarting AS (cluster only)
  • Fixed issue with legacy search API when site parameter is empty
  • Fixed minor security vulnerabilities

How to upgrade - Jahia Cloud

The upgrade operation will be performed by the Jahia Cloud team, to make it as seamless as possible and to reduce as much as possible the search unavailability during the necessary re-indexing. Jahia Cloud customers will be contacted prior to this upgrade.

Upgrading to Augmented Search 3.3.0 requires to re-index your platform. During this re-indexing, search will not be available

After starting Augmented Search 3.3.0, a re-indexing of the platform is required. During this initial indexing, the search feature will not be available. This is due to the changes done for file indexing, as the index is built differentely, and new indices are created (jahia_as_v3*). These changes have no impact at API level, meaning that no code change is required when upgrading to Augmented Search 3.3.0. These changes also reduce the weight of files in the search results, compared to page contents, solving an issue where files were appearing too high in the search results. If the impact on the file releavance is too important, you can adjust the file boost configuration.

Note that the old indices jahia_as_v2* will not be deleted from Elasticsearch, allowing you to validate the new version. In the eventuality that errors happened, you will be able to go back to the previous version of the Augmented Search module, and have a functioning search without having to reindex.
Once the upgrade has been validated, we advise you to remove the old jahia_as_v2* indices.

Follow the steps below to upgrade to Augmented Search 3.3.0:

  1. Stop Jahia
  2. Set elasticsearch.prefix in jahia.properties (ex: “jahia”)
  3. Start Jahia
  4. Deploy the newest version of elasticsearch-connector for your platform (3.2.0 for 8.1.0+)
  5. Deploy the newest version of jContent for your platform (2.7.0 for 8.1.0+)
    • As there's an incompatibility between the new Augmented Search version and the previous versions of jContent
  6. Deploy a recent version of graphql-dxm-provider for your platform (2.8.0+ for 8.1.0+)
    • As it provides the upload feature required by the boost feature in a json file
  7. Deploy (but DO NOT start yet) the newest version of augmented-search for your platform (3.3.0 for 8.1.0+)
    • As previously mentioned, starting the new Augmented Search version will make the search unavailable until the first indexation is complete
  8. Configure Augmented Search by setting or updating the following properties in org.jahia.modules.augmentedsearch.cfg 
    1. org.jahia.modules.augmentedsearch.prefix (ex: “jahia_as” will make indexes starting with “jahia_as_v3”)
    2. org.jahia.modules.augmentedsearch.numberOfShards (-1 by default - meaning automatically calculated)
    3. org.jahia.modules.augmentedsearch.numberOfReplicas (-1 by default - meaning automatically calculated)
    4. org.jahia.modules.augmentedsearch.maxShardSizeInGb (20 by default)
    5. org.jahia.modules.augmentedsearch.minHeapPerShardInMb (50 by default)
  9. Once the configuration has been done, start Augmented Search 3.3.0
  10. Start the re-indexing
    This can be done by clicking on Index all in Administration > Server > Configuration > Augmented Search, or using the following GraphQL query:
    mutation {
      admin {
        search {
          startIndex {
            jobs {
              id
              status
              project {
                siteKey
              }
            }
          }
        }
      }
    }
  11. After validation of the upgraded environment, remove the jahia_as_v2* indices in your Elasticsearch environment

What's new?

Augmented Search 1.7.0 is a maintenance release, with a focus on error management:

  • if an issue prevented the first indexing of a page, the page indexing will be triggered upon the next modification/publication of the page
  • a GraphQL mutation has been added to trigger the indexing of specific nodes

Changelog

  • Added recording of Elasticsearch errors per site in JCR (only the last 100)
  • Added error management to automatically reindex the full document if parent document is missing
  • Added graphql endpoint to be able to index a specific node
  • Added the possibility to disable the indexation of a folder
  • Fixed issue with node type mapping for files not taken into account when searching in all content
  • Fixed issue with deleted page not removed from the default elasticsearch index (used in edit / preview)
  • Fixed healthcheck error when there's no site deployed
  • Fixed exceptions during indexation due to BulkProcessor objects still in memory
  • Fixed issue with the page title having less weight than the page content by updating the Elasticsearch main field
  • Fixed issue with lastModification and lastPublication info not indexed for files and jmix:mainResource
It is required to be using AS 1.6.0+ to upgrade to AS 1.7.0. In the eventuality that you are using a version prior to AS 1.6.0, please first upgrade to AS 1.6.0 (see below) before upgrading to AS 1.7.0
  1. Deploy and start AS 1.7.0
  2. Undeploy the previous Augmented Search version(s)
  3. Start a reindexing of the platform
    • Several indexing issues are fixed in AS 1.7.0, so we do recommend a reindexing to benefit from them and to have a consistent index. Please note that the search will remain available during this operation, using the previous index. 

What's new?

Augmented Search 1.6.0 comes with the following noticeable improvements:

  • More possibility to configure boosts, now including file dedicated boosts
  • A new way to index files, resulting in a reduction of the disk space use and indexation time in multilingual context
    • This change also reduces the weight of files in the search results, compared to page contents, solving an issue where files were appearing too high in the search results. If the impact on the file releavance is too important,  the file boost configuration can be adjusted.
  • The best number of shards is now automatically calculated before reindexing (this behavior is configurable)

Changelog

  • Added a new index for files only
  • Added a language detection plugin
  • Added a new property "org.jahia.modules.augmentedsearch.extracted.text.maximum" for the file content indexation (from 100Ko to 1Mo)
  • Added search-in functionality for the new file index
  • Added default index and component templates (more details here)
  • Added new boost values in configuration file for fine tuning search results
  • Added possibility to boost specific content with a json file
  • Improved indexation by avoiding files references
  • Added property "org.jahia.modules.augmentedsearch.file.mappedNodeTypes"
  • Added usage of the prefix for ElasticSearch index name if available (see elasticsearch-connector-2.1.0)
  • Added a new GraphQL entry point to estimate the best number of shards to use
  • Added SAM probe to compare shards number with module recommendation
    Verifies the number of shards. This probe has a default severity of HIGH and can return a GREEN, YELLOW, or RED. The following messages are available:
    1- AS status is OK (GREEN - everything is OK)
    2- Number of shards can be improved, should be between min and max, is now nb (YELLOW - indicates it differs by 1)
    3- Number of shards can be improved, should be between min and max, is now nb (RED - indicates it differs by more than 1)
    4- Cannot get information from AS (RED- the server produced an Exception)
  • Added a step to update the shards number before full indexation
  • Added indexation of extension nodes in edit and live
  • Added a force re-indexation option in GraphQL query
  • Optimized the query used to retrieve indexed sites
  • Improved memory usage by refreshing the JCR session more frequently
  • Improved license handling by adding violation message in Augmented Search UI
  • Fixed issue with force indexation when indexationEnd doesn't exist
  • Fixed performance issue by avoiding heavy query inside the JCR listener
  • Fixed issue with event replay after stopping / restarting AS (cluster only)
  • Fixed issue with legacy search API when site parameter is empty

How to upgrade - Jahia Cloud

The upgrade operation will be performed by the Jahia Cloud team, to make it as seamless as possible and to reduce as much as possible the search unavailability during the necessary re-indexing. Jahia Cloud customers will be contacted prior to this upgrade.

Upgrading to Augmented Search 1.6.0 requires to re-index your platform. During this re-indexing, search will not be available

After starting Augmented Search 1.6.0, a re-indexing of the platform is required. During this initial indexing, the search feature will not be available. This is due to the changes done for file indexing, as the index is built differentely, and new indices are created (jahia_as_v3*). These changes have no impact at API level, meaning that no code change is required when upgrading to Augmented Search 1.6.0. These changes also reduce the weight of files in the search results, compared to page contents, solving an issue where files were appearing too high in the search results. If the impact on the file releavance is too important, you can adjust the file boost configuration.

Note that the old indices jahia_as_v2* will not be deleted from Elasticsearch, allowing you to validate the new version. In the eventuality that errors happened, you will be able to go back to the previous version of the Augmented Search module, and have a functioning search without having to reindex.
Once the upgrade has been validated, we advise you to remove the old jahia_as_v2* indices.

Follow the steps below to upgrade to Augmented Search 1.6.0:

  1. Stop Jahia
  2. Set elasticsearch.prefix in jahia.properties (ex: “jahia”)
  3. Start Jahia
  4. Deploy the newest version of elasticsearch-connector for your platform (2.1.0 for 7.3.5+)
  5. Deploy (but DO NOT start yet) the newest version of augmented-search for your platform (1.6.0 for 7.3.5+)
    1. As previously mentioned, starting the new Augmented Search version will make the search unavailable until the first indexation is complete
  6. Add the following properties in org.jahia.modules.augmentedsearch.cfg 
    1. org.jahia.modules.augmentedsearch.prefix (ex: “jahia_as” will make indexes starting with “jahia_as_v3”)
    2. org.jahia.modules.augmentedsearch.numberOfShards (you can use -1 by default - meaning automatically calculated)
    3. org.jahia.modules.augmentedsearch.numberOfReplicas (you can use -1 by default - meaning automatically calculated)
    4. org.jahia.modules.augmentedsearch.maxShardSizeInGb (suggestion to use 20 by default)
    5. org.jahia.modules.augmentedsearch.minHeapPerShardInMb (suggestion to use 50 by default)
  7. Once the configuration has been done, start Augmented Search 1.6.0
  8. Start the re-indexing using the following GraphQL query:
    mutation {
      admin {
        search {
          startIndex {
            jobs {
              id
              status
              project {
                siteKey
              }
            }
          }
        }
      }
    }
  9. After validation of the upgraded environment, remove the jahia_as_v2* indices in your Elasticsearch environment