Updating module versions with minimal downtime

December 21, 2021

Starting a new version of a module can take some time (from several seconds to few minutes), depending on its complexity, which will make the module unavailable during this operation, as well as the modules depending on it. This can be even more problematic when it involves a template set module: the consequence will be the unavailability of the sites using this template set during the start operation. The following procedure will explain you how to perform a module deployment in a cluster with limited downtime.

Reminder: The definitions provided by the newer version of the module must be backward compatible. Please consult the definition modification page for more information.

Strategy

The general idea is to redirect the traffic of your site(s) on some cluster nodes, while deploying the modules on the other cluster nodes. And then redirect the traffic on the nodes with the new modules while performing the deployment on the remaining cluster nodes.
For this, the nodes, which will get the deployment of modules in a first turn (including the processing node), are isolated into a dedicated cluster group. By default, all cluster nodes are part of a reserved ‘default’ group.
Alternative procedure is creating cluster groups per node and performing the deployment of modules node by node. Both approaches are described in next sections.

Deployment using two groups

In the following we assume that the cluster consists of four nodes: 192.168.0.10:7860, 192.168.0.20:7860, 192.168.0.30:7860, 192.168.0.40:7860, where 192.168.0.10 is the processing node. Note, please, here we are using the IPs and port numbers of the nodes, which are defined in the jahia.node.properties by cluster.hazelcast.bindPort.
The deployment of modules will first go to the processing node (192.168.0.10:7860) and one of the browsing nodes (192.168.0.20:7860).
Note, please, that during those operations any other module management actions should be avoided.

  1. Using a load balancer, redirect all the incoming requests to the second and third browsing nodes (192.168.0.30:7860, 192.168.0.40:7860) thus excluding first two, where the deployment of modules will happen.
  2. Create target deployment group which will get the module deployment first. This will create a new cluster group with the name deployment, which will contains two nodes (the nodes will quit the ‘default’ group):
    curl -s --user root --request POST --data "name=deployment&nodes=192.168.0.10:7860,192.168.0.20:7860" http://localhost:8080/modules/api/cluster/groups
  3. You can verify the cluster group and node state using:
    curl -s --user root http://localhost:8080/modules/api/cluster/groups
  4. Deploy and start the new versions of the modules on the nodes of group ‘deployment’, using the REST API and by specifying the target group. Note, please, it is important that the request is executed against the node, which is part of the target cluster group:
    curl -s --user root --form bundle=@modulea-2.0.jar --form start=true --form target=deployment http://localhost:8080/modules/api/bundles
  5. Verify that the modules are successfully deployed and started on all the nodes of the group ‘deployment’
  6. Using a load balancer, redirect all the incoming requests to the nodes of  ‘deployment’ group.
  7. Deploy and start the new versions of the modules on the remaining nodes (group ‘default’), using the REST API:
    curl -s --user root --form bundle=@modulea-2.0.jar --form start=true http://localhost:3030/modules/api/bundles
    Note, that here the deployment request is executed against the browsing node, which is part of the default group.
  8. Verify that the modules are successfully deployed and started on the nodes of the group ‘default’.
  9. Remove the temporary created cluster group ‘deployment’:
    curl -s --user root --request DELETE http://localhost:8080/modules/api/cluster/groups/deployment
  10. This will drop the group and let all nodes join back the ‘default’ group.

Deployment using a group per node

In the following we assume that the cluster consists of four nodes: 192.168.0.10:7860, 192.168.0.20:7860, 192.168.0.30:7860, 192.168.0.40:7860, where 192.168.0.10 is the processing node. Note, please, here we are using the IPs and port numbers of the nodes, which are defined in the jahia.node.properties by cluster.hazelcast.bindPort.

Note, please, that during those operations any other module management actions should be avoided.

  1. Create cluster groups per node. This command will create four cluster groups (one per cluster node) and move nodes to their corresponding groups:
    curl -s --user root --request POST --data "" http://localhost:8080/modules/api/cluster/groups/_per-node
    The groups will be named in our case (automatically):
    • group-192-168-0-10-7860
    • group-192-168-0-20-7860
    • group-192-168-0-30-7860
    • group-192-168-0-40-7860
  2. You can verify the cluster groups and node membership using:
    curl -s --user root http://localhost:8080/modules/api/cluster/groups
  3. Using a load balancer, disable incoming requests for processing node (192.168.0.10:7860).
  4. Deploy and start the new versions of the modules on the processing node (group-192-168-0-10-7860) using the REST API and by specifying the target group. Note that it is important that the request is executed against the node, which is part of the target cluster group:
    curl -s --user root --form bundle=@modulea-2.0.jar --form start=true --form target=group-192-168-0-10-7860 http://localhost:8080/modules/api/bundles
  5. Verify that the modules are successfully deployed and started on processing node
  6. Using a load balancer, you can enable the incoming requests to processing node
  7. Repeat the steps 3. to 6. for each of the remaining nodes, using their corresponding cluster groups as target and using the URL of the node, which is part of that group:
    • curl -s --user root --form bundle=@modulea-2.0.jar --form start=true --form target=group-192-168-0-20-7860 http://localhost:2020/modules/api/bundles
    • curl -s --user root --form bundle=@modulea-2.0.jar --form start=true --form target=group-192-168-0-30-7860 http://localhost:3030/modules/api/bundles
    • curl -s --user root --form bundle=@modulea-2.0.jar --form start=true --form target=group-192-168-0-40-7860 http://localhost:4040/modules/api/bundle 
  8. Important: after the deployment is done on all nodes and is successful, perform the deployment on default group (default deployment target so the parameter can be omitted):
    curl -s --user root --form bundle=@modulea-2.0.jar --form start=true http://localhost:8080/modules/api/bundles
  9. Remove the temporary created cluster groups by using the REST call
    curl -s --user root --request DELETE http://localhost:8080/modules/api/cluster/groups/_all
  10. This will drop the temporary groups and let all nodes join back the ‘default’ group
  11. You can verify the cluster groups and node membership using:
    curl -s --user root http://localhost:8080/modules/api/cluster/groups