jCustomer event checker
With the introduction of JSON Schemas in jCustomer 2x, any events submitted to a jCustomer 2x environment must comply with a pre-existing JSON Schema to be accepted and processed.
Although the migration process (from jCustomer 1x to jCustomer 2x) will handle the migration of existing events, this migration itself is not related to JSON Schemas and any event received an environment has been migrated will still require a valid Schema.
To help in validating your readiness to migrate to jCustomer 2x, we build a custom event checker tool that will automatically fetch all of your existing events (in the past 60 days by default), migrate them to jCustomer 2x data model, and validate them against existing JSON Schemas. By looking at error messages, you will be able to update your JSON Schemas accordingly.
Prerequisites
To use the event checker, you will need to following:
- A jCustomer environment running jCustomer 1.9.1 or above. This environment can be a copy of your production data with a single jCustomer 1.9.1 instance connected to it.
- A jCustomer environment running jCustomer 2.3.0+. This can be a development instance used to create/update JSON Schemas.
- A machine able to run either Docker or Node 18+ and capable of reaching both jCustomer environments
/cxs/(admin) endpoints
Note that the tool only validates events, it does not modify existing events nor submit new events for processing.
Configuration
On your machine, create a configuration file containing credentials to both jCustomer environments.
{
"source": {
"url": "http://localhost:8181",
"username": "karaf",
"password": "karaf"
},
"target": {
"url": "http://localhost:8181",
"username": "karaf",
"password": "karaf"
}
}
With source being the jCustomer 1.9.1 environment and target being the jCustomer 2.3.0 environment.
We will assume this file is saved in /home/jahia/checker-config.json for the rest of this document.
Prepare for running the script
For the sake of verifying the script can be properly executed on your machine, we will execute it with the --help flag.
The rest of the document will be agnostic to NodeJS or Docker, simply adjust the instructions according to the runtime you're using.
Using NodeJS 18+
To run the script on your local NodeJS 18+ environment, simply enter the following commands:
➜ ~ npx @jahia/jcustomer-custom-event-checker@latest --help
Need to install the following packages:
@jahia/jcustomer-custom-event-checker@0.7.0
Ok to proceed? (y) y
Script that check events coming from a jCustomer to another one
VERSION
@jahia/jcustomer-custom-event-checker/0.7.0 darwin-x64 node-v18.12.1
USAGE
$ jcustomer-custom-event-checker [COMMAND]
COMMANDS
help Display help for jcustomer-custom-event-checker.
validateEvents This script will get the events from a jCustomer instance and will validate them on another one.
We're using the --help flag to verify the script is available.
Using Docker
➜ ~ docker pull jahia/jcustomer-custom-event-checker:latest
latest: Pulling from jahia/jcustomer-custom-event-checker
Digest: sha256:a36acb906e8eb40d5b1ab9580249407823e5d1fe7749e2bef94d9228d8f2832d
Status: Image is up to date for jahia/jcustomer-custom-event-checker:latest
docker.io/jahia/jcustomer-custom-event-checker:latest
➜ ~ docker run -v /home/jahia/checker-config.json:/root/checker-config.json:ro jahia/jcustomer-custom-event-checker jcustomer-custom-event-checker --help
Unable to find image 'jahia/jcustomer-custom-event-checker:latest' locally
latest: Pulling from jahia/jcustomer-custom-event-checker
f56be85fc22e: Pull complete
3f026796e5ad: Pull complete
08556a236d1c: Pull complete
06e990ca428a: Pull complete
2dd8694ad18b: Pull complete
bd28cd755344: Pull complete
207f17b80fd3: Pull complete
Digest: sha256:a36acb906e8eb40d5b1ab9580249407823e5d1fe7749e2bef94d9228d8f2832d
Status: Downloaded newer image for jahia/jcustomer-custom-event-checker:latest
Script that check events coming from a jCustomer to another one
VERSION
@jahia/jcustomer-custom-event-checker/0.7.0 linux-x64 node-v20.1.0
USAGE
$ jcustomer-custom-event-checker [COMMAND]
COMMANDS
help Display help for jcustomer-custom-event-checker.
validateEvents This script will get the events from a jCustomer instance and
will validate them on another one.
Available arguments
The list of available arguments can be retrieved by displaying the --help for the validateEvents command.
➜ ~ docker run jahia/jcustomer-custom-event-checker jcustomer-custom-event-checker validateEvents --help
This script will get the events from a jCustomer instance and will validate them on another one.
USAGE
$ jcustomer-custom-event-checker validateEvents -f -o -s
-d -t
FLAGS
-d, --limitOfDays= (required) [default: 60] Exclude events
older than this flag in days
-f, --configFile= (required) [default: ./defaultConfig.json]
jCustomer JSON configuration file location
-o, --out= (required) [default: ./errors.json] Exported
file path
-s, --step= (required) [default: 1000] Number of events
to process per batch
-t, --scrollTimeValidity= (required) [default: 2h] Period to retain
the search context for scrolling query .
Value in time unit
--createScopes If scopes are missing, the script will attempt to create those in the remote jCustomer
DESCRIPTION
This script will get the events from a jCustomer instance and will validate
them on another one.
The structure of the configuration file can be found in the defaultConfig.json
file at the root of this project
EXAMPLES
$ jcustomer-custom-event-checker validateEvents --configFile=./path/to/your/config/config.json --out=./out.json
Start the events analysis
Looking for configuration in file ./path/to/your/config/config.json
Processed 315 events in 1546 ms
Events scopes
During schema validation, jCustomer will ensure events are assigned to an existing scope and will fail validation if they are not.
The migration process will take care of creating the missing scopes, but if you're validating events against a new and "un-migrated" jCustomer instance (a new instance you just created), scopes will be missing and will generate many schema validation errors in the output file. Since the creation of these scopes is handled during the migration itself these errors, although legitimate, are irrelevant in the context of verifying a migration.
As such, the script:
- by default, will halt the validation process if it finds a missing scope. If you're validating against an instance that was already migrated, this is not an issue since the needed scopes will be present (created during the migration). If you are validating against a new instance, the script will halt during its initial phase.
- contains an non-default argument "--createScopes", that will automatically create missing scopes on the target jCustomer 2x instance.
Review results
➜ ~ npx @jahia/jcustomer-custom-event-checker@latest validateEvents --configFile=/home/jahia/checker-config.json --out=./out.json
Looking for configuration in file ./defaultConfig.json
Start the events analysis... done
Processed 30659 events in 58269 ms
Errors will be recorded grouped by event type in the file provided via the --out parameter (by default in ./errors.json) and will look like this:
{
"view": [
{
"error": "Unknown scope value at $.target.scope for value acme_europe"
},
{
"error": "Unknown scope value at $.target.scope for value acme_asia"
},
{
"error": "Unknown scope value at $.source.scope for value acme_canada"
},
{
"error": "Unknown scope value at $.scope for value ehf_myproduct"
},
{
"error": "There are unevaluated properties at following paths $.eventType, $.itemId, $.itemType, $.persistent, $.profileId, $.properties, $.scope, $.sessionId, $.source.itemId, $.source.itemType, $.source.properties"
},
{
"error": "Unknown scope value at $.target.scope for value myproduct"
}
]
}
Using the content of the errors file, you will be able to pinpoint how these events were generated and how the corresponding schema should be updated.
The Elasticsearch search context is only retained for 2 hours, which means that for situations in which the script will run for more than 2 hours (for example with a vast amount of data), the