About the DX back-end layer

  Written by The Jahia Team
   Estimated reading time:

This section presents all the various technologies and frameworks available in Digital Experience Manager's back-end layer.

OSGi

OSGi (aka Open Services Gateway initiative) is a dynamic module system for Java. It is currently the most powerful and most mature dynamic module system available for Java applications. In OSGi, modules are actually called "bundles". They are specialized JARs that include extra MANIFEST.MF entries that declare the dependencies between modules, as well as versioning and package information made available to other modules. In effect, most existing JAR can be converted to an OSGi bundle with little work (they are even automatic transformation tools available). In the OSGi runtime, only packages that are exported will be available and visible to other bundles, only if the using bundles also import them. So in effect there can be fine-grained control of accessible Java packages (as well as associated versions) between bundles.

Starting with Digital Experience Manager 7, new modules must now be written as OSGi bundles. As OSGi is a mature and powerful dynamic module system for Java, it becomes a lot easier to build full-fledged Jahia modules that can interact with each other, while avoiding complex interdependencies that might make maintenance and deployment complex. At the same time, it also leverages the already available OSGi bundles such as the Felix Web Console or the Felix Shell to quickly add functionality to a Digital Experience Manager installation.

In the following table we illustrate how working modules behaves before the introduction of OSGi and after:

  Before OSGi With OSGi
Class or library deployment Requires web app restart  No restart needed
Module is "exploded" on deployment
Quick changes to source files don't require deploment Only works in exploded directory, not module's source
External libraries are deployed into WEB-INF/lib and exposed to all other modules
Undeployment cleans up everything immediately
Modules depending on others cannot be deployed without their dependency
Modules started/stopped after installation

OSGi offers to module developers:

  • true hot deployment: it is now possible to deploy and fully undeploy modules that contain complex dynamic content such as Java code or libraries
  • proper dependency management: the OSGi standard clearly defines the interdependencies between bundles, including bundle versioning, making it possible to use different versions of bundles in different parts of the application
  • isolation: OSGi modules can finely control what they expose and what they use, making sure that no authorized access to resources inside modules are accessed
  • ready-made tools: with little effort it is possible to use existing OSGi tools and libraries in your projects. These tools may offer extensions or new functionality, such as extensions to the Felix Web Console, the Felix Gogo shell, etc…

Figure 1 - Integrated Apache Felix OSGi web console

OSGi is a powerful framework that can scale from embedded systems to large server-side applications, and does require a little understanding of the way the framework works to get started. Fortunately, Digital Experience Manager comes with tooling that makes the transition easy, including Maven plugins that convert existing modules to OSGi bundle projects, or even transforming on-the-fly simple legacy (WAR) Jahia modules to deploy directory onto Digital Experience Manager.

Spring Webflow

Spring Web Flow builds on Spring MVC and allows implementing the "flows" of a web application. A flow encapsulates a sequence of steps that guide a user through the execution of some business task. It spans multiple HTTP requests, has state, deals with transactional data, is reusable, and may be dynamic and long-running in nature.

Figure 2 - Example of a Spring web flow

The sweet spot for Spring Web Flow are stateful web applications with controlled navigation such as checking in for a flight, applying for a loan, shopping cart checkout, or even adding a confirmation step to a form. What these scenarios have in common is one or more of the following traits:

  • There is a clear start and an end point.
  • The user must go through a set of screens in a specific order.
  • The changes are not finalized until the last step.

Once complete it shouldn't be possible to repeat a transaction accidentally.

Digital Experience Manager integrates Spring Webflow as a rendering technology available in modules to make it possible to build complex form handling views. For example, Spring Webflow is used in Digital Experience Manager in the new Server Settings and Site settings administration panels.

Module developers can leverage the power of Spring Webflow to build their own user interfaces that may require multiple step form input, and validate form processing before performing an action.

Webflow are defined through an XML file that defines the application flow. Here below is an example of a simple flow:

<?xml version="1.0" encoding="UTF-8"?>
<flow xmlns="http://www.springframework.org/schema/webflow"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://www.springframework.org/schema/webflow
http://www.springframework.org/schema/webflow/spring-webflow-2.0.xsd">
    <var name="adminPropertiesHandler" class="org.jahia.modules.serversettings.flow.AdminPropertiesHandler"/>
    <on-start>
        <evaluate expression="adminPropertiesHandler.init()" />
        <evaluate expression="adminPropertiesHandler.adminProperties" result="flowScope.adminProperties"/>
    </on-start>
    <view-state id="adminPropertiesForm" model="adminProperties">
        <on-render>
            <evaluate expression="adminPropertiesHandler.userMembership" result="requestScope.userGroups"/>
        </on-render>
        <transition on="submit">
            <evaluate expression="adminPropertiesHandler.save(messageContext)" />
        </transition>
    </view-state>
</flow>

As you can see in the above example, the flow definition file makes it easy to define the different view states as well as the transitions between the views. If you're interested in learning more about Spring Webflow, checkout the resources available here : http://docs.spring.io/spring-webflow/docs/2.3.2.RELEASE/reference/html/ . We also presented the integration of Spring Webflow at JahiaOne 2014. You may find the video here : https://www.youtube.com/watch?v=TUESY3l5XIw&feature=youtu.be

External data providers

Digital Experience Manager External Data Providers allow for the integration of external systems as content providers in the Java Content Repository (JCR) managed by Digital Experience Manager. All external data providers must at least provide access to content (aka "read" external data provider), and may optionally allow for content searching or even allow writing/updating new or existing content. One of the most interesting  feature of External Data Providers is that some of them may provide "enhanceable" content, which means that the raw content they provide can be extended by Digital Experience Manager content (such as for example being able to add comments to an object provided by an External Data Provider). In order to be accessible, external content has to be mapped as nodes inside Digital Experience Manager so the server can manipulate them as regular nodes (edit/copy/paste/reference etc.). This means that the external provider module must provide a CND definition file for each content object type that will be mapped into the JCR back-end.

Here is an example CND definition file from the ModulesDataSource:

[jnt:editableFile] > jnt:file
- sourceCode (string) itemtype = codeEditor
[jnt:cssFile] > jnt:editableFile
[jnt:cssFolder] > jnt:folder
[jnt:javascriptFolder] > jnt:folder
[jnt:javascriptFile] > jnt:editableFile


External Data is made accessible by registering a provider declared in a Spring XML bean definition file that may specify some properties of the provider (such as the mount point, unique key, the data source, etc.)

<bean id="ExternalMappedDatabaseProvider" class="org.jahia.modules.external.ExternalContentStoreProvider" parent="AbstractJCRStoreProvider">
    <property name="key" value="ExternalMappedDatabaseProvider"/>
    <property name="mountPoint" value="/external-database-mapped"/>
    <property name="identifierMappingService" ref="ExternalProviderIdentifierMappingService"/>
    <property name="dataSource">
        <bean class="org.jahia.modules.external.test.db.MappedDatabaseDataSource"/>
    </property>
</bean>

A provider then accesses the underlying data source (implementing the ExternalDataSource and other optional Java interfaces if needed) to read, save or search data. An implementation of the ExternalDataSource interface must also list the node types that it is capable of handling. This can be done programmatically or inside a Spring configuration file. Here is an example of declarative nodeType support from the ModuleDataSource:

<bean id="ModulesDataSourcePrototype" class="org.jahia.modules.external.modules.ModulesDataSource"
      scope="prototype">
    <property name="supportedNodeTypes">
        <set>
            <value>jnt:cssFolder</value>
            <value>jnt:cssFile</value>
            <value>jnt:javascriptFolder</value>
            <value>jnt:javascriptFile</value>
        </set>
    </property>
</bean>

These content are simple content (simple text files where the content of the file is mapped to a simple string (with a specific editor)) As you can see, External Data Providers are a very interesting technology to fully integrate external data with the full power of the Digital Experience Manager CMS features. For example, the eCommerce Factory product is mostly built using an External Data Provider that plugins into an eCommerce back-end.

Workflow

Digital Experience Manager integrates the jBPM 6 workflow engine (http://www.jboss.org/jbpm) that provides support for advanced workflow integrations through the definition of workflows, using the standardized Business Process Model and Notation (BPMN) 2.0 specification. Digital Experience Manager's UI is integrated with the workflow screens so that the experience is seamless for end-users who will never have to leave the UI to perform workflow operations (unless integrators wish to, of course). It is also compatible with any jBPM compatible GUI workflow definition tools such as the web base JBPM Designer (https://www.jboss.org/jbpm/components/designer.html) or the Eclipse plugin (https://www.jboss.org/jbpm/components/eclipse-plugin.html). Please note that these components are additional components that must be installed and deployed by the integrator as they are not part of Digital Experience Manager's core.

JBoss Drools and event listeners

Often, as modifications of content happen, integrators will face the need to perform a specific action when an event occurs. In previous versions of Digital Experience Manager, event listeners could be either written in Java classes or in JSP files, or even Groovy scripts. Starting with version 6.5, it was replaced with a much more powerful and easier to use rule system based on JBoss Drools. An example of such a rule is given below:

rule "Image update"
salience 25
#Rebuild thumbnail for an updated image and update height/width
when
    A file content has been modified
    - the mimetype matches image/.*
then
    Create an image "thumbnail" of size 150
    Create an image "thumbnail2" of size 350
    Set the property j:width of the node with the width of the image
    Set the property j:height of the node with the height of the image
    Log "Image updated " + node.getPath()
end

As you can see rules are similar to the English language formulation and this makes it easy for integrators to use and read. Of course the vocabulary of conditions, making it easy to respond to any event with any action.

File repository

Digital Experience Manager 6 was the first version to include the Java Content Repository as its standard file repository and build services on top of it. Actually, the integration of Jackrabbit is not a strong dependency, as Digital Experience Manager uses the standard JCR API to offer access to multiple repositories. In Digital Experience Manager 6 it was already possible to access CIFS/SMB file repositories. On top of the file repository services, different interfaces expose content through various interfaces such as WebDAV, template file browsing and Digital Experience Manager's AJAX UI. On the other side of the repository, integration with a rules engine is used among other things for image thumbnail generation and metadata extraction. This rule engine is also pluggable and can be extended by integrators to perform other specific logic upon repository events.

Searching and indexing

Digital Experience Manager comes built-in with strong support for searching and indexing, and does so by combining multiple frameworks in order to offer the following features:

  • Full-text searches (Apache Lucene)
  • Multi query languages support (Apache Jackrabbit)
  • Facets (Apache Solr)
  • "Did you mean" (Apache Solr)
  • Open search
  • Connectors to other search repositories such as Alfresco via EntropySoft connectors (available in our Digital Experience Manager Unified Content Hub extension)

In order to use the above features, Digital Experience Manager provides two technologies: full-text search tag libraries and query languages

Full text queries using search tag libraries

The full-text query feature is made available through a set of tags that are focused on searching using basic text matching with content object properties or file contents. It produces results as hits that contain information such as the target object that matched the result, an extract of the matching content and the matching score. It also supports search options defined by the JSR-283 standard such as optional words, mandatory or excluded words, search by sentence or word, etc.

Here is an overview of the features available when using full text queries:

  • Search on all content within a site.
  • Search on multiple sites on the server.
  • Search the contents of files in the content repository.
  • Search the contents of files in external repository (only with the Digital Experience Manager United Content Hub extension).
  • Highlight searched terms in the results page.
  • Order by matching score.
  • Exclusion of matching property (through content definition parameters).
  • Limit results to improve performance.

Full text queries are a great way to offer an easy to use yet powerful search feature on a Digital Experience Manager installation, but they are not very useful to perform more targeted queries, such as retrieving a list of the last 10 news entries or similar queries. This is where the query languages become interesting.

Query languages

The query language feature is actually a native functionality of a JCR-compliant implementation such as Apache Jackrabbit. It offers different query languages that are functionally equivalent, but differ in implementation and usage scenarios. It allows for complex condition matching and result ordering, as well as in some cases joins on multiple content object types. The result of the query is a list of matching nodes, ordered by the specified properties.

Here is an overview of the features offered by this type of querying:

  • Search by content type (news, article, etc.)
  • Complex conditions based on properties, or even in the case of SQL-2 or Query Object Model, joins.
  • Integration with faceted search
  • Integration with front-end HTML caching

The following query languages are available:

  • SQL- 2: Similar to the standard SQL database language so easy to use for developers. The queries are parsed and transformed into Java Query Object Model queries and then executed as such. As this is part of JCR v2, it is relatively new and therefore there are some issues with its implementation, notably on join performance. For most simple queries it will do fine, but for more complex ones it is recommended to use XPath until this language implementation matures.
  • JQOM (Java Query Object Model): this is a Java object representation of a JCR v2 query. It is possible to build these using Digital Experience Manager's provided query tag libraries, or to build them directly from Java code. SQL-2 and JQOM queries are quite similar, except that JQOM avoid the parsing stage, so they are a bit faster. In practice, it is quite seldom that JQOM is used, but it might be interesting in some cases.
  • XPath: although it has been marked as deprecated in JCR v2, it is still available in Apache Jackrabbit and is by far the most optimized query language. It is not as easy to use as SQL-2, but it is very useful to build very fast queries; therefore, often worth the extra effort in designing the query. There are some tricks to know how to search for multi-language content, as it is not handled transparently, in the case of the other two implementations. But even Digital Experience Manager uses it internally for speed in the cases where SQL-2 performance is not fast enough.

Digital Experience Manager also comes built-in with modules that use queries to provide their functionality. An example of this includes the "last news retrieval" feature in the news module. Also available is a generic "query" module that will ask for the query when added to a content page. This makes it easy for editors to be able to build custom queries at content creation time, without requiring any assistance from the integrators (of course this module should not be made available if this possibility is to be restricted).

Authentication and authorization

One of Digital Experience Manager's strengths has always been its powerful authentication and authorization sub-system. It allows for modular yet precise controls of permissions on a wide-variety of objects or actions. Permissions may be very granular or as coarse as desired, which makes it a great tool for deployment in small to large enterprises.

Single sign-on

Digital Experience Manager integrates with the following SSO frameworks:

  • Central Authentication Service (CAS) SSO, http://www.jasig.org/cas
  • Java EE container authentication support
  • Pluggable authentication pipeline that can be easily implemented to add support for more SSO solutions

The last framework is useful in the case of integration with non-standard SSO technologies or custom-built ones. One possible example would be the case of a mobile service provider that uses phone numbers as authentication logins. Interfacing with a custom database will integrate into Digital Experience Manager's back-end, exposing user and group information directly to Digital Experience Manager's UI and permissions.

While it is possible to integrate with Kerberos http://web.mit.edu/kerberos/ (the authentication valve is present in the distribution) this integration is not officially part of the tested and supported stack for Digital Experience Manager.

Please get in touch with the company to know the usage conditions.

Once the user is properly identified, the authorization sub-system is composed of:

  • Access control lists on content objects
  • Roles the user may participate in
  • Permissions on any user actions for a specific role

In order to be able to set access control lists, user and group services are provided, and are of course also pluggable. By default Digital Experience Manager comes with its own user and group provider service, as well as a connector to LDAP repositories, but it is also possible to develop custom services to plugin to either a custom database or a remote service. Digital Experience Manager is also capable of storing properties and user information for external users and groups inside its own services, making it possible to store personalization data in Digital Experience Manager. It should also be noted that all these service implementations are available at the same time, so there is no need to replace one with the other.

Roles and permissions

New to Digital Experience Manager is the introduction of full-fledged roles. Roles are basically a collection of permissions, regrouped under a logical name. For example an "editor" role regroups permissions for editing content and starting workflow processes. Digital Experience Manager comes with default roles built-in, as well as with a powerful UI to modify the default assignments. Integrators may of course define their own roles and permissions, as well as change the default assignments. It is also possible to add permissions in modules and automatically assign them to existing roles upon deployment.

Roles can then be assigned to users and/or groups at any location in the content repository. For example, you may define a role "editor" to a specific group in a specific section of the website. They will be able to act as that role only in that specific location in the content repository, and nowhere else. This makes it easy to delegate responsibilities to collaborate on content editing, reviewing and overall content management. It is of course recommended to re-use roles through the various sites and sections, as a minimal set of roles will be good both for site management and authorization performance (as HTML caching is also using roles to determine which content is viewable or not).

Import/export

Digital Experience Manager's import/export feature is an extremely powerful mechanism for migrating content in various ways between Digital Experience Manager sites, or even between Digital Experience Manager installations. It uses the JSR-170 (or JCR) XML format as a basis for content export, along with other specific files such as file hierarchies for binary data export. All these different files are compressed in a ZIP file that may be used by the import sub-system. This makes it possible to export a complete Digital Experience Manager installation, a set of sites, a single site or even a sub-section of a site using the same import/export technology. Using this feature, users can migrate content between sites, or even between sections of sites, or also use it to export content to non- Digital Experience Manager systems or import from non- Digital Experience Manager systems. The same system is also used for migrations from previous versions of Digital Experience Manager to the newest available version.

Distant publication

Digital Experience Manager may be deployed in multiple instances to cover scenarios where a Digital Experience Manager instance is located inside a firewall, and a public instance is located inside a DMZ zone accessible from the public web. To publish data from the inside to the outside, Digital Experience Manager has a feature called distant publication (also known as remote publication), which makes it easy to automate the process of migrating data from an authoring server to a browsing one. Note that this is still compatible with user-generated content such as a deployed forum on the public instances, meaning that remote publication will not touch user generated content created on the public instance.

Portlets

Digital Experience Manager includes an embedded portal server, which is based on the Apache Pluto reference implementation of the JCR Portlet API specification. The goal of this implementation is to offer support for integrators who need to embed portlets on content pages. This means that any portlet API compliant application may be integrated with Digital Experience Manager in a few simple steps.

Portlet versus modules

To differentiate portlets from modules, we offer the following table that summarizes the differences:

  Portlet Module
Classification Older technology, extension to traditional Web server model using well defined approach based on strict isolation mode. Using newer, loosely defined "Web 2.0" techniques, integration at both server or client level.
Philosophy/Approach Approaches aggregation by splitting role of Web server into two phases: markup generation and aggregation of markup fragments. Uses APIs provided by different modules as well as content server to aggregate and reuse the content.
Content dependencies Aggregates presentation-oriented markup fragments (HTML, WML, VoiceXML, etc.). Can operate on pure content and also on presentation-oriented content (e.g., HTML, JSON, etc.).
Location dependencies Traditionally content aggregation takes place on the server. Content aggregation can take place either on the server-side or on the client-side, but usually happens on the server.
Aggregation style "Salad bar" style: Aggregated content is presented 'side-by-side' without overlaps. "Melting Pot" style: Individual content may be combined in any manner, resulting in arbitrarily structured hybrid rendering and editing.
Event model Read and update event models are defined through a specific portlet API. CRUD operations are based on JCR architectural principles, and on the client REST interfaces allow content interactions.
Relevant standards Portlet behavior is governed by standards JSR 168, JSR 286 and WSRP, although portal page layout and portal functionality are undefined and vendor-specific. Base standards are JCR API, REST, JSON and XML. Defacto standards include JQuery as a Javascript framework.
Portability Portlets developed with the portlet API are in theory portable to any portlet container. Modules are Digital Experience Manager specific.
Repositories Portlet repositories have been a pipe dream for a long time, but despite multiple efforts they have never taken off and they stay usually specific to a portlet container implementation. . Modules are available on Digital Experience Manager's Private App Store, developers and integrators are encouraged and free to post them there, or anywhere else they wish.
Performance A page will be fully dependent of the rendering speed of each portlet to achieve good performance, which may be difficult if closed source portlets are present in the system. Modules have built-in support for page caching if they re-use Digital Experience Manager -stored content, which is generally the case.

In general, integrators looking for a powerful and rapid integration solutions will probably want to use modules. The main use case for portlet usage is the integration of legacy applications that are only available as portlets. In the case of other systems (servlets, web services) it is preferred to use modules as the integration will be more robust, easier to re-use and deploy and to maintain, of course.