OSGi tooling in DX

  Written by The Jahia Team
 
Developers
   Estimated reading time:

1 Jahia Maven Plugin

The Jahia Maven plugin has a few new goals to help you with your OSGi module projects. Here is a quick overview:

  • jahia:dependencies : this goal will parse your project and its dependencies to generate a realistic list of imports for your module. It does this by scanning a lot of different resources to produce a list of package imports as well as content definition export and dependencies.
  • jahia:find-package-uses : will use the BND tool from Peter Kriens internally to figure out where the package import came from, specifically which class coming from which Maven dependency that is generating this import. You could then use a tool such as JD-GUI (http://java.decompiler.free.fr/?q=jdgui) to open the JAR and decompile the class that is referencing the package to understand its uses, and if it is mandatory or not. One thing you could do once you understand the package use is marking it as an optional OSGi resolution.
  • jahia:find-packages : will scan all the projects dependencies, including optional and provided ones, to find a package. So you can use this if you suspect the package must be provided by a JAR in the project’s dependencies but have trouble finding it (for example because it is optional).
  • jahia:osgi-inspect : will output a nicely formatted and easy to read view of the MANIFEST.MF headers and optionally also the packages contained inside a JAR (it doesn’t even have to be an OSGi bundle).

1.1 jahia:dependencies goal

The jahia:dependencies goal helps you build the following OSGi MANIFEST headers:

  • Import-Package (list of packages required by the OSGi bundle)
  • Export-Package (list of packages exported by the OSGi bundle
  • Provide-Capability (list of capabilities provided by the OSGi bundle)
  • Require-Capability (list of capabilities required by the OSGi bundle)

It is capable of scanning a lot of different resource types, to complement the class scanning that the Felix Maven Bundle Plugin already does, notably:

  • .jsp files (both page import and taglib dependencies)
  • .tld (tag library definition) files
  • .cnd (content node definition) files
  • .drl (Drools Rule) files
  • .jbpm.xml (JBPM Workflow definition) files
  • Spring context files
  • Jackrabbit XML import files

The following graph explains how the jahia:dependencies goal integrates with the Felix Bundle Maven plugin to generate extended package imports.

Build steps:

  1. The Jahia Maven plugin jahia:dependencies goal scans the project source code to detect all package references inside resources that are not supported by the Felix Bundle Maven Plugin class scanning
  2. It generates a list of packages that is then made available to the Maven runtime in the following variable: ${jahia.plugin.projectPackageImport}
  3. The Felix Bundle Maven plugin is configured to generate an OSGi bundle using its own configuration and also references the variable generated by the Jahia Maven plugin goal to generate the final OSGi manifest headers.

1.1.1 Package scanning

Most of the work the plugin does is scan different resource types to see which packages are used. For example in the case of a JSP file it will scan the directives at the beginning of the file to see if any page import or taglibs are used. In the case of a page import it will simply retrieve the list of packages, but in the case of a tag library it will retrieve the corresponding TLD file, and scan its content to find the packages used inside the TLD file. This makes it easy for integrators to make sure they are importing the proper packages even when they use tag libraries, the plugin does all the work behind the scenes to make sure that the proper imports are generated. Also, it will also check for inconsistencies such as a missing dependency if a tag library is used in a JSP but missing from the project’s Maven dependencies.

Once all the package scanning is completed, the plugin sets the following Maven project property: ${jahia.plugin.projectPackageImport}. This property contains a list of all the packages found in the project, and can then be used as import to the Import-Package instruction of the Maven Bundle plugin as in the example below:

<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <extensions>true</extensions>
    <configuration>
        <instructions>
            <Bundle-Name>${project.name}</Bundle-Name>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Bundle-Category>jahia-module</Bundle-Category>

            <Implementation-Title>${project.name}</Implementation-Title>
            <Implementation-Version>${project.version}</Implementation-Version>
            <Implementation-Vendor>${project.organization.name}</Implementation-Vendor>
            <Implementation-URL>${project.organization.url}</Implementation-URL>
            <Specification-Title>${project.name}</Specification-Title>
            <Specification-Version>${project.version}</Specification-Version>
            <Specification-Vendor>${project.organization.name}</Specification-Vendor>

            <!-- Jahia manifest attributes -->
            <Jahia-Depends>default</Jahia-Depends>
            <Jahia-Module-Type>module</Jahia-Module-Type>
            <Jahia-Root-Folder>${project.artifactId}</Jahia-Root-Folder>
            <Jahia-Source-Folders>${project.basedir}</Jahia-Source-Folders>
            <Jahia-Static-Resources>/css,/icons,/images,/img,/javascript</Jahia-Static-Resources>

            <Export-Package></Export-Package>
            <Import-Package>*,${jahia.plugin.projectPackageImport}</Import-Package>
            <Embed-Dependency>*; scope=compile; type=!pom;
                inline=true</Embed-Dependency>
            <Embed-Transitive>true</Embed-Transitive>
            <_removeheaders>
                Include-Resource,
                Private-Package,
                Embed-Dependency,
                Embed-Transitive
            </_removeheaders>
        </instructions>
        <archive>
            <addMavenDescriptor>false</addMavenDescriptor>
        </archive>
    </configuration>
</plugin>

1.1.2 Content definition scanning

Much in the same way that we scan a project for package dependencies, the jahia:dependencies goal always scans all the resources for content node type definitions and references. Specifically it scans the following file types:

  • *.cnd : content definition files. Here it actually parses the files to extract all the new node type definitions as well as all the node types used either as super types, child node types, or even property types that can use a node type in their values.
  • JCR import files (*.xml) : the scanner uses the following XPath queries to retrieve content node type references : //@jcr:primaryType and //@jcr:mixinTypes
  • The result of the scanning is the stored in Maven project properties:
  • ${jahia.plugin.providedNodeTypes}: a list of all content node type definitions defined in the project, formatted in OSGi Provide-Capability format.
  • ${jahia.plugin.requiredNodeTypes}: a list of all required content node type definitions found in the project, formatted in OSGi Require-Capability format.

Here is an example of what these properties look like when generated for a Digital Experience Manager module:

Provide-Capability: com.jahia.services.content; nodetypes:List<String>="jmix:retrievableContent,jnt:contentRetrieval"
Require-Capability: com.jahia.services.content; filter:="(nodetypes=jmix:basicContent)",com.jahia.services.content; filter:="(nodetypes=jmix:cache)",com.jahia.services.content; filter:="(nodetypes=jmix:editorialContent)",com.jahia.services.content; filter:="(nodetypes=jmix:list)",com.jahia.services.content; filter:="(nodetypes=jmix:queryContent)",com.jahia.services.content; filter:="(nodetypes=jmix:renderableList)",com.jahia.services.content; filter:="(nodetypes=jnt:content)",com.jahia.services.content; filter:="(nodetypes=jnt:page)",com.jahia.services.content; filter:="(nodetypes=mix:title)"

In the above example we see that the project defined the new node type definitions jmix:retrievableContent and jnt:contentRetrieval and that it needs content node definitions such as jmix:basicContent in order to work correctly. If these requirements are not fulfilled the module will not be able to start.

We can therefore integrate the capability generation with the Maven Bundle plugin by expanding on the example provided previously in the package scanning example, as seen here:

<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <extensions>true</extensions>
    <configuration>
        <instructions>
            <Bundle-Name>${project.name}</Bundle-Name>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Bundle-Category>jahia-module</Bundle-Category>

            <Implementation-Title>${project.name}</Implementation-Title>
            <Implementation-Version>${project.version}</Implementation-Version>
            <Implementation-Vendor>${project.organization.name}</Implementation-Vendor>
            <Implementation-URL>${project.organization.url}</Implementation-URL>
            <Specification-Title>${project.name}</Specification-Title>
            <Specification-Version>${project.version}</Specification-Version>
            <Specification-Vendor>${project.organization.name}</Specification-Vendor>

            <!-- Jahia manifest attributes -->
            <Jahia-Depends>default</Jahia-Depends>
            <Jahia-Module-Type>module</Jahia-Module-Type>
            <Jahia-Root-Folder>${project.artifactId}</Jahia-Root-Folder>
            <Jahia-Source-Folders>${project.basedir}</Jahia-Source-Folders>
            <Jahia-Static-Resources>/css,/icons,/images,/img,/javascript</Jahia-Static-Resources>

            <Export-Package></Export-Package>
            <Import-Package>*,${jahia.plugin.projectPackageImport}</Import-Package>
            <Provide-Capability>${jahia.plugin.providedNodeTypes}</Provide-Capability>
            <Require-Capability>${jahia.plugin.requiredNodeTypes}</Require-Capability>
            <Embed-Dependency>*; scope=compile; type=!pom;
                inline=true</Embed-Dependency>
            <Embed-Transitive>true</Embed-Transitive>
            <removeheaders>
                Include-Resource,
                Private-Package,
                Embed-Dependency,
                Embed-Transitive
            </removeheaders>
        </instructions>
        <archive>
            <addMavenDescriptor>false</addMavenDescriptor>
        </archive>
    </configuration>
</plugin>

1.1.3 Troubleshooting dependencies

Sometimes understanding why a dependency was generated can be tedious, so fortunately launching the jahia:dependencies goal in debug mode will generate a lot of interesting logging information on where it has extracted a dependency. You can activate the debug mode simply by using the -X option on the command line as in the following example:

mvn -X jahia:dependencies

1.1.4 Deactivating/controlling content definition scanning

If for some reason you need to control the content definition scanning, there are three possibilities:

  • artifactExcludes property: deactivate it for specific files by using the artifactExcludes configuration option
<artifactExcludes>
    <exclude>org.jahia.modules:*</exclude>
    <exclude>org.jahia.templates:*</exclude>
    <exclude>org.jahia.test:*</exclude>
    <exclude>*.jahia.modules</exclude>
</artifactExcludes>

scanDirectories property: using the scanDirectory you can specify a list of directories to scan. The default value is:

scanDirectories.add(project.getBasedir() + "/src/main/resources");
scanDirectories.add(project.getBasedir() + "/src/main/import");
scanDirectories.add(project.getBasedir() + "/src/main/webapp");
  • excludeFromDirectoryScan : you can specify which files to exclude from the directory scan
<excludeFromDirectoryScan>
    <exclude>imports/import.xml</exclude>
    <exclude>imports/importIndexOptionNodes.xml</exclude>
</excludeFromDirectoryScan>

1.2 jahia:find-package-uses goal

Our Jahia Maven plugin has new tools to help with resolving dependency problems when generating bundles using the Maven bundle plugin. Often this plugin will generate imports for packages that you might not know where they are coming from. It is usually due to some third-party library that has a dependency on a package either by importing it, or by using it in a Class.forName() reflection API call (BND actually finds these).

Here is an example with our Digital Experience Manager test module project (jahia/test/jahia-test-module). We noticed on deployment that it was generating two strange imports: kaffe.util and weblogic. We can use the new find-packages-uses goal to figure out where they are used:

mvn jahia:find-package-uses -DpackageNames=weblogic,kaffe.util

This will generate the following result:

[INFO] =================================================================================
[INFO] SEARCH RESULTS
[INFO] ---------------------------------------------------------------------------------
[INFO] Found package weblogic used in class org.apache.tools.ant.taskdefs.rmic.WLRmic from trail org.jahia.test:jahia-test-module:bundle:7.0.0.0-SNAPSHOT -> org.apache.ant:ant:jar:1.8.2
[INFO] Found package kaffe.util used in class org.apache.tools.ant.util.JavaEnvUtils from trail org.jahia.test:jahia-test-module:bundle:7.0.0.0-SNAPSHOT -> org.apache.ant:ant:jar:1.8.2

We can then go into the JARs for these two dependencies and decompile the classes using JD-GUI to understand the usage. We can then mark the dependencies as optional in the Maven Bundle Plugin configuration as in the following example:

<plugin>
    <groupId>org.apache.felix</groupId>
    <artifactId>maven-bundle-plugin</artifactId>
    <extensions>true</extensions>
    <configuration>
        <instructions>
            <Bundle-Name>${project.name}</Bundle-Name>
            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>
            <Bundle-Category>jahia-module</Bundle-Category>

            <Implementation-Title>${project.name}</Implementation-Title>
            <Implementation-Version>${project.version}</Implementation-Version>
            <Implementation-Vendor>${project.organization.name}</Implementation-Vendor>
            <Implementation-URL>${project.organization.url}</Implementation-URL>
            <Specification-Title>${project.name}</Specification-Title>
            <Specification-Version>${project.version}</Specification-Version>
            <Specification-Vendor>${project.organization.name}</Specification-Vendor>

            <!-- Jahia manifest attributes -->
            <Jahia-Depends>default</Jahia-Depends>
            <Jahia-Deploy-On-Site>all</Jahia-Deploy-On-Site>
            <Jahia-Module-Type>system</Jahia-Module-Type>
            <Jahia-Root-Folder>${project.artifactId}</Jahia-Root-Folder>
            <Jahia-Source-Folders>${project.basedir}</Jahia-Source-Folders>
            <Jahia-Static-Resources>/css,/icons,/images,/img,/javascript</Jahia-Static-Resources>

            <Import-Package>*,
                ...
                kaffe.util;resolution:=optional,
                weblogic;resolution:=optional,
                ...
                ${jahia.modules.importPackage}</Import-Package>
            <Embed-Dependency>*;scope=compile;type=!pom;inline=false</Embed-Dependency>
            <Export-Package>!org.jahia.services.*,org.jahia.test.*,junit.*,org.junit.*</Export-Package>
            <Embed-Transitive>true</Embed-Transitive>
            <_removeheaders>
                Include-Resource,
                Private-Package,
                Embed-Dependency,
                Embed-Transitive
            </_removeheaders>
        </instructions>
    </configuration>
</plugin>

1.3 jahia:find-packages goal

We could use the findPackages goal if we suspected that a dependency included some specific packages, as in the following example:

mvn jahia:find-packages -DpackageNames=weblogic,kaffe.util

This generates the following result:

[INFO] =================================================================================
[INFO] SEARCH RESULTS
[INFO] ---------------------------------------------------------------------------------
[WARNING] Couldn't find weblogic anywhere !
[WARNING] Couldn't find kaffe.util anywhere !

This confirms we were right in marking the dependencies as optional, as we don’t have them in our project dependencies but they might be provided at runtime by another OSGi bundles, but they are also not required at runtime.

1.4 jahia:osgi-inspect goal

This new goal makes easy to dump the headers of an OSGi bundle JAR file. It will also work with normal JARs though, so it can be useful to check if the headers were properly generated. By default, it will look for the project’s generated artifact (must have been previously generated). Here is an example that will print out the contents of the project’s artifact JAR:

mvn jahia:osgi-inspect

If you prefer to pass a parameter to specify which JARs should be inspected, you can simply use the jarBundles parameter as in the following example:

mvn jahia:osgi-inspect -DjarBundles=target/project-1.0-SNAPSHOT.jar,target/project-1.0-SNAPSHOT-sources.jar

This will print out the headers for both JAR files.

2 Felix Maven Bundle Plugin

(Note: the following section is a reproduction of the online Felix Maven Bundle plugin documentation, available in full here: http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html)

This plugin for Maven 2 is based on the BND tool from Peter Kriens. The way BND works is by treating your project as a big collection of classes (e.g., project code, dependencies, and the class path). The way you create a bundle with BND is to tell it the content of the bundle's JAR file as a subset of the available classes. This plugin wraps BND to make it work specifically with the Maven 2 project structure and to provide it with reasonable default behavior for Maven 2 projects.

2.1 More information about the plugin

You can find more documentation and information about the plugin at the official site: http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html

As well as the Maven standard plugin documentation available here:

http://svn.apache.org/repos/asf/felix/releases/maven-bundle-plugin-2.3.7/doc/site/index.html

3 The Manage Modules administration UI

The "Manage module" server administration UI in Digital Experience Manager is a user-friendly yet powerful user interface to manage the modules deployment, state and view more details about its dependencies and other meta-information. We won’t go into many details here as this UI is covered in other Digital Experience Manager documents such as the administrator’s guide or the templating development guide.

4 The Apache Felix Web Console

Screen Shot 2016-12-13 at 11.29.26.png

The Apache Felix Web Console is an OSGi tool that is integrated into Digital Experience Manager. To access it, open a browser at the URL: http://localhost:8080/tools, enter the requested login information and then click on "OSGi console".

The Web Console is a powerful tool to see the internals of the Digital Experience Manager OSGi framework. As this is an integrated external tool, to learn more please go to the Apache Felix Web Console project website: http://felix.apache.org/site/apache-felix-web-console.html

5 The Apache Felix Gogo shell

Integrated directly into Digital Experience Manager but disabled by default for security reasons, the Felix Gogo command line shell is incredibly useful to diagnose and query the OSGi framework in the cases where the Web Console might not yet be available (during Digital Experience Manager startup for example), or simply when users prefer a command line interface.

5.1 Activating the shell

To activate the shell, while the Digital Experience Manager server is not running, open up the Digital Experience Manager configuration file located at WEB-INF/etc/config/jahia.properties, uncomment the following line and set its value to something like this:

######################################################################
### OSGi settings ####################################################
######################################################################
# The following setting is used to change the port on which the
# Apache Felix OSGi command line shell will listen for telnet
# connections. If it is set to -1 it will be deactivated (default).
# The usual port number is 2019.
felix.gogo.shell.telnet.port = 2019

You can then start the server.

5.2 Accessing the shell

The Felix Gogo shell is accessible via TELNET but only on the localhost interface. So to connect make sure you are on the same server as the Digital Experience Manager server and type in (you might have to install the telnet command that is no longer installed by default on some UNIX systems):

telnet localhost 2019

You should be greeted with a screen that looks something like this:

Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
____________________________
Welcome to Apache Felix Gogo
g!

You can then type in command such as:

jahia:modules

This will generate an output that looks something like this:

Module State: Started
----------------------------------------
40 : translateworkflow v2.0.0-SNAPSHOT depends on [default]
41 : calendar v2.0.0-SNAPSHOT depends on [default]
42 : visibility v7.0.0.0-SNAPSHOT depends on [default]
43 : contact v2.0.0-SNAPSHOT depends on [default]
44 : person v2.0.0-SNAPSHOT depends on [default]
45 : formbuilder v2.0.0-SNAPSHOT depends on [default, grid]

The jahia:modules command gives precise state information on the deployed Digital Experience Manager modules.

Other useful commands are quickly listed here :

  • help : lists all commands
  • felix:lb will list all the bundles and their state, including their ID
  • felix:start XX will start bundle with ID XX
  • felix:stop XX will stop bundle with ID XX
  • felix:uninstall XX will uninstall bundle with ID XX
  • exit 0 will shutdown Digital Experience Manager with exit code 0

As you can imagine, any OSGi bundle can extend the Felix Gogo Shell, and more commands are available by deploying additional OSGi bundles. You can find more information about the Felix Gogo shell on the official website: http://felix.apache.org/site/apache-felix-gogo.html.

Of interest also is the Felix Gogo shell extension documentation, available here : http://felix.apache.org/site/61-extending-the-console.html