Connector Bundles

Owned by Francesco Chicchiriccò
Last updated: Dec 28, 2012Version comment
6 min read

A bundle is nothing more than a specialized JAR. There are special attributes within the JAR's manifest that are recognized by the connectors framework. This document describes the plan for connector bundles and classloading strategy.

The goals are as follows:

  1. Eliminate classloading conflicts.
    Connectors are the point of integration with third-party systems. This inherently introduces a number of dependencies on third-party jars. As the number of dependencies increase, the chance of a class conflict increases as well.
  2. Support different versions of the same connector from a single provisioner.
    Often you will have different versions of the same resource that need to be managed from the same provisioner. In these cases, you will typically have multiple versions of the same third-party class and they need to reside in the same JVM.
  3. Better patch model for delivering hotfixes.
    By their very nature, connectors are one of the most error prone areas of the system, requiring more hotfixes than any other component. It is essential that we provide a way to deliver a patch without impacting other components in the system or other connectors.

For these reasons, each connector will have its own classloader. Bundles are the deployable unit for a connector that contain the connector class and dependencies, thus defining the contents of a connector's classloader.

Inside the Bundle JAR

Once built, the bundle becomes a self-contained JAR, with internals as follows:

  • META-INF/MANIFEST.MF (file)
    This is the manifest. Here is a list of the core bundle attributes (used by all bundles, regardless of whether they declare any connectors):
    • ConnectorBundle-FrameworkVersion: 
      The minimum version of the framework required by the connector. If the version is greater than the version of the deployment, an error will be thrown.
    • ConnectorBundle-!Name: 
      Qualified name for the connector bundle. Within a given deployment, the pair (ConnectorBundle-Name,ConnectorBundle-Version) must be unique.
    • ConnectorBundle-Version: 
      The version of the bundle. Within a given deployment, the pair (ConnectorBundle-Name,ConnectorBundle-Version) must be unique.
  • lib/ (directory)

    The libraries (or bundles) distributed with the connector. These are automatically added to the classpath of the bundle at runtime. (This is a non-standard JAR feature, provided by the connectors class loader).

  • (files)
    Source files, classes, and Java resources. (These are standard JAR features). These are intentionally kept as top-level entities to allow you to compile downstream dependencies against the classes within a bundle by simply adding the bundle to your build's classpath. This is to facilitate extending a connector (see "Extending a Connector" below).

 

Connector Registration

The connector bundle manifest is used merely for the purpose of establishing the classpath and class loader for the bundle. All other meta-data is controlled via Java annotations defined by the Connectors Framework. In order to register a connector, its class must be present in a connector bundle and must have the annotation org.identityconnectors.framework.spi.ConnectorClass. As an implementation note, the Connector Framework scans the top level .class files in a connector bundle, looking for classes that have the @ConnectorClass annotation, thus auto-discovering connectors which are defined in the bundle.

Class Loading

Each bundle has its own class loader which resolves classes in the following order:

  1. Classes in the bundle
  2. Classes in JARs in the bundle's lib directory, in alphabetical order.
    Note that the JARS in the bundle's lib directory may themselves be bundles which have their own lib directory. These will be recursively processed and added to the classpath in depth-first-search order.
  3. Classes in the framework's classloader (and parent class loaders).

 

As an implementation detail, each bundle will get a BundleClassLoader. A BundleClassLoader is a (subclass of) !URLClassLoader where the URLs are the top-level directory of the bundle, the jars in its lib folder, and the same for each of the embedded bundles. The BundleClassLoader will have, as a parent class loader, the ClassLoader that loaded the connectors framework. The BundleClassLoader will override the appropriate methods such that child class loading happens before parent. In addition, when fetching a class from the parent class loader, it will restrict it such that the class cannot access any internal framework classes.

ContextClassLoader

The context class loader (Thread.currentThread().getContextClassLoader()) must be set such that during any invocation to a connector method, the connector's class loader is the context class loader. It will be the responsibility of the ConnectorFacade to arrange for this. At the beginning of any invocation to a connector method it should be set and at the end, it should be restored to its previous value.

Deployment in a WAR

When deploying in a war, the connector framework jar(s) should be placed in WEB-INF/lib. The bundles themselves (also .jars) should be placed in a "/bundles" directory at the root of the web application.

The bundles will be autodiscovered there and the connector API will provide a way to list the bundles, create connectors from the bundles, etc.

Extending a Connector

One of the things we allow for is the ability to extend (subclass) a given connector. The way to do this is to add the extended bundle as a dependency of a new bundle and create a class which subclasses the target class.

The DatabaseConnector is a great example of this. The common logic would be in a common bundle as follows:

  • META-INF/MANIFEST.MF
    • ConnectorBundle-FrameworkVersion: 1.0
    • ConnectorBundle-Name: org.identityconnectors.database.common
    • ConnectorBundle-Version: 1.0
  • org.identityconnectors/database/common/AbstractDatabaseConnector.class
    NOTE: this connector would not have a @ConnectorClass annotation.
  • org/identityconnectors/database/common/ (other common source files)
  • lib/

 

Then there would be as many database (resource) specific bundles as necessary, for example:

  • META-INF/MANIFEST.MF
    • ConnectorBundle-FrameworkVersion: 1.0
    • ConnectorBundle-Name: org.identityconnectors.database.mysql
    • ConnectorBundle-Version: 1.0
  • org/identityconnectors/database/mysql/!MySQLConnector.class (subclass of AbstractDatabaseConnector)
    NOTE: this connector would have a @ConnectorClass annotation.
  • org/identityconnectors/database/mysql/ (other !MySQL source files)
  • lib/org.identityconnectors.database.common-1.0.jar (this is the bundle described above)
  • lib/ (any specific database drivers and libraries as needed)

 

Distribution of Connectors

As much as possible, connectors will be distributed in the form of connector bundles. However there are cases where we may need to remove third-party JARs from the bundles we distribute. For these cases, upon deployment, you will need to re-jar the distributed bundle with the required dependencies.

Handling Multiple Resource Versions

A connector should document which versions of the target resource it supports, and these are expected to evolve and mature over time. For the sake of this example, we will use the fictional vendor Humanitrobe. Let's say they have two major product versions to support, Humanitrobe 8.x and their new release, Humanitrobe 9.x. Let's also assume that we do not have the right to distribute the humanitrobe8x.jar or the humanitrobe9x.jar. We already have a HumanitrobeConnector v1.0 in place which supports Humanitrobe 8.x. Regardless of whether their API has changed from release to release, when we have a new version of the same resource, the obvious and intended solution is to produce a new version of the same Connector. In this case we would distribute two separate bundles:

  • META-INF/MANIFEST.MF
    • ConnectorBundle-FrameworkVersion: 1.0
    • ConnectorBundle-Name: org.identityconnectors.humanitrobe
    • ConnectorBundle-Version: 1.0 #supports 8.x
  • org/identityconnectors/humanitrobe/HumanitrobeConnector.class (and other sources)
  • lib/ (directory is empty for distro)

 

and
  • META-INF/MANIFEST.MF
    • ConnectorBundle-FrameworkVersion: 1.0
    • ConnectorBundle-Name: org.identityconnectors.humanitrobe
    • ConnectorBundle-Version: 1.1 #supports 9.x
  • org/identityconnectors/humanitrobe/HumanitrobeConnector.class (and other sources)
  • lib/ (directory is empty for distro)

Upon deployment these bundles would be re-packaged and the humanitrobe[8x/9x].jar would be added to the lib directories, accordingly.

On a side note, keep in mind that a new version of a Connector may require a new version of the framework, but a new version of the framework will never require newer versions of individual Connectors. Thus, framework upgrades should be harmless to existing deployments.

Patching Connectors

One of the goals of connectors is to provide an easier patch process. In general, when a customer requests a patch it is for a specific connector, rather than a fix to a shared component. If the fix is to a shared component, they would rather that not impact other connectors in the system (unless they explicitly ask for it). For this case the choice has traditionally been either a)copy and paste the common component into the patch or b)change the common component and hope for the best. This is solved in Connectors by the fact that each connector bundle is completely self-contained. A bundle and all of its dependencies are a single unit.

This functionality becomes even more important in the future once the framework is relatively mature and stable. The end goal is to allow customers to grab connectors off of the trunk and drop them into their deployment from a prior release (see above, "Handling Multiple Resource Versions"). As long as the ConnectorBundle-FrameworkVersion of the connector on the trunk is compatible with what the customer has in their deployment, they should be able to do this. In this case, you can see it is critical that bundles are self contained.

Build Notes

From a build standpoint, one thing we have done is structure our bundle manifests so that the version is automatically populated. For each bundle, there is a bundle.properties (see above "Bundle Structure"). The attribute ConnectorBundle-Version will be left out since it will be determined by a global setting passed in by the build machine. The attribute ConnectorBundle-FrameworkVersion will be left hard-coded for now. We should rarely bump this up and, when we do, we need to do so in a controlled fashion.

 

Also, as implied in the previous section, the names of our bundle.jars must include the full version number.