Monday, August 29, 2011

java.util.ServiceLoader in OSGi

One of the bigger stumbling blocks for migrating an existing application to OSGi has been java.util.ServiceLoader and its predecessors (the various FactoryFinders that can be found in technologies such as JAXP, JAXRS, etc...).

ServiceLoader is a class provided by the JRE aimed at loading pluggable implementations of something. For those who need a little refreshing of the brain, here's what typical ServiceLoader usage looks like... Let's say you have an SPI called MySPI with a single method doit():
public abstract class MySPI {
    public abstract String doit();
}

For some reason SPI providers are often realized as abstract classes, so that's what I'm doing here too. Obviously using an interface is also possible...
You load the SPI implementations by calling ServiceLoader.load(). To obtain the actual implementation objects you iterate over the result of that call:
ServiceLoader ldr = ServiceLoader.load(MySPI.class);
for (MySPI spiObject : ldr) {
  spiObject.doit(); // invoke my SPI object
}
So why is this such a big problem in OSGi? The problem is that the JRE's ServiceLoader is non-modular by design. It relies on a single classloader that has visibility over everything in your system that can possibly contain an implemetation of the service requested. The load() method as used above uses the ThreadContextClassLoader (TCCL) to find all the ServiceLoader implementations in the system. There are a number of issues with this from an OSGi perspective.
  1. The value of the TCCL is generally not defined in an OSGi context.
  2. The implementations are found by looking for a resource in META-INF/services. Every jar that contains an implementation of a service (e.g. my org.acme.MySPI implementation) should advertise this by putting a META-INF/service/org.acme.MySPI file in its jar file.
  3. Once the above resources have been found, the contents of them are parsed to obtain the class names of implementations, normally these classes reside in an implementation package, something like org.acme.impl.MySPIImpl... ServiceLoader will instantiate the classes found using their no-arg constructor.

Issue 2 clearly poses an issue if this resource is to be obtained through the classloading rules set by OSGi as OSGi wires a package import only to precisely one exporter. This gives great predictability (you know exactly where you're getting a package from) but it doesn't work if you need to get the same resource in the same package from multiple bundles. And even if you were able to get all the META-INF/services resources in your system you would still have the issue that the classes listed in there are generally private implementation classes that you don't want to export outside of a bundle. To load and instantiate them from a consuming bundle you'd also have to import these private implementation packages which would be totally horrible (issue 3).

So how should we do this in OSGi? Clearly the best answer is: use the OSGi service registry. It doesn't suffer from any of the problems ServiceLoader has: no need to expose and import private packges, no need to expose in a single package across multiple bundles. Plus the OSGi Service Registry supports a dynamic service lifecycle: services can come can go, additional suitable services can appear, existing services can get replaced. Furthermore, the OSGi ServiceRegistry supports a rich language of selecting services among possible candidates by using LDAP filters that work over service properties, while ServiceLoader simply gives you a snapshot of all the known service implementations at a particular point in time.

However - you might be looking at some code that you'd like to use as-is. You might not have the luxury of using the OSGi Service Registry right now. Maybe you don't have the source code so you can't modify it or maybe you're just focusing on other things and don't want to modify the code. You just like to use the same code that works with ServiceLoader outside of OSGi, in OSGi...

For this use-case I've started looking at a solution. The solution is based on the premise that you don't want or can't modify the code that uses ServiceLoader. Since this code is almost certainly non-OSGi you will have to turn it into an OSGi bundle by either wrapping it (put the original jar unmodified inside an OSGi wrapper bundle) or by modifying the MANIFEST.MF file to add things like Bundle-SymbolicName and Import-Package statements. Since you're modifying this file anyway, it's not a big deal to add an extra header in there to identify the bundle as a ServiceLoader-user. The header is called SPI-Consumer. The simplest form is:
  SPI-Consumer: *
Which means that this bundle is identified as an SPI consumer with no restrictions. This means that when code such as

  ServiceLoader ldr = ServiceLoader.load(MySPIProvider.class);
is encountered, something special will have to be done. What needs to be done is that the TCCL is set to the bundle class loader of the bundle that contains the implementation of MySPIProvider for the duration of the load() call. It needs to be the classloader of the bundle itself, so it has visibility of the private packages which are needed for ServiceLoader to do its business.

In my implementation, which is available in the Apache Aries project, the consumer byte-code is slightly modified so that for the duration of the ServiceLoader.load() call the TCCL is set to the classloader of the providing bundle(s).

How do we know that a bundle is providing an implementation for use with java.util.ServiceLoader? Similar to the consumer side, this code was most likely not written as an OSGi bundle, so it either needs to be wrapped or its MANIFEST.MF needs to be enhanced with Bundle-SymbolicName etc. Here we can identify the bundle as an SPI provider by adding:
  SPI-Provider: *
This means that all the implementations found in META-INF/services are considered as SPI providers. (You can also provide a subset by enumerating all the SPI names provided, instead of specifying *).

Specifying this header does two things:
  1. It marks the bundle as an SPI provider for use with bundles that are marked as SPI consumer.
  2. It registers the implementations found in META-INF/services in the OSGi Service Registry so that they can be used from there too.
Try it out!
Ok - let's try it out. For that you need:
  • An OSGi Framework - I'm going to use JBoss AS7.
  • An SPI Consumer, Provider and the SPI itself.
  • The Aries SPI-Fly project which implements the functionality associated with the SPI-Consumer and SPI-Provider Manifest headers.
Start the OSGi Framework
I'm taking JBoss AS7 which contains a fully compliant OSGi 4.2 Core Framework. Get version 7.0.1 from here: http://www.jboss.org/jbossas/downloads/
Download and unzip it, then run bin/standalone.sh (or standalone.bat):
...
11:49:45,016 INFO  [org.jboss.as.deployment] (MSC service thread 1-12) Started FileSystemDeploymentService for directory /Users/davidb/jboss/jboss-as-web-7.0.1.Final/standalone/deployments
11:49:45,028 INFO  [org.jboss.as] (Controller Boot Thread) JBoss AS 7.0.1.Final "Zap" started in 1671ms - Started 93 of 148 services (55 services are passive or on-demand)

BTW the AS7 appserver started up in 1671ms on my plain mac!

Get some example bundles
I'm going to use some of the example bundles from the Aries SPI-Fly project...

You need a SPI Consumer. For instance this one that uses the SPI from the Bundle Activator:
public class Activator implements BundleActivator {
    public void start(BundleContext context) throws Exception {
        ServiceLoader ldr = ServiceLoader.load(SPIProvider.class);
        for (SPIProvider spiObject : ldr) {
            System.out.println(spiObject.doit()); // invoke the SPI object
        }
    }

    public void stop(BundleContext context) throws Exception {}
}

I have a separate bundle that contains the SPI itself:
public abstract class SPIProvider {
    public abstract String doit();
}

I used an abstract class here instead of an interface as that seems to be common practise with SPIs used with ServiceLoader.

And I have an SPI Provider bundle with an implementation:
package org.apache.aries.spifly.mysvc.impl2;
import org.apache.aries.spifly.mysvc.SPIProvider;

public class SPIProviderImpl extends SPIProvider {
    public String doit() {
        return "Doing it too!";
    }
}

The provider bundle has the SPI-Provider: * Manifest header in it and the consumer bundle has the SPI-Consumer: * header. 
As the SPI-Fly project hasn't been released yet, these bundles can't be downloaded from Maven yet. You can build them yourself from here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples
You can also download them from these locations where I put my built artifacts:
Add OSGi-ServiceLoader support
The Apache Aries SPI Fly component provides two ways to add ServiceLoader support to an OSGi Framework:
  1. Dynamically by using the OSGi 4.3 Weaving Hooks. With this you can install the above bundles as-is in the OSGi Framework and the treatment of the ServiceLoader.load() calls will happen on the fly.
  2. Statically by transforming the bundle Jar file before installing it in the OSGi framework. This mechanism doesn't need the 4.3 Weaving Hooks.
As not all OSGi frameworks support the Weaving Hooks yet, I'll be showing the static mechanism here. If you want to try the dynamic mechanism see the Aries SPI-Fly docs for more info.
The static mechanism consists of a tool that transforms your bundles inserting the bytecode needed to set the TCCL when ServiceLoader.load() is called. You can build the tool yourself from here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-static-tool or download the copy that I built from here: spifly-static-tool.jar .

  java -jar spifly-static-tool.jar consumer-bundle.jar
  [SPI Fly Static Tool] Processing: consumer-bundle.jar

When it's finished you'll find a new bundle called consumer-bundle_spifly.jar in the same directory. This bundle has the ServiceLoader.load() calls treated so that the TCCL is set appropriately for it to work.

Use it at Runtime
Our transformed client bundle has an additional dependency on some SPI-fly classes which are used to appropriately set the TCCL once ServiceLoader.load() is called. This bundle can be found here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-static-bundle and I have created a pre-built version as well: spifly-static-bundle.jar .

Deploy them into your OSGi Framework. AS7 supports a variety of deployment mechanisms, via maven, AS7 Web Console or Command Line, Felix OSGi Console or JMX. For the sake of simplicity I'm using deployment by copying to the standalone/deployments folder here. Note that although deployment by copying is extremely simple, it does require attention to the order as the bundles found in the deployments folder are instantly started.

So I'm deploying:
On the console we'll see various log messages appear, including these from our SPI Consumer:

12:44:27,480 INFO  [stdout] (MSC service thread 1-3) *** Result from invoking the SPI directly:
12:44:27,483 INFO  [stdout] (MSC service thread 1-3) Doing it too!

Which shows that our ServiceLoader Consumer bundle is now working property inside OSGi.

Conclusion
This example uses java.util.ServiceLoader provider and consumer implementations as-is in OSGi. Using the OSGi Service Registry is clearly a nicer solution, but you simply don't always have the option to change the code from using ServiceLoader to using the OSGi ServiceRegistry. That's where OSGi ServiceLoader support may come in useful.

It can already be used today in any compliant OSGi 4.2 Framework, but in that case consumer bundles need to be transformed before they are installed. If you have a 4.3 (or later) framework the transformation can happen on the fly which means that the whole process becomes more user-friendly: just install ServiceLoader support in the Framework and deploy your bundles - no static tool is needed in that case.

ServiceLoader support is being turned in an OSGi Specification scheduled to be part of the upcoming OSGi Enterprise Release (due early 2012). You can read more details about it in RFC 167 in the Early Access Draft here: http://www.osgi.org/download/osgi-early-draft-2011-05.pdf

15 comments:

Mike Norman said...

I wrote a blog entry about precisely this issue: http://mwnorman.blogspot.com/2010/09/overriding-built-in-java-httpserver.html

I have entered a bug (https://bugs.openjdk.java.net/show_bug.cgi?id=100152), but unfortunately, it hasn't been assigned to anyone.

Cole said...

When I started reading your article, I had thoughts forming in my head of how this might be accomplished. It turned out to be a very different approach from yours so I thought I would share.

For starters, we have been big fans of Declarative Services for a while now, specifically the Equinox implementation. That got me thinking that service providers could just be picked up the same way and offered as declarative services. Perhaps it would be possible to just look for META-INF/services for each bundle that is registered, and then register the service with the OSGi service registry as the SPI interface. Equinox DS uses a proxy instance so that the actual service implementation is not instantiated until it is requested so this could be done here as well.

For the consumer side, perhaps the framework could provide a custom implementation of java.util.ServiceLoader that would just look in the OSGi service registry and find the appropriate service. Since the framework has control over the classloader it should be able to do this I think.

Then it should all *just work*, with the added benefit that both sides wouldn't actually have to be using ServiceLoader. You could have providers and/or consumers using proper OSGi.

I'll have to do some digging and see if this would be possible. Anyone else have a though on this idea?

Erik Brakkee said...

I used a solution in the past where we had a lot of existing services that were designed for a java SE environment but where we wanted a more dynamic way of loading them.

For that we used OSGi, and using a bundle tracker it was possible to read the META-INF/services/ file from each deployed bundle.

Then the classes were loaded using the bundle class loader (bundle.loadClass()), i.e. delegating to the bundle. The bundles containing the services did not export the package.

For the client side use an API similar but different from java.util.ServiceLoader was used. This was acceptable since there was only one client but many services and modifying the client was acceptable.

I also didn't go that far to identify bundles as SPI-Providers.

From what I gather from your description, your approach is more generic since you use the same client API as Java SE but modified using AOP to implement this new behavior and also restricting the overhead of the mechanism using the SPI headers. In this way it is an attractive solution since clients can remain unmodified as well.

Interesting stuff.

davidb said...

@Cole:
"look for META-INF/services for each bundle ... and register the service with the OSGi Service Registry" is actually something that I'm doing too. Granted, I only mention it briefly above:
"It registers the implementations found in META-INF/services in the OSGi Service Registry so that they can be used from there too"
I think that could fit very much with your ideas around using these SPIs with DS. It's described in more detail in RFC 167: http://www.osgi.org/download/osgi-early-draft-2011-05.pdf

On the consumer side we have a bigger problem. Because java.util.ServiceLoader is in the java.* package, the JVM doesn't allow you to replace it (that's a security feature of Java). And because it's a final class you can't even subclass it to provide an alternative implementation. So if you want to make existing consumers that happen to use java.util.ServiceLoader work without changing their code you'd have to work within the boundaries of that class, which is what I did. Another option could be to use byte code manipulation to change all the invocations java.util.ServiceLoader.load() to something else (e.g. something that looks up the implementation in the SR). That would be an alternative implementation option - for the user they would be more or less equivalent I think?

davidb said...

@Erik:
Using a bundle tracker and delegating the loading to the bundle is exactly what I'm doing too :) And I'm registering these objects in the OSGi Service Registry...

"For the client side an API similar but different from java.util.ServiceLoader was used"
Yes, if you can use a different API in the client that's obviously better. I would suggest using the OSGi ServiceRegistry instead of java.util.ServiceLoader if you can :)

cmoulliard said...

Hi David,

Thx for this excellent publication which is an excellent intro about Aries Spi-Fly. For your information the dropbox links are not longer available. Could you reactivate them or provide other link reference ? I see that some examples have been published in Aries project They could maybe help us too.

Regards,

Charles

David Bosschaert said...

Hi Charles,

Sorry for removing those downloads - I cleaned up my dropbox recently and apparently removed a bit too much :)

In any case the SPI Fly project has moved on quite a bit since as it now implements the OSGi Enterprise R5 Spec Chapter 133.
All of the download links that I put in drop can be obtained by simply doing a 'mvn install' in http://svn.apache.org/repos/asf/aries/trunk/spi-fly so they're quite easy to recreated. In the mean time I'll work on getting an SPI Fly 1.0 release out and will update the article with permanent links once its there.

BTW the SPI Fly documentation has recently been updated too with information on how to get this running, see here: http://aries.apache.org/modules/spi-fly.html

cmoulliard said...

Thx for the update David. Appreciate also that you will work on SpiFly release. As Aries will publish soon also a 1.0 release, that could be great to have also SpiFly.

Remark : Could you also mention in your post that for static weaving, we must use the tool as mentioned on Aries Web Site.

David Bosschaert said...

Hi Charles,

"Could you also mention in your post that for static weaving, we must use the tool as mentioned on Aries Web Site."

Thanks for the feedback, yes I will keep that in mind. I've recently changed the build so that it will automatically build the static tool by default. Once the SPI Fly 1.0 release is out it should be possibly to download this as an executable jar file from maven. That should simplify things a little hopefully.

Dieter Cailliau said...

I'm using jboss 7.1.3 and java.util.ServiceLoader.load(class_).iterator() just works, it succeeds in loading all implementations from different jars within my ear file. Some of these jars are in the ear file root, others in the /lib folder.

So i got curious and printed the classloader in the default constructor of my implementations, and it printed a classloader that referred to the jar containing the implementation class.

So i guess jboss 7.1.3 somehow has your solution on board, but without the additional manifest entries.

Dieter Cailliau said...

It just works (using AS 7.1.3). Am i missing something here?

I have an .ear file containing many jars, one having the interface and the ServiceLoader.load(class_).iterator() code, and other ones having implementations and the META-INF/services/* files, and my code _does_ obtain all implementations.

This solution is probably built in somehow (without the need for the additional manifest headers).

David Bosschaert said...

Hi Dieter, it seems like you're using pure JavaEE deployments here. As far as I understand JBoss AS has built-in support for these. This article relates to using java.util.ServiceLoader in OSGi...

Setya said...

Hi David,

When ServiceLoader is used in OSGI environment as is, does the problem include as depicted the following error message pattern:

Caused by: java.util.ServiceConfigurationError: xxx : Provider xxx not a subtype

Cause I've followed your static weaving approach yet the problem persists.

Regards,
Setya

David Bosschaert said...

Hi Setya,

I haven't come across that one yet. Would it be possible to create an issue in ARIES jira https://issues.apache.org/jira/browse/ARIES with little testcase so that I can try to reproduce?

Thanks,

David

Setya said...

Hi David,

I've created ARIES-1470 JIRA issue, but I doubt the description is enough to reproduce the problem since it's intermittent. I wish I could attach Virgo Jetty Server instance with our small apps deployed, but the size is prohibitive.
From my observation the error seems to be caused by the interface classloader being different from the implementation, so it's regarded not assignable.

Regards,
Setya