Tuesday, February 3, 2009

Distributed OSGi – A Simple Example

One of the exciting things to come out in 2009 is the new version of the OSGi Specification. It's called 4.2 but should really have been called 5.0 since there's a lot of new stuff in it. Distributed OSGi for example. (You can find a preview of the Distributed OSGi specification in this doc: http://www.osgi.org/download/osgi-4.2-early-draft2.pdf look for RFC 119 from page 196).
Distributed OSGi enables the distribution of OSGi Services across VMs and provides a standardized way of doing this. It doesn't specify what protocol or binding should be used on the wire for the inter-VM communication, that's up to the implementation. What it does specify is which standard properties you set on your OSGi Service to declare that it should be remoted.



The Service Implementation side
So here it is in its simplest form. Take an OSGi Auction Service that implements the following Java Interface:
public interface AuctionService {
  int listNewItem(String description, int startPriceInCents, Date expiryDate);
  void bid(int itemID, String user, int priceInCents)

  Collection<Integer> getItemIDs(String query);
  Collection<AuctionItem> getItems(int ... id);
}

With this you can list a new item, bid on an item, search for items and get item details given the item's IDs.
I've got a noddy little implementation of this service here: AuctionServiceImpl.java.
Note that I designed the interface to be suitable for distribution with the technology that I'll be using. This means that I don't rely on pass-by-reference for my method arguments and return types. In most cases distribution technologies will use pass-by-value so I made my design compatible with that.
I have put this interface in a separate bundle called AuctionIntf so that it can be used by both the service implementor and the consumer.

Now I want make this service available to my remote OSGi service consumers. The only thing I have to do for this is set a property on the OSGi service when I register it in the Activator with the local OSGi Service Registry: osgi.remote.interfaces=*. This means that I declare all the interfaces passed to bundleContext.registerService() are suitable for remoting. In my case I'm only passing one interface. (You could list individual interfaces instead of passing '*' if you only want to expose a subset).

public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    sr = context.registerService(AuctionService.class.getName(),
        new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}

I'm putting the service implementation and the Activator in a bundle called AuctionImpl which uses the AuctionService interface from the AuctionIntf bundle.

Any my Distributed Service is done!
Well... as long as I've got an implementation of Distributed OSGi running as well.

Lets do an experiment with the server side running on Equinox and the client side running on Felix. For Equinox you need version 3.5 built in December 2008 or later. For Felix you need version 1.4.1 or newer.
I'm using the Distributed OSGi DSW Reference Implementation which is based on Apache CXF and have the single-bundle version of that installed.

It's currently not yet released, but you can get a snapshot version via Maven from here (the bundle name is cxf-dosgi-ri-singlebundle-distribution).

Install this, plus its dependencies in Equinox and you'll get the following bundles:
osgi> ss

Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20081201-1815
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE javax.servlet_2.5.0.v200806031605
3 ACTIVE cxf-dosgi-ri-singlebundle-distribution
4 ACTIVE AuctionIntf_1.0.0
5 ACTIVE AuctionImpl_1.0.0

The distributed OSGi bundle depends on a few API's from the OSGi compendium spec, which are found in the org.eclipse.osgi.services bundle, which in turn depends on the javax.servlet bundle.

Hey, my service is now remoted using Web Services and available via SOAP over HTTP! CXF even automatically generates a WSDL: http://localhost:9000/org/coderthoughts/auction/AuctionService?wsdl should look like this:


This in itself is very useful. I can use this WSDL to invoke my OSGi service using any technology that can make Web Services invocations. So I could write a C++ or even a Visual Basic client for it! (In the next posting I'll use it from a remote AJAX Web client: calling a remote OSGi service straight from a browser!)

There is another way of finding out whether your service has been made available remotely, which works independently of the middleware used. Look in the OSGi Service Registry for services implementing the org.osgi.service.discovery.ServicePublication interface:

osgi> services (objectClass=*Publication)
{org.osgi.service.discovery.ServicePublication}={
  osgi.remote.endpoint.location=http://localhost:9000/org/coderthoughts/auction/AuctionService,
  service.properties={...},
  service.interface=[org.coderthoughts.auction.AuctionService],   service.id=34}
Registered by bundle: cxf-dosgi-ri-singlebundle-distribution [3]

When a Distributed OSGi implementation is remotely exposing a service, it will in turn register a service implementing the ServicePublication interface to advertise the remote service to any available Discovery mechanisms.

The Consumer side
My consumer code looks like a very ordinary OSGi Service Consumer: a little bit messy :) In real life I'd rather use something like the OSGi Blueprint (aka Spring-DM), iPOJO or Guice-OSGi, but in this posting I'd like to keep things a little focused so I'll go with writing plain OSGi service consumer code using a ServiceTracker.

public class Activator implements BundleActivator {
  private ServiceTracker st;

  public void start(final BundleContext bc) throws Exception {
    st = new ServiceTracker(bc, AuctionService.class.getName(), null) {
      @Override
      public Object addingService(ServiceReference reference) {
        Object svc = bc.getService(reference);
        if (svc instanceof AuctionService) {
          printServiceInfo((AuctionService) svc);
        }

        return super.addingService(reference);
      }
    };
    st.open();
  }

  protected void printServiceInfo(AuctionService auctionService) {
    System.out.println("Items available for auction:");
    Collection<Integer> ids = auctionService.getItemIDs(null);

    int[] idArray = toIntArray(ids);
    for (AuctionItem item : auctionService.getItems(idArray)) {
      System.out.printf("%3d: %25s $%,6.2f %tc\n",
        item.getID(),
        item.getDescription(),
        (item.getPriceInCents() / 100.0),
        item.getExpires());
    }
  }

  private int[] toIntArray(Collection<Integer> ids) {
    // details omitted
  }

  public void stop(BundleContext bc) throws Exception {
    st.close();
  }
}

As soon as the ServiceTracker gets a callback that the AuctionService is there, it prints out some details about the items available for auction.
Euh yeah, but where are the distribution bits in the consumer? Correct – there aren't any! A remote OSGi service is looked up exactly the same way as a local OSGi service. The simple fact that the client is expressing an interest in the service is enough for Distributed OSGi to go out and check for the existence of the service as a remote service. Behind the scenes it asynchronously checks any registered OSGi Remote Service Discovery services.
In this posting I'm not using a Discovery Service yet. I'm using a alternate, more static way, of specifying where my remote service is. Via a remote-services.xml file. This file sits by default in the OSGI-INF/remote-service directory of a bundle:
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
 <service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/org/coderthoughts/auction/AuctionService
  </property>
 </service-description>
</service-descriptions>

The format of this file is specified by Distributed OSGi. osgi.remote.interfaces is the same standard property that I set on the service. The osgi.remote.configuration.pojo.address is a CXF-specific one that simply denotes where my remote service is running.

So let's run the consumer in Felix.
-> ps
START LEVEL 1
ID State Level Name
[ 0] [Active ] [ 0] System Bundle (1.4.1)
[ 1] [Active ] [ 1] Apache Felix Shell Service (1.0.2)
[ 2] [Active ] [ 1] Apache Felix Shell TUI (1.0.2)
[ 3] [Active ] [ 1] Apache Felix Bundle Repository (1.2.1)
[ 4] [Active ] [ 1] OSGi R4 Compendium Bundle (4.1.0)
[ 5] [Active ] [ 1] Distributed OSGi Distribution Software Single-Bundle Distribution
[ 6] [Active ] [ 1] AuctionIntf (1.0.0)
[ 7] [Active ] [ 1] AuctionClient (1.0.0)

Besides the default Felix system, shell and bundle repository bundles, I've also installed the OSGi Compendium bundle which contains some OSGi standard interfaces that we need.

After the client bundle has started it will receive a callback that the (remote) AuctionService it wants to use is available and it will print out some information about the available items:
Items available for auction:
1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009


You can also see the remote service represented in the local service registry the same way a local service is. It is registered on demand by the DSW bundle:
-> services 5
...
objectClass = org.coderthoughts.auction.AuctionService
osgi.remote = true
osgi.remote.configuration.type = pojo
osgi.remote.configuration.pojo.address = http://localhost:9000/org/coderthoughts/auction/AuctionService
osgi.remote.interfaces = *
service.id = 30


There's our remote service! Just like an ordinary OSGi service. An extra property is set on client side proxies: osgi.remote. This can be used to find out whether a service that you are dealing with is remote or not. It can also be used to change the default lookup behaviour. By default remote services are always included in the service lookups, but this default can be tweaked by registering a FindHook/EventHook, which is a new feature of the Service Registry specified in RFC 126.

You can find the full implementation of the AuctionIntf, AuctionImpl and AuctionClient OSGi bundles as Eclipse projects in SVN: http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite

Next posting: an AJAX client UI written using the Google Web Toolkit (GWT).

Links
OSGi Specs: http://www.osgi.org/Specifications
Disributed OSGi Reference Implementation based on Apache CXF: http://cwiki.apache.org/confluence/display/CXF/Distributed+OSGi
Apache CXF Home page: http://cxf.apache.org

27 comments:

Frederic Conrotte said...

Nice demo, thanks !

For me on of the main use case for Distributed OSGi is logical separation of related bundle in different runtimes, to avoid the mess of dealing with 150 bundles in your VM.

What do you think ?

David Bosschaert said...

Well, OSGi scales pretty well certainly up to a couple of thousand bundles I think, so 150 bundles should be no problem at all.

I can think of many potential uses of Distributed OSGi :) One of them could be to connect OSGi (or non-OSGi) systems that are distributed by their nature.

Frederic Conrotte said...

I agree about scalability. 150 was just an example.

My concern was more about manageability. From experience having a lot of unrelated bundles in the same VM makes it harder to organize and support, especially in terms of start levels.

Hopefully Distribution can help on this matter.

Anonymous said...

Nice article, but I have to comment on semantics...

Java is strictly pass-by-value.

What you're referring to as "pass-by-reference" really should be worded as "passing non-primitive/non-String-pointer" values.

You make a good point, but it makes language/compiler folks like me cringe when "pass-by-reference" is used in a Java context...

Please see http://javadude.com/articles/passbyvalue.htm for details.

Nalla Senthilnathan said...

I have a version of the demo with easy to reproduce steps: http://code.google.com/p/apache-cxf-dosgi-demo/

for anyone interested.

Dr. Abdullah Zaghameem said...

Hi all,

All the D-OSGi Demos that I had inspected choose Felix as a consumer-side. I am struggling in building an pure Eclipse (3.5) Demo that uses Equinox as OSGi container for both sides (Service provider and Service Consumer). Following the steps mentioned in this post, I success to publish the Service (as I request the WSDL), but the Consumer bundle (which is installed correctly) fails to discover it!!

I post this comment just to declare my amazement on the D-OSGi Demos !!

Regards,
AOZ

David Bosschaert said...

Hi AOZ, have a look at http://cxf.apache.org/distributed-osgi-greeter-demo-walkthrough.html. That uses Equinox 3.5 as the client side. I've just tried it with Equinox 3.5 GA on both the provider side and the consumer side and it worked for me.

NB the name of the equinox org.eclipse.osgi_xxx.jar and org.eclipse.osgi.services_xxx.jar might vary slightly depending on your 3.5 build.

Hope this helps,

David

Georgi Dimitrov said...

Hi David, great tutorial, but how can one expose RESTful instead of SOAP Service with cxf dosgi?

Sergey Beryozkin said...

RESTful services can be exposed nearly the same way as SOAP services can, some info is here

http://cxf.apache.org/docs/jax-rs.html#JAX-RS-IntegrationwithDistributedOSGi

cheers, Sergey

Unknown said...

Good demo,

Can the OSGi bundle access bundle access the web service implemented in web server?

Nirmal said...

I recently studied about combination of Cloud computing and OSGi. I am
particularly interested in scalability of the service registry in the cloud. Is it possible for service registry to be moved to separate
instance in cloud for manageability?

Thanks

David Bosschaert said...

Hi Nirmal,

Yes it is possible to create a remote service registry. This is one of the models suggested in the Remote Service Admin specification (Chapter 122 in the Enterprise Spec). Together with the Service Registry Hooks (Chapter 12 in the Core Spec) services can be looked up in the remote service registry when needed. Services will still be mirrored in the local service registry BTW.

Additionally, I don't think it's a good idea to put all your OSGi Services in the Remote Service Registry. Certain services are designed for local use only (e.g. think about the log service), so it won't make much sense to put them in a remote service registry, so which services you put in there depends on their role in the system.

Nirmal said...

Hi David,

Thanks for your reply. I do agree putting all services in remote is not a good idea.

But what would happen if multiple containers expose the same service. How can the consumer choose which service to hit dynamically? Is there any load balancer inside container?

David Bosschaert said...

Hi Nirmal,

If the same service is exposed by multiple remote frameworks it will appear multiple times in the registry, as you say. Having the same service appear multiple times in the OSGi Service Registry is not unusual at all. It can also happen locally and whether you want one of the services or are interested in all of them depends on the nature of the service.
I once wrote a card game using OSGi services where every card was represented as a service. In that case I wanted all of them. However, in other cases you may want to select one service that best fits your needs. Selecting a service is done using an LDAP-style Filter which is typically passed into the OSGi ServiceTracker. The filter selects the service based on it's properties (a service has an arbitrary map of properties associated with it).

The Distributed case is no different. If the same service appears multiple times and you only need to invoke one you can select the best one based on its properties. You could use this information to select the service automatically with a load balancer. A service could publish (and periodically update) a property which advertises its load and the load balancer could automatically select the service with the lowest load... Such a load balancer is not typically built in to Distributed OSGi products, but it can be built on top of the existing primitives...

Unknown said...

Hi david,

Thank you for this demo, I notice that you are using the interface bundle on both the client and server side. Is it possible to not use the bundle on the client side, and use the remote interface: In a distributed environment,we can discover unknown services, and then we don't know the interface describing them on the client side.

Regards,
Hamdi

Unknown said...

Hi David,

Thank you for this demo.

In the demo you are using the interface bundle on both the client and server side. Is it possible to not use the interface bundle on the client side and use the remote interface: in a distributed environment, we can discover unknown services and we don't have the interfaces describing them.

Regards,
Hamdi

Anonymous said...

Thanks for the demo.

I have inspected the dosgi bundle and get this print:
_________________
objectClass = org.coderthoughts.auction.AuctionService
org.apache.cxf.remote.dsw.client = Distributed OSGi Distribution Software Single-Bundle Distribution, version : 1.1.0.SNAPSHOT
osgi.remote.configuration.pojo.address = http://localhost:8080/auction/
osgi.remote.configuration.type = pojo
osgi.remote.endpoint.id = 5e4e9de1-a733-4953-8f36-ef5e038ca9fb
osgi.remote.interfaces = *
osgi.remote.service.interfaces =
service.id = 52
service.imported = true
service.imported.configs = org.apache.cxf.ws
_________________________________

I had two questions:
1 - I suppose I can use a distant url in ws property, meaning I want to IMPORT a remote ws as an OSGi service ?

2 - If so, how can I use it on the OSGi platfrom when it has been imported ?

Thanks

David Bosschaert said...

Hi Pierre-Henry

Yes, you can use a remote url in the osgi.remote.configuration.pojo.address. If I understand correctly you want to invoke a remote Web Service (which is not an OSGi service) as an OSGi Service.
What you need to do in this case is get a Java interface to that service and use that from the client side. As the osgi.remote.configuration.pojo.address (newer versions use org.apache.cxf.ws.address) the URL of your remote web service. Effectively what you're doing is replacing one of the sides of the Distributed OSGi demo with a non-OSGi implementation.
You might find more information here: http://cxf.apache.org/distributed-osgi.html...

Best regards,

David

Anonymous said...

Hi David,

____________________________________________
INFO: Notified - AVAILABLE: [org.myproject.webservices.yyy.Monitor] endpoint id: 984b7c00-044d-4433-927d-638c9a657e1a
27 janv. 2011 12:00:53 org.apache.cxf.dosgi.dsw.hooks.AbstractClientHook unknownEndpointId
INFO: registering proxy for endpoint ID: 984b7c00-044d-4433-927d-638c9a657e1a
27 janv. 2011 12:00:53 org.apache.cxf.dosgi.dsw.hooks.AbstractClientHook cacheEndpointId
INFO: caching proxy registration for endpoint ID: 984b7c00-044d-4433-927d-638c9a657e1a
____________________________________________

The single bundle dosgi cxf distrib shows at inspection:
____________________________________________
objectClass = org.myproject.webservices.yyy.Monitor
org.apache.cxf.remote.dsw.client = Distributed OSGi Distribution Software Single-Bundle Distribution, version : 1.1.0.SNAPSHOT
osgi.remote.configuration.pojo.address = http://xx.yy.zz.tt//webservice/service.php?wsdl
osgi.remote.configuration.type = wsdl
osgi.remote.endpoint.id = 984b7c00-044d-4433-927d-638c9a657e1a
osgi.remote.interfaces = *
osgi.remote.service.interfaces =
service.id = 13
service.imported = true
service.imported.configs = wsdl
________________________________________


So I could simply GET this service with a tracker or other mecanisms - iPOJO, DS,...

But I tried with a ServiceTracker and couldn't get any reference to this service.

Are there specific version of bundles to use, I mean compendium-4.2 with cxf-dosgi-1.1 ....?

Or what else ....?

Unknown said...

Nice post for looking at dosgi. I met a problem for could not access the wsdl for all the dosgi posts using the latest dosgi release. One possible problem is that the new 1.2 dosgi release is not compatible? Because when i try to replace the 1.1snap shot release in this post: http://blog.akquinet.de/2009/09/14/distributed-osgi-application-with-apache-cxf-dosgi/#more-191, i could not access the wsdl either.

Unknown said...

Hi,
I was wondering if it's possibile to configure Zookeeper in order to first discover the ZooKeeper server and then access it. My intention is to configure the ZooKeeper clients without a static URL of the ZooKeeper.
Do you know if this is possibile?

Thank you

Unknown said...

Hi David,

I have the same problem as weishan with the server in your example.
If I request

"services (objectClass=*Publication)"

via the OSGI console, the message "No registered services." are printed out. Requesting the WSDL via Webbrowser, there is an HTTP404 error because the requested URI can't be found.
I'm using the current CXF singlebundle distribution (1.2.0).

Is there anything needed to change in your example sothat it works well?

Thanks
Hannes

David Bosschaert said...

Hi Hannes,

This blog article is a little bit outdated and some of the properties have been renamed in the Remote Services specification when it was finalized.

Have you tried the examples linked from the CXF-DOSGi page: http://cxf.apache.org/distributed-osgi.html? They should all work well with the distribution you are using. A good place to start is the Greeter demo.

Huynh Ngoc Tho said...

Hi, i have a problem relating to DOSGi, i tried this example and i saw that DOSGi use TCP (at transport layer) for communication.

Can we configure it with UDP ?

David Bosschaert said...

Hi Huynh Ngoc Tho, this implementation uses TCP, but the OSGi Remote Services specification doesn't prescribe the technology being used. If you look at the OSGi Specification Implementations page (chapter 100 section) you may find other implementations that support other protocols including UDP...

Huynh Ngoc Tho said...

Thank you for your response.
I’ve read some related documents here http://www.osgi.org/Specifications/HomePage, however I have not found TCP or UDP protocol specification. Can you help me more detail, please?

David Bosschaert said...

The OSGi Remote Services specifications do not specify the protocol being used. That is up to the implementation. You might find an implementation that supports UDP if you look at the available implementations in http://en.wikipedia.org/wiki/OSGi_Specification_Implementations.