A Distributed OSGi Powered AJAX WebApp

A pretty cool new way of using Distributed OSGi services is from a non-OSGi environment. In the previous post I used an OSGi AuctionService from an OSGi AuctionConsumer in a different VM.
This posting is about using that OSGi service from a non-OSGi consumer. From an AJAX-powered web application that's running in the browser!

I'm writing the webapp using the Google Web Toolkit (GWT). Both the webapp as well as my OSGi service are hosted in the same OSGi container. With AJAX you can create a rich client application running inside the browser. So no Servlet/JSP style server-side execution. The webapp runs on the client machine. GWT makes creating the AJAX app easy: you write them in plain old Java. No need to sidestep into Javascript as GWT turns the Java into a Javascript AJAX app and all (well, almost) of the browser specific madness is taken care of by GWT too.

As with any website, the end-users users simply open it in their browser. No client install, no browser plugins. Any modern browser supports this stuff natively. Remote communication, such as obtaining a list of available items, listing a new item, placing a bid, is done via SOAP/HTTP calls directly on my OSGi AuctionService (the one from the previous posting) which has been remoted using the Apache CXF-based Distributed OSGi Reference Implementation.

To do all this I'm adding a component called Pax Web Extender to my OSGi container, on top of the Distributed OSGi bundles. Pax Web Extender turns the OSGi container into a web container which means that I can deploy a .WAR file into it. It internally leverages the OSGi HTTP Service. (Spring DM Server can do this too BTW).

My OSGi container is configured like this:

Besides the CXF Distributed OSGi and Pax-Web-Extender Bundles I have my AuctionService and AuctionInterface bundles deployed (see previous posting for the details), plus the .WAR file that holds my AJAX webapp.

The reason I can invoke my OSGi Service is because it was made available remotely over a standards-based mechanism. Distributed OSGi itself doesn't prescribe how you do your remoting, it only says that you can use standards-based wire protocols and bindings if you wish and that this can make OSGi Services available to non-OSGi consumers.

The CXF implementation of Distributed OSGi turns an OSGi Service into a Web Service that is accessible via SOAP/HTTP. GWT can already natively make remote HTTP invocations. I didn't actually even care looking for a SOAP library for GWT. SOAP is just XML and GWT has some pretty good XML parsing functionality.
The last thing that I need to worry about (besides security, but that's a separate topic) is the browser's single-origin policy. But hey: I've deployed my webapp in the same runtime as my OSGi Service so as long as I make them available over the same port there's no problem! I achieved this by sharing the same OSGi HTTP Service by both the Distributed OSGi runtime and the Pax Web Extender. Thanks to the OSGi Services architecture!

So I'm taking the AuctionService of the previous posting, but first of all I want to expose it on a slightly simpler URL. The default URL of http://localhost:9000/org/coderthoughts/auction/AuctionService is just a bit too long, let's use /auction as the context instead. I also want to make it use the OSGi HTTP Service (which CXF doesn't do by default). I can do both in one shot by setting the osgi.remote.configuration.pojo.httpservice.context service property to the value /auction. The HTTP Service that comes with CXF runs by default on port 8080 (you can change this by setting the org.osgi.service.http.port system property) so my Web Service now runs on http://localhost:8080/auction. Change the activator that registers the service to the following:
public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    props.put("osgi.remote.configuration.type", "pojo");
    props.put("osgi.remote.configuration.pojo.httpservice.context",
      "/auction");
    sr = context.registerService(AuctionService.class.getName(),
      new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}
To double check that the service is operational on the new address, simply open http://localhost:8080/auction?wsdl and see if the WSDL appears again (just like in the previous posting).

I won't go into all the detail of writing the GWT app here. Look at the code in the SVN project for all of that.
I constructed my GWT application the usual way by creating a starting point with the projectCreator and applicationCreator scripts that come with GWT. This creates a nice Eclipse project skeleton for GWT from which I can start hacking my AJAX site in Plain Old Java :)

All you need to do in the GWT app to invoke on the Distributed OSGi service is:

• figure out where it's running, since the Web Service is running on the same host and port as where the AJAX app is served from on the /auction context, I can construct the Web Service URL as follows:
  webServiceURL = "http://" + Window.Location.getHost() + "/auction/";
  
• create a SOAP message, just a bit of XML really.
  
• send it via a HTTP POST request to the WebService using a com.google.gwt.http.client.RequestBuilder that comes with GWT.
  
• register a callback object to process the SOAP (XML) response.
  

All of this happens in the AuctionSite class which is the webapp's main class. There are a few more classes such as data classes and a few dialogs. Check out the project from SVN to see them all...

To prepare my webapp (.WAR file) for running with the Pax Web Extender I'm turning it into an OSGi bundle by adding the following META-INF/MANIFEST.MF file:
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-SymbolicName: AuctionSite
Bundle-Version: 1.0.0
Webapp-Context: auctionsite
The Webapp-Context is important as this tells the Pax Web Extender where to register the webapp.

I also have to add a WEB-INF/web.xml file to the .WAR as this is the signature file that the Pax-Web Extender needs in order to spot a bundle as a webapp. However, I don't really need to put anything in it. My webapp consists only of static files (no servlets etc.) as far as the webserver is concerned, so my web.xml is simply:
<web-app>
  <display-name>AuctionSite</display-name>
</web-app>

GWT comes with scrips that you run to turn your Java into AJAX.
I wrote a tiny little ant script that calls the GWT script and then creates a .WAR file from my GWT project in Eclipse.

Now lets boot is all up! I'm using Equinox 3.5M5 with the Multi Bundle Distribution of CXF-DOSGi.

To install the Multi Bundle Distribution of CXF-DOSGi, download it from here. Then unzip it in your Felix/Equinox installation dir.
Take the contents of the felix.config.properties.append or equinox.config.ini.append and append it to the configuration file of your OSGi container.
This will automatically load all the DOSGi bundles when starting up the OSGi container.

• Start Equinox with the DOSGi Multi Bundle configuration to make it load all the DOSGi bundles at startup.
  
• Then install and start the PAX Web Extender from here and your AuctionIntf and AuctionImpl bundles.
  
• Finally install and start the AuctionSite.war file, just like any other bundle in OSGi.

You should now have the following bundles running in Equinox:
osgi> ss
Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20090127-1630
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE org.apache.geronimo.specs.geronimo-annotation_1.0_spec_1.1.1
3 ACTIVE org.apache.geronimo.specs.geronimo-activation_1.1_spec_1.0.2
4 ACTIVE org.apache.geronimo.specs.geronimo-javamail_1.4_spec_1.2.0
5 ACTIVE org.apache.geronimo.specs.geronimo-ws-metadata_2.0_spec_1.1.2
6 ACTIVE com.springsource.org.apache.commons.logging_1.1.1
7 ACTIVE com.springsource.org.jdom_1.0.0
8 ACTIVE org.springframework.bundle.spring.core_2.5.5
9 ACTIVE org.springframework.bundle.spring.beans_2.5.5
10 ACTIVE org.springframework.bundle.spring.context_2.5.5
11 ACTIVE com.springsource.org.aopalliance_1.0.0
12 ACTIVE org.springframework.bundle.spring.aop_2.5.5
13 ACTIVE org.springframework.bundle.osgi.io_1.1.2
14 ACTIVE org.springframework.bundle.osgi.core_1.1.2
15 ACTIVE org.springframework.bundle.osgi.extender_1.1.2
16 ACTIVE org.ops4j.pax.web.service_0.5.1
17 ACTIVE org.apache.servicemix.specs.locator-1.1.1
18 ACTIVE org.apache.servicemix.bundles.jaxb-impl_2.1.6.1
19 ACTIVE org.apache.servicemix.bundles.wsdl4j_1.6.1.1
20 ACTIVE org.apache.servicemix.bundles.xmlsec_1.3.0.1
21 ACTIVE org.apache.servicemix.bundles.wss4j_1.5.4.1
22 ACTIVE org.apache.servicemix.bundles.xmlschema_1.4.2.1
23 ACTIVE org.apache.servicemix.bundles.asm_2.2.3.1
24 ACTIVE org.apache.servicemix.bundles.xmlresolver_1.2.0.1
25 ACTIVE org.apache.servicemix.bundles.neethi_2.0.4.1
26 ACTIVE org.apache.servicemix.bundles.woodstox_3.2.7.1
27 ACTIVE org.apache.cxf.cxf-bundle-minimal_2.2.0.SNAPSHOT
28 ACTIVE org.apache.servicemix.specs.saaj-api-1.3_1.1.1
29 ACTIVE org.apache.servicemix.specs.stax-api-1.0_1.1.1
30 ACTIVE org.apache.servicemix.specs.jaxb-api-2.1_1.1.1
31 ACTIVE org.apache.servicemix.specs.jaxws-api-2.1_1.1.1
32 ACTIVE cxf-dosgi-ri-discovery-local_1.0.0.SNAPSHOT
33 ACTIVE cxf-dosgi-ri-dsw-cxf_1.0.0.SNAPSHOT
34 ACTIVE org.ops4j.pax.web.extender.war_0.4.0
35 ACTIVE AuctionIntf_1.0.0
36 ACTIVE AuctionImpl_1.0.0
37 ACTIVE AuctionSite_1.0.0


Open the webapp at http://localhost:8080/auctionsite/AuctionSite.html. This kicks off the AJAX app in your browser. When you hit the reload button it makes a SOAP/HTTP invocation on the server to list all the items and get their details. The SOAP messages it sends map to the AuctionService.getItemIDs() and AuctionService.getItems() APIs. (At the bottom of the screen you see the SOAP message as it comes back form the server)


Let's list a new item...


After filling in the Javascript popup, hit OK.
It now sends the following SOAP message from the browser to the server:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
 <soap:Body>
  <ns1:listNewItem xmlns:ns1="http://auction.coderthoughts.org/">
   <ns1:arg0>Joke</ns1:arg0>
   <ns1:arg1>1000</ns1:arg1>
   <ns1:arg2>2009-02-09T15:48:22-00:00</ns1:arg2>
  </ns1:listNewItem>
 </soap:Body>
</soap:Envelope>

Once the remote invocation has completed, the list of items is updated again, which is the same as hitting the reload button.


I can still access the web service from my remote OSGi client that I developed in the previous post. The same remote OSGi service can obviously be shared among many consumers. When I run the AuctionClient bundle I'm also seeing my newly listed item.
Before I can do that I need to update the remote-services.xml file in the consumer bundle with the new location of the Web Service (when using Discovery this is not needed BTW):
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
 <service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/auction
  </property>
 </service-description>
</service-descriptions>

Now run the AuctionClient bundle:
Items available for auction:
 1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
10:                 Joke $ 10.00 Mon Feb 09 15:48:12 GMT 2009
 2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
 3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009
You can get all the code in this posting under the Apache license from the google code project. It's all in SVN at http://coderthoughts.googlecode.com/svn/trunk/gwt_auctionsite and http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite .
Since it's all Eclipse projects, the easiest way to get it by doing the SVN checkout of the projects directly in Eclipse. It will give you a workspace that looks like this:

Distributed OSGi – A Simple Example

One of the exciting things to come out in 2009 is the new version of the OSGi Specification. It's called 4.2 but should really have been called 5.0 since there's a lot of new stuff in it. Distributed OSGi for example. (You can find a preview of the Distributed OSGi specification in this doc: http://www.osgi.org/download/osgi-4.2-early-draft2.pdf look for RFC 119 from page 196).
Distributed OSGi enables the distribution of OSGi Services across VMs and provides a standardized way of doing this. It doesn't specify what protocol or binding should be used on the wire for the inter-VM communication, that's up to the implementation. What it does specify is which standard properties you set on your OSGi Service to declare that it should be remoted.



The Service Implementation side
So here it is in its simplest form. Take an OSGi Auction Service that implements the following Java Interface:
public interface AuctionService {
  int listNewItem(String description, int startPriceInCents, Date expiryDate);
  void bid(int itemID, String user, int priceInCents)

  Collection<Integer> getItemIDs(String query);
  Collection<AuctionItem> getItems(int ... id);
}
With this you can list a new item, bid on an item, search for items and get item details given the item's IDs.
I've got a noddy little implementation of this service here: AuctionServiceImpl.java.
Note that I designed the interface to be suitable for distribution with the technology that I'll be using. This means that I don't rely on pass-by-reference for my method arguments and return types. In most cases distribution technologies will use pass-by-value so I made my design compatible with that.
I have put this interface in a separate bundle called AuctionIntf so that it can be used by both the service implementor and the consumer.

Now I want make this service available to my remote OSGi service consumers. The only thing I have to do for this is set a property on the OSGi service when I register it in the Activator with the local OSGi Service Registry: osgi.remote.interfaces=*. This means that I declare all the interfaces passed to bundleContext.registerService() are suitable for remoting. In my case I'm only passing one interface. (You could list individual interfaces instead of passing '*' if you only want to expose a subset).

public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    sr = context.registerService(AuctionService.class.getName(),
        new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}
I'm putting the service implementation and the Activator in a bundle called AuctionImpl which uses the AuctionService interface from the AuctionIntf bundle.

Any my Distributed Service is done!
Well... as long as I've got an implementation of Distributed OSGi running as well.

Lets do an experiment with the server side running on Equinox and the client side running on Felix. For Equinox you need version 3.5 built in December 2008 or later. For Felix you need version 1.4.1 or newer.
I'm using the Distributed OSGi DSW Reference Implementation which is based on Apache CXF and have the single-bundle version of that installed.

It's currently not yet released, but you can get a snapshot version via Maven from here (the bundle name is cxf-dosgi-ri-singlebundle-distribution).

Install this, plus its dependencies in Equinox and you'll get the following bundles:
osgi> ss

Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20081201-1815
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE javax.servlet_2.5.0.v200806031605
3 ACTIVE cxf-dosgi-ri-singlebundle-distribution
4 ACTIVE AuctionIntf_1.0.0
5 ACTIVE AuctionImpl_1.0.0
The distributed OSGi bundle depends on a few API's from the OSGi compendium spec, which are found in the org.eclipse.osgi.services bundle, which in turn depends on the javax.servlet bundle.

Hey, my service is now remoted using Web Services and available via SOAP over HTTP! CXF even automatically generates a WSDL: http://localhost:9000/org/coderthoughts/auction/AuctionService?wsdl should look like this:


This in itself is very useful. I can use this WSDL to invoke my OSGi service using any technology that can make Web Services invocations. So I could write a C++ or even a Visual Basic client for it! (In the next posting I'll use it from a remote AJAX Web client: calling a remote OSGi service straight from a browser!)

There is another way of finding out whether your service has been made available remotely, which works independently of the middleware used. Look in the OSGi Service Registry for services implementing the org.osgi.service.discovery.ServicePublication interface:

osgi> services (objectClass=*Publication)
{org.osgi.service.discovery.ServicePublication}={
  osgi.remote.endpoint.location=http://localhost:9000/org/coderthoughts/auction/AuctionService,
  service.properties={...},
  service.interface=[org.coderthoughts.auction.AuctionService],   service.id=34}
Registered by bundle: cxf-dosgi-ri-singlebundle-distribution [3]

When a Distributed OSGi implementation is remotely exposing a service, it will in turn register a service implementing the ServicePublication interface to advertise the remote service to any available Discovery mechanisms.

The Consumer side
My consumer code looks like a very ordinary OSGi Service Consumer: a little bit messy :) In real life I'd rather use something like the OSGi Blueprint (aka Spring-DM), iPOJO or Guice-OSGi, but in this posting I'd like to keep things a little focused so I'll go with writing plain OSGi service consumer code using a ServiceTracker.

public class Activator implements BundleActivator {
  private ServiceTracker st;

  public void start(final BundleContext bc) throws Exception {
    st = new ServiceTracker(bc, AuctionService.class.getName(), null) {
      @Override
      public Object addingService(ServiceReference reference) {
        Object svc = bc.getService(reference);
        if (svc instanceof AuctionService) {
          printServiceInfo((AuctionService) svc);
        }

        return super.addingService(reference);
      }
    };
    st.open();
  }

  protected void printServiceInfo(AuctionService auctionService) {
    System.out.println("Items available for auction:");
    Collection<Integer> ids = auctionService.getItemIDs(null);

    int[] idArray = toIntArray(ids);
    for (AuctionItem item : auctionService.getItems(idArray)) {
      System.out.printf("%3d: %25s $%,6.2f %tc\n",
        item.getID(),
        item.getDescription(),
        (item.getPriceInCents() / 100.0),
        item.getExpires());
    }
  }

  private int[] toIntArray(Collection<Integer> ids) {
    // details omitted
  }

  public void stop(BundleContext bc) throws Exception {
    st.close();
  }
}
As soon as the ServiceTracker gets a callback that the AuctionService is there, it prints out some details about the items available for auction.
Euh yeah, but where are the distribution bits in the consumer? Correct – there aren't any! A remote OSGi service is looked up exactly the same way as a local OSGi service. The simple fact that the client is expressing an interest in the service is enough for Distributed OSGi to go out and check for the existence of the service as a remote service. Behind the scenes it asynchronously checks any registered OSGi Remote Service Discovery services.
In this posting I'm not using a Discovery Service yet. I'm using a alternate, more static way, of specifying where my remote service is. Via a remote-services.xml file. This file sits by default in the OSGI-INF/remote-service directory of a bundle:
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
 <service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/org/coderthoughts/auction/AuctionService
  </property>
 </service-description>
</service-descriptions>
The format of this file is specified by Distributed OSGi. osgi.remote.interfaces is the same standard property that I set on the service. The osgi.remote.configuration.pojo.address is a CXF-specific one that simply denotes where my remote service is running.

So let's run the consumer in Felix.
-> ps
START LEVEL 1
ID State Level Name
[ 0] [Active ] [ 0] System Bundle (1.4.1)
[ 1] [Active ] [ 1] Apache Felix Shell Service (1.0.2)
[ 2] [Active ] [ 1] Apache Felix Shell TUI (1.0.2)
[ 3] [Active ] [ 1] Apache Felix Bundle Repository (1.2.1)
[ 4] [Active ] [ 1] OSGi R4 Compendium Bundle (4.1.0)
[ 5] [Active ] [ 1] Distributed OSGi Distribution Software Single-Bundle Distribution
[ 6] [Active ] [ 1] AuctionIntf (1.0.0)
[ 7] [Active ] [ 1] AuctionClient (1.0.0)
Besides the default Felix system, shell and bundle repository bundles, I've also installed the OSGi Compendium bundle which contains some OSGi standard interfaces that we need.

After the client bundle has started it will receive a callback that the (remote) AuctionService it wants to use is available and it will print out some information about the available items:
Items available for auction:
1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009

You can also see the remote service represented in the local service registry the same way a local service is. It is registered on demand by the DSW bundle:
-> services 5
...
objectClass = org.coderthoughts.auction.AuctionService
osgi.remote = true
osgi.remote.configuration.type = pojo
osgi.remote.configuration.pojo.address = http://localhost:9000/org/coderthoughts/auction/AuctionService
osgi.remote.interfaces = *
service.id = 30

There's our remote service! Just like an ordinary OSGi service. An extra property is set on client side proxies: osgi.remote. This can be used to find out whether a service that you are dealing with is remote or not. It can also be used to change the default lookup behaviour. By default remote services are always included in the service lookups, but this default can be tweaked by registering a FindHook/EventHook, which is a new feature of the Service Registry specified in RFC 126.

You can find the full implementation of the AuctionIntf, AuctionImpl and AuctionClient OSGi bundles as Eclipse projects in SVN: http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite

Next posting: an AJAX client UI written using the Google Web Toolkit (GWT).

Links
OSGi Specs: http://www.osgi.org/Specifications
Disributed OSGi Reference Implementation based on Apache CXF: http://cwiki.apache.org/confluence/display/CXF/Distributed+OSGi
Apache CXF Home page: http://cxf.apache.org