A Web Site Concentrator to ease AJAX development

Every now and then I do a little AJAX development and there is one thing that I always find frustrating during the development of AJAX apps: The browser's single-origin policy.

Now I agree that for non-development cases this policy is absolutely justified, but during development it's a big PITA.
Let's say you are developing an AJAX app that calls into a back-end app over XML and you want to debug both. Like in my case I have a little GWT app that invokes on a Distributed OSGi Service running on the back end in CXF over port 9000. Ultimately you'd deploy it all over a single server, so there wouldn't really be a problem in production, but during development my GWT app is running on http://localhost:8888/org.coderthoughts.dinner.DinnerClient while my CXF back end runs on http://localhost:9000/org/apache/cxf/dosgi/samples/springdm/DinnerService

If I try to invoke my back-end app in this setup my AJAX app complains:


Since your front-end and back-end are probably written in different languages chances are that they have different development and debugging environments, which makes adhering to the single-origin hard to do. Besides, you don't want to go through the hassle of rebuilding your .WAR files (or whatever format you might use) for every single change. I know MSIE has some options to alleviate this, but they don't always work. Also using JSON as a data format gives you some options to bypass the problem, but that only works if your data format is JSON. In my case it's XML.

Enter ProxyServlet. There's a few of these around, but they're generally a bit raw. I particularly liked the one from Frank Spieleck because it has no dependencies, so is simple to use.
Typically these ProxyServlets forward requests from one context to another location, so to build the multi-site concentrator that you need to make the browser think both of them come from the same source you need at least two of them. In my case I need the following proxy settings:

• org.coderthoughts.dinner.DinnerClient => http://localhost:8888/org.coderthoughts.dinner.DinnerClient


• org => http://localhost:9000/org
  

To ease this task I wrote a little commandline wrapper around Frank's ProxyServlet that simply generates a .WAR file with the right name to set the root context, the correct bits in the web.xml etc... So I generate 2 .WAR files:

java -jar genwebproxy.jar \
  org.coderthoughts.dinner.DinnerClient \
  http://localhost:8888/org.coderthoughts.dinner.DinnerClient
  Created: org.coderthoughts.dinner.DinnerClient.war

java -jar genwebproxy.jar org http://localhost:9000/org
  Created: org.war
and simply deploy them into Tomcat by copying them into the webapps directory. These are standard .WAR files so you can deploy them into any Webapp container. I've also put the PAX-WEB headers in, in case you want to deploy them into that.

Et voilà. It works!


Note that I access my AJAX app using the Tomcat port of 8080 instead of the GWT debug port of 8888. I did have to change the port number at with my webapp makes an invocation from 9000 to 8080 as well, but that's easy.

With Frank's permission I put the ProxyServlet plus my wrapping code for the genwebproxy.jar file in SVN here: http://coderthoughts.googlecode.com/svn/trunk/GenWebProxy

You can download the binary from here, the command line is:
java -jar genwebproxy.jar [context_root] [real_location]

An OSGi Bundle dependency visualizer in GMF

When working with larger OSGi-base systems you sometimes wonder: why are all these bundles there? What other bundles does my bundle depend on at runtime? Is there any duplication going on?
For a project that I was working on I started manually creating Bundle Dependency diagrams. This got me thinking: it should be really easy to create a tool that does this for me using EMF (Eclipse Modeling Framework) and GMF (the Graphical Modeling Framework from Eclipse).

As always in Model Driven Development, you start with a model. In my case a model of how OSGi Bundles relate to each other.
Having the model, you just fill it with data, then use some GMF goodness to display is graphically.

Sounds easy? It was easy! With well under 300 lines of manually written code (and a whole bunch of EMF/GMF generated stuff) the end result looks like this:


In this picture I ran the visualizer in the ServiceMix 4 Kernel to see what was in it.

Main Components

There are two components in this application:

1. A bundle that you drop into an OSGi container, when you start the bundle it will walk all the other bundles in the OSGi container and figure out what their dependencies are. The result is stored in an EMF model which is stored a file called save.bundles at the end of the process.
   
2. An Eclipse GMF application that can create a visual diagram from the save.bundles file and allows you to tweak the resulting picture any way you like.
   

How it's done
The model

1. Start with a simple model of a Framework (representing the OSGi Container) which contains bundles. Each bundle in turn could be importing stuff from other bundles:

Here are the model details:


After generating the genmodel file from my model file (I only modified the base package in the genmodel package properties) I generated the model code so I could start coding with it. (See here for an earlier post that outlines how to generate code from an EMF model).

The Introspector Bundle
To create a model instance with the bundle dependency data, I've written an self-contained OSGi bundle that I can drop on any OSGi runtime (well, at this point I only tried Equinox and Felix). When the bundle is started it walks all the active bundles in the container and figures out what other bundles each bundle depends on by using the bundle's classloader to resolve imported classes. (Since Package Imports are resolved dynamically this can only be done at runtime in an active OSGi container). The collected info is stored in an instance of my model. I stayed away from using any implementation-specific admin API's, instead I'm just using the bundle's classloader to resolve the packages listed in the Import-Package header to and figure out what bundle the OSGi runtime has linked them to.
Additionally, Require-Bundle dependencies are stored in the model too.

Finally, after all the bundles have been processed, I use the built-in EMF XMI serialization to save model instance to a file (currently always called save.bundles in your temp directory).

That's the hard bit done. You can see the code here in the BundleIntrospector and BundleIntrospectorTools classes which are invoked from my simple Activator.

The Visualizer
Believe it or not, the easy bit is in creating the graphical visualizer. GMF does all the hard work for me. I'm by no means a GMF connaisseur, so I used the GMF Cheat Sheet that comes with GMF in Eclipse. I ran all the wizards from the Cheat Sheet, and used all the defaults to create the .gmfgraph, .gmftool, .gmfmap and .gmfgen files. For some reason, there was one setting that I had to manually change: the Label Mapping, but the cheat sheet told me what to do.
Finally I hit 'generate diagram code' on the .gmfgen file and I'm done.

My Eclipse Workspace looks like this.


Running it

Running the introspector with Equinox
This is very easy from within Eclipse. To try this out, create a new OSGi Framework launch configuration with the introspector bundle and, to give it something to look at, I also include the org.mortbay.jetty bundle and it's dependencies:


Make sure that the introspector bundle is not automatically started. When I launch Equinox I have the following bundles loaded:

osgi> ss
Framework is launched.

id   State    Bundle
0    ACTIVE   org.eclipse.osgi_3.3.1.R33x_v20070828
1    ACTIVE   org.mortbay.jetty_5.1.11.v200706111724
2    ACTIVE   org.apache.commons.logging_1.0.4.v200706111724
3    ACTIVE   javax.servlet_2.4.0.v200706111738
4    RESOLVED osgi_bundles.introspector.bundle_0.0.3.qualifier

When I start the introspector bundle it starts analyzing the other bundles:

osgi> start 4
Initializing data from bundle: org.eclipse.osgi
Initializing data from bundle: org.mortbay.jetty
Initializing data from bundle: org.apache.commons.logging
...
Saving... Done dependency model saved to: c:\temp\save.bundles
Finished introspecting, stopping bundle.

Running the Visualizer
Now that I have a data file, I can run the visualizer. Just right-click on one of your projects in Eclipse and select Run As Eclipse Application.
In the Eclipse Application that gets started, create a new Project and copy the save.bundles file in there.

At this point the save.bundles file is purely an EMF serialization into XMI, but GMF can initialize a picture from it, right-click 'initialize bundles_diagram file', and you'll get one:

Tadaa, a nice visualization of the bundles and their dependencies. The Jetty bundle uses the commons logging and servlet bundles (the system bundle is ommitted).
Now you can use the GMF features to rearrange your diagram, choose colours, line smoothness etc to make your diagram the way you want it (which is quite useful once you have a lot of bundles). You can even export the picture as a PNG, JPG or SVG file.

Running the introspector in ServiceMix 4 Kernel
The ServiceMix 4 Kernel is an OSGi based system built using Apache Felix. If you want to run the introspector in the ServiceMix 4 Kernel you can follow these steps:
1. Get hold of a SMX 4 Kernel distribution. I created mine by checking out the code from http://svn.apache.org/repos/asf/servicemix/smx4/kernel/trunk and running mvn install to build it. You will find the distribution in assembly/target subdirectory.
2. Run bin/servicemix

 ____                  _          __  __ _
/ ___|  ___ _ ____   _(_) ___ ___|  \/  (_)_  __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
 ___) |  __/ |   \ V /| | (_|  __/ |  | | |>  <
|____/ \___|_|    \_/ |_|\___\___|_|  |_|_/_/\_\

 ServiceMix (1.0-m3-SNAPSHOT)

Type 'help' for more information.
------------------------------------------------
servicemix>

Move to the osgi shell and install the introspector, I'm doing this straight from the Google Code project downloads section:

servicemix> osgi
servicemix osgi> install http://coderthoughts.googlecode.com/files/osgi_bundles.
introspector.bundle_0.0.3.200804202025.jar
Bundle ID: 28
servicemix osgi>

When I start bundle 28 it will do the introspection and save this to the save.bundles file. When loaded in the visualizer, you'll get a picture like the first one in this posting.

Getting your environment set up
I used Eclipse 3.3.1.1 with the EMF/GMF plugins loaded. See this posting for details on setting it up.

Getting the code
All the code used here is available under the Apache License from the Google Code project via SVN. The Eclipse Projects are accessible from here: http://coderthoughts.googlecode.com/svn/trunk/osgi_visualizer/

Getting the distribution
If you just want to use or play with the visualizer, you can download the introspector bundle and visualizer plugins from the the download area: http://code.google.com/p/coderthoughts/downloads/list

Improvements are welcomed!
The project is by no means finished. I'm still in the process of learning more about GMF and any suggestions or improvements are appreciated.

Using Jersey (JSR311) inside OSGi with Spring-DM

Or: using REST & OSGi to create a number guessing game

Jersey is an implementation of JSR 311 also know as JAX-RS. It makes writing a REST service quite simple with Java Annotations. So I wanted to use Jersey inside my OSGi bundle with the OSGi HTTP Service. However, as I found out, Jersey is not written to run in an OSGi container...

Fortunately, the problematic code in Jersey is pluggable. In this posting I'm writing a little Jersey REST Application that runs in my OSGi container. I'm using Spring-DM (aka Spring-OSGi) to do the OSGi Service dependency injection which keeps my code really lean & clean (Spring Clean ;) .

To illustrate I wrote a little number guessing game. The very first program I ever wrote was a game like this. This was on the Commodore Pet computer and I was about 10 years old. So beware...

Development Environment Setup
The Jersey version I got is 0.7-ea, which I downloaded from the Jersey download site.

I'm using the Equinox OSGi runtime in Eclipse 3.3.2 (get it here) with Spring-DM added in.

I downloaded Spring-DM 1.0.2 from here and copied all the jars from the dist and the lib directory of the download in my eclipse/plugins directory. The jars in the Spring-DM lib directory have been OSGi-ified by the Spring guys so we can nicely drop them in as OSGi bundles.

Bundle Dependencies
The general dependency picture is like the image below. I'm writing a Jersey app in a bundle that depends on a library bundle which contains the Jersey jars. This way multiple bundles can use the Jersey functionality to implement REST services while sharing the same Jersey Library bundle.


The Jersey Library Bundle
This is a simple wrapper around the Jersey libraries. At the moment I'm only depending on jersey.jar, jsr311-api.jar and asm-3.1.jar. So I'm simply creating an OSGi Bundle Project in Eclipse (Plug-in Project - see this posting for some details on how to do this), add these three jars from the Jersey install to the root of the bundle and then put them on the Bundle-Classpath. Finally simply export all the classes defined by these bundles. I guess a subset of the classes would do too, anyone fancy figuring out what the minimum Export-Package would be?

My MANIFEST.MF looks like this:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: JerseyLibBundle
Bundle-SymbolicName: JerseyLibBundle
Bundle-Version: 1.0.0
Bundle-ClassPath: asm-3.1.jar,
 jersey.jar,
 jsr311-api.jar,
 .
Import-Package: javax.servlet,
 javax.servlet.http
Export-Package: ...,
 ... export all the packages in these jars ...


That's the Jersey Library Bundle done. My Eclipse screen looks like this (click on it to see all the details):


The REST Bundle
This is the interesting bit. Here's where we'll have our REST Guessing Game implemented.

The REST application will be available on:

  http://localhost/fun/numbers
Lists all the numbers available in XML format.

  http://localhost/fun/numbers/{number}
Returns information about the requested number in text format. The information says what number it is (the same one you asked for) and whether or not you guessed the right number.

I have a jersey_osgi.rest.Number class:
package jersey_osgi.rest;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class Number {
  enum Location {LESS, FOUND, MORE};
  private static final int MAX_NUM = 10;

  static List<Number> NUMBERS;
  static {
    // Initialize with numbers 1 to 10,
    // and pick a random on to guess.
    NUMBERS = new ArrayList<Number>();

    Random random =
      new Random(System.currentTimeMillis());
    int theOne = random.nextInt(MAX_NUM) + 1;

    for (int i=1; i <= MAX_NUM; i++) {
      Location location;
      if (i == theOne) {
        location = Location.FOUND;
      } else if (i < theOne) {
        location = Location.LESS;
      } else {
        location = Location.MORE;
      }
      NUMBERS.add(new Number(i, location));
    }
  };

  private final int number;
  private final Location location;
  public Number(int num, Location l) {
    number = num;
    location = l;
  }

  public int getNumber() {
    return number;
  }

  @Override
  public String toString() {
    StringBuilder sb = new StringBuilder();
    sb.append("I'm number ");
    sb.append(number);
    sb.append(". ");
    switch (location) {
    case FOUND:
      sb.append("You guessed right!");
      break;
    case LESS:
      sb.append("You need to guess higher.");
      break;
    case MORE:
      sb.append("You need to guess lower.");
      break;
    }
    return sb.toString();
  }
}

Next thing we need is a JSR311 compliant class that binds our Number class to a REST API:
package jersey_osgi.rest;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.ProduceMime;

@Path("/numbers")
public class NumberResource {
  @GET
  @ProduceMime("text/xml")
  public String listNumbers() {
    StringBuilder sb = new StringBuilder();

    sb.append("<!-- Guess the right number! The following numbers are available. -->");
    sb.append("\n");
    sb.append("<numbers>");
    sb.append("\n");
    for (Number number : Number.NUMBERS) {
      sb.append("<number>");
      sb.append(number.getNumber());
      sb.append("</number>");
      sb.append("\n");
    }
    sb.append("</numbers>");
    return sb.toString();
  }

  @GET
  @Path("/{id}")
  @ProduceMime("text/plain")
  public String getNumber(@PathParam("id") String id) {
    try {
      int i = Integer.parseInt(id);
      Number n = Number.NUMBERS.get(i-1);
      return n.toString();
    } catch (Throwable th) {
      return "Sorry that number is out of range";
    }
  }
}
That's our REST implementation done. Only thing left to do is use the OSGi HTTPService to launch the Jersey Servlet and tell it what our REST Resource are, right? You wish! You would think that the following piece does the trick:

HttpService httpService = ...; // The OSGi HTTP Service
Hashtable initParams = new Hashtable();
initParams.put(
  // The following are standard Jersey properties
  "com.sun.ws.rest.config.property.resourceConfigClass",
  "com.sun.ws.rest.api.core.PackagesResourceConfig");
initParams.put(
  "com.sun.ws.rest.config.property.packages",
  "jersey_osgi.rest");

Servlet jerseyServlet =
  new com.sun.ws.rest.spi.container.servlet.ServletContainer();
httpService.registerServlet("/fun", jerseyServlet, initParams, null);

But hey! I'm getting an ugly exception:
com.sun.ws.rest.api.container.ContainerException: The ResourceConfig instance does not contain any root resource classes.

This is because the com.sun.ws.rest.api.core.PackagesResourceConfig class contains OSGi unfriendly classloader code (it tries to treat classes as resources, which at least does'nt work on Equinox). Luckily this class is plugged into Jersey, so I can write an OSGi-friendly version. PackagesResourceConfig searches the specified packages for any classes with the JSR311 annotations.
My alternative implementation is called jersey_osgi.common.OSGiResourceConfig. The source is slightly too big for the posting, so just click on the link if you want to see it.

For my OSGi version I have to actually specify the classes that are annotated in a property called jersey_osgi.classnames. So I change to properties like this:
HttpService httpService = ...; // The OSGi HTTP Service
Hashtable initParams = new Hashtable();
initParams.put(
  "com.sun.ws.rest.config.property.resourceConfigClass",
  "jersey_osgi.common.OSGiResourceConfig");
initParams.put(
  "jersey_osgi.classnames",
  "jersey_osgi.rest.NumberResource");

Servlet jerseyServlet =
  new com.sun.ws.rest.spi.container.servlet.ServletContainer();
httpService.registerServlet("/fun", jerseyServlet, initParams, null);
And run it. Now I'm getting:
com.sun.ws.rest.api.container.ContainerException: No WebApplication provider is present aaaargh! After some debugging I found out that Jersey heavily depends on the META-INF/services SPI architecture which is quite non-OSGi friendly. There are only two possible solutions to this that I can think of:

1. Export-Package META-INF.services in the Jersey Library Bundle. This works but is kinda nasty because you can only do it once in a single OSGi runtime. If you have multiple bundles exporting the same package, the OSGi runtime will only link you to only one of them for this package. If multiple bundles export META-INF.services you will get unexpected behaviour.


2. Copy the META-INF/services directory from the jersey.jar into our own bundle. Other projects with OSGi bundles seem to be taking this approach too.
   

So I'm taking approach 2 as well and copy all the META-INF/services files of jersey.jar into my bundle. And now it works! I can nicely build my Jersey apps as OSGi bundles that all depend on shared Jersey code.

Where's the Spring stuff?
Spring-DM makes it really easy for me to interact with the OSGi HTTP Service as it simply injects it in my JerseyServletFactory class which then registers the Jersey Servlet. To trigger this, I put a jersey_osgi_spring.xml file in the META-INF/spring directory of my bundle:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:osgi="http://www.springframework.org/schema/osgi"
  xsi:schemaLocation="...">

  <osgi:reference id="httpServiceRef"
    interface="org.osgi.service.http.HttpService"/>

  <bean name="jerseyServletFactory"
    class="jersey_osgi.common.JerseyServletFactory" >
    <property name="jerseyClassNames"
      value="jersey_osgi.rest.NumberResource"/>
    <property name="rootContext" value="/fun"/>
    <property name="httpService" ref="httpServiceRef"/>
  </bean>
</beans
The JerseyServletFactory registers the Jersey Servlet with the OSGi HTTP Service. For this it needs that HTTP Service in the first place. Those who had the pleasure of using the pure OSGi API's to get hold of an OSGi Service know that this is actually slightly tricky. So OSGi Declarative Services come to the rescue (which I used in the earlier article about writing a Card Game with OSGi Services). Spring-DM provides an alternative to OSGi-DS and brings with it all of the goodies that Spring has to offer. With the above Spring file, my JerseyServletFactory is nicely reusable (the Root Context and Jersey resource classes are also externalized into the Spring Configuration file) and no bigger than this:
package jersey_osgi.common;

import java.util.Hashtable;
import javax.servlet.Servlet;
import org.osgi.service.http.HttpService;
import com.sun.ws.rest.spi.container.servlet.ServletContainer;

public class JerseyServletFactory {
  private String classNames;
  private String rootContext;

  // Spring uses the setters to inject our stuff
  public void setJerseyClassNames(String names) {
    classNames = names;
  }

  public void setRootContext(String ctx) {
    rootContext = ctx;
  }

  public void setHttpService(HttpService httpService) throws Exception {
  Hashtable<String, String> initParams =
    new Hashtable<String, String>();
    initParams.put(
      "com.sun.ws.rest.config.property.resourceConfigClass",
      "jersey_osgi.common.OSGiResourceConfig");
    initParams.put(
      "jersey_osgi.classnames",
      classNames);

    Servlet jerseyServlet =
      new ServletContainer();
    httpService.registerServlet(rootContext,
      jerseyServlet, initParams, null);
  }
}

Running it all inside Eclipse
Running Spring-DM inside Equinox as shipped with Eclipse requires careful definition of the runtime environment. You set this up by going to Run | Open Run Dialog ... in Eclipse where you create a new OSGi Framework run configuration. Deselect the Target Platform tickbox and only select the following bundles:
JerseyLibBundle_1.0.0
JerseyOsgiSpringDM_1.0.0
javax.servlet_2.4.0.v200706111738
org.apache.commons.logging_1.0.4.v200706111724
org.eclipse.core.jobs_3.3.1.R33x_v20070709
org.eclipse.core.runtime.compatibility.registry_3.2.100.v20070316
org.eclipse.equinox.common_3.3.0.v20070426
org.eclipse.equinox.http.jetty_1.0.1.R33x_v20070816
org.eclipse.equinox.http.registry_1.0.1.R33x_v20071231
org.eclipse.equinox.http.servlet_1.0.1.R33x_v20070816
org.eclipse.equinox.registry_3.3.1.R33x_v20070802
org.eclipse.osgi.services_3.1.200.v20070605
org.eclipse.osgi_3.3.2.R33x_v20080105
org.mortbay.jetty_5.1.11.v200706111724
org.springframework.bundle.osgi.core_1.0.2
org.springframework.bundle.osgi.extender_1.0.2
org.springframework.bundle.osgi.io_1.0.2
org.springframework.bundle.spring.aop_2.5.1
org.springframework.bundle.spring.beans_2.5.1
org.springframework.bundle.spring.context_2.5.1
org.springframework.bundle.spring.core_2.5.1
org.springframework.osgi.aopalliance.osgi_1.0.0.SNAPSHOT
Like this (I edited the unselected bundles out of the screenshot):


I found that if I don't do it exactly like this that I'm getting problems with the correct logger being instantiated, but this configuration works. Just hit 'Run' and you're off. You can now access the REST application from http://localhost/fun/numbers


I can guess a number: http://localhost/fun/numbers/5

... and try again: http://localhost/fun/numbers/3

Yay!

Get the code
You can get all the code (Apache License) as usual from the google code site. It's at available as 2 Eclipse projects at http://coderthoughts.googlecode.com/svn/trunk/jersey_osgi/. The easiest way to get them into your Eclipse installation is by using an SVN client inside Eclipse and check them out as projects. This is described in more detail in this posting (just search for the 'subclipse' in there).