java.util.ServiceLoader in OSGi

2011-08-29T06:27:00.000-07:00

One of the bigger stumbling blocks for migrating an existing application to OSGi has been java.util.ServiceLoader and its predecessors (the various FactoryFinders that can be found in technologies such as JAXP, JAXRS, etc...).

ServiceLoader is a class provided by the JRE aimed at loading pluggable implementations of something. For those who need a little refreshing of the brain, here's what typical ServiceLoader usage looks like... Let's say you have an SPI called MySPI with a single method doit():
public abstract class MySPI {
public abstract String doit();
}
For some reason SPI providers are often realized as abstract classes, so that's what I'm doing here too. Obviously using an interface is also possible...
You load the SPI implementations by calling ServiceLoader.load(). To obtain the actual implementation objects you iterate over the result of that call:

ServiceLoader ldr = ServiceLoader.load(MySPI.class);

for (MySPI spiObject : ldr) {

  spiObject.doit(); // invoke my SPI object

}
So why is this such a big problem in OSGi? The problem is that the JRE's ServiceLoader is non-modular by design. It relies on a single classloader that has visibility over everything in your system that can possibly contain an implemetation of the service requested. The load() method as used above uses the ThreadContextClassLoader (TCCL) to find all the ServiceLoader implementations in the system. There are a number of issues with this from an OSGi perspective.

The value of the TCCL is generally not defined in an OSGi context.
The implementations are found by looking for a resource in META-INF/services. Every jar that contains an implementation of a service (e.g. my org.acme.MySPI implementation) should advertise this by putting a META-INF/service/org.acme.MySPI file in its jar file.
Once the above resources have been found, the contents of them are parsed to obtain the class names of implementations, normally these classes reside in an implementation package, something like org.acme.impl.MySPIImpl... ServiceLoader will instantiate the classes found using their no-arg constructor.

Issue 2 clearly poses an issue if this resource is to be obtained through the classloading rules set by OSGi as OSGi wires a package import only to precisely one exporter. This gives great predictability (you know exactly where you're getting a package from) but it doesn't work if you need to get the same resource in the same package from multiple bundles. And even if you were able to get all the META-INF/services resources in your system you would still have the issue that the classes listed in there are generally private implementation classes that you don't want to export outside of a bundle. To load and instantiate them from a consuming bundle you'd also have to import these private implementation packages which would be totally horrible (issue 3).

So how should we do this in OSGi? Clearly the best answer is: use the OSGi service registry. It doesn't suffer from any of the problems ServiceLoader has: no need to expose and import private packges, no need to expose in a single package across multiple bundles. Plus the OSGi Service Registry supports a dynamic service lifecycle: services can come can go, additional suitable services can appear, existing services can get replaced. Furthermore, the OSGi ServiceRegistry supports a rich language of selecting services among possible candidates by using LDAP filters that work over service properties, while ServiceLoader simply gives you a snapshot of all the known service implementations at a particular point in time.

However - you might be looking at some code that you'd like to use as-is. You might not have the luxury of using the OSGi Service Registry right now. Maybe you don't have the source code so you can't modify it or maybe you're just focusing on other things and don't want to modify the code. You just like to use the same code that works with ServiceLoader outside of OSGi, in OSGi...

For this use-case I've started looking at a solution. The solution is based on the premise that you don't want or can't modify the code that uses ServiceLoader. Since this code is almost certainly non-OSGi you will have to turn it into an OSGi bundle by either wrapping it (put the original jar unmodified inside an OSGi wrapper bundle) or by modifying the MANIFEST.MF file to add things like Bundle-SymbolicName and Import-Package statements. Since you're modifying this file anyway, it's not a big deal to add an extra header in there to identify the bundle as a ServiceLoader-user. The header is called SPI-Consumer. The simplest form is:

  SPI-Consumer: *

Which means that this bundle is identified as an SPI consumer with no restrictions. This means that when code such as

  ServiceLoader ldr = ServiceLoader.load(MySPIProvider.class);

is encountered, something special will have to be done. What needs to be done is that the TCCL is set to the bundle class loader of the bundle that contains the implementation of MySPIProvider for the duration of the load() call. It needs to be the classloader of the bundle itself, so it has visibility of the private packages which are needed for ServiceLoader to do its business.

In my implementation, which is available in the Apache Aries project, the consumer byte-code is slightly modified so that for the duration of the ServiceLoader.load() call the TCCL is set to the classloader of the providing bundle(s).

How do we know that a bundle is providing an implementation for use with java.util.ServiceLoader? Similar to the consumer side, this code was most likely not written as an OSGi bundle, so it either needs to be wrapped or its MANIFEST.MF needs to be enhanced with Bundle-SymbolicName etc. Here we can identify the bundle as an SPI provider by adding:

  SPI-Provider: *

This means that all the implementations found in META-INF/services are considered as SPI providers. (You can also provide a subset by enumerating all the SPI names provided, instead of specifying *).

Specifying this header does two things:

It marks the bundle as an SPI provider for use with bundles that are marked as SPI consumer.
It registers the implementations found in META-INF/services in the OSGi Service Registry so that they can be used from there too.

Try it out!
Ok - let's try it out. For that you need:

An OSGi Framework - I'm going to use JBoss AS7.
An SPI Consumer, Provider and the SPI itself.
The Aries SPI-Fly project which implements the functionality associated with the SPI-Consumer and SPI-Provider Manifest headers.

Start the OSGi Framework
I'm taking JBoss AS7 which contains a fully compliant OSGi 4.2 Core Framework. Get version 7.0.1 from here: http://www.jboss.org/jbossas/downloads/
Download and unzip it, then run bin/standalone.sh (or standalone.bat):
...

11:49:45,016 INFO  [org.jboss.as.deployment] (MSC service thread 1-12) Started FileSystemDeploymentService for directory /Users/davidb/jboss/jboss-as-web-7.0.1.Final/standalone/deployments
11:49:45,028 INFO  [org.jboss.as] (Controller Boot Thread) JBoss AS 7.0.1.Final "Zap" started in 1671ms - Started 93 of 148 services (55 services are passive or on-demand)

BTW the AS7 appserver started up in 1671ms on my plain mac!

Get some example bundles
I'm going to use some of the example bundles from the Aries SPI-Fly project...

You need a SPI Consumer. For instance this one that uses the SPI from the Bundle Activator:

public class Activator implements BundleActivator {
    public void start(BundleContext context) throws Exception {
        ServiceLoader ldr = ServiceLoader.load(SPIProvider.class);
        for (SPIProvider spiObject : ldr) {
            System.out.println(spiObject.doit()); // invoke the SPI object
        }
    }

    public void stop(BundleContext context) throws Exception {}

}

I have a separate bundle that contains the SPI itself:
public abstract class SPIProvider {
public abstract String doit();
}
I used an abstract class here instead of an interface as that seems to be common practise with SPIs used with ServiceLoader.

And I have an SPI Provider bundle with an implementation:

package org.apache.aries.spifly.mysvc.impl2;
import org.apache.aries.spifly.mysvc.SPIProvider;

public class SPIProviderImpl extends SPIProvider {
    public String doit() {
        return "Doing it too!";
    }
}

The provider bundle has the SPI-Provider: * Manifest header in it and the consumer bundle has the SPI-Consumer: * header.
As the SPI-Fly project hasn't been released yet, these bundles can't be downloaded from Maven yet. You can build them yourself from here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-examples
You can also download them from these locations where I put my built artifacts:

Add OSGi-ServiceLoader support
The Apache Aries SPI Fly component provides two ways to add ServiceLoader support to an OSGi Framework:

Dynamically by using the OSGi 4.3 Weaving Hooks. With this you can install the above bundles as-is in the OSGi Framework and the treatment of the ServiceLoader.load() calls will happen on the fly.
Statically by transforming the bundle Jar file before installing it in the OSGi framework. This mechanism doesn't need the 4.3 Weaving Hooks.

As not all OSGi frameworks support the Weaving Hooks yet, I'll be showing the static mechanism here. If you want to try the dynamic mechanism see the Aries SPI-Fly docs for more info.
The static mechanism consists of a tool that transforms your bundles inserting the bytecode needed to set the TCCL when ServiceLoader.load() is called. You can build the tool yourself from here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-static-tool or download the copy that I built from here: spifly-static-tool.jar .

java -jar spifly-static-tool.jar consumer-bundle.jar
[SPI Fly Static Tool] Processing: consumer-bundle.jar
When it's finished you'll find a new bundle called consumer-bundle_spifly.jar in the same directory. This bundle has the ServiceLoader.load() calls treated so that the TCCL is set appropriately for it to work.

Use it at Runtime
Our transformed client bundle has an additional dependency on some SPI-fly classes which are used to appropriately set the TCCL once ServiceLoader.load() is called. This bundle can be found here: http://svn.apache.org/repos/asf/aries/trunk/spi-fly/spi-fly-static-bundle and I have created a pre-built version as well: spifly-static-bundle.jar .

Deploy them into your OSGi Framework. AS7 supports a variety of deployment mechanisms, via maven, AS7 Web Console or Command Line, Felix OSGi Console or JMX. For the sake of simplicity I'm using deployment by copying to the standalone/deployments folder here. Note that although deployment by copying is extremely simple, it does require attention to the order as the bundles found in the deployments folder are instantly started.

So I'm deploying:

spifly-static-bundle.jar (provides runtime support)
spi-bundle.jar (our SPI class)
provider-bundle.jar (our implementation)
consumer-bundle_spifly.jar (the one that we created from by running spifly-static-tool above)

On the console we'll see various log messages appear, including these from our SPI Consumer:

12:44:27,480 INFO  [stdout] (MSC service thread 1-3) *** Result from invoking the SPI directly: 
12:44:27,483 INFO  [stdout] (MSC service thread 1-3) Doing it too!

Which shows that our ServiceLoader Consumer bundle is now working property inside OSGi.

Conclusion
This example uses java.util.ServiceLoader provider and consumer implementations as-is in OSGi. Using the OSGi Service Registry is clearly a nicer solution, but you simply don't always have the option to change the code from using ServiceLoader to using the OSGi ServiceRegistry. That's where OSGi ServiceLoader support may come in useful.

It can already be used today in any compliant OSGi 4.2 Framework, but in that case consumer bundles need to be transformed before they are installed. If you have a 4.3 (or later) framework the transformation can happen on the fly which means that the whole process becomes more user-friendly: just install ServiceLoader support in the Framework and deploy your bundles - no static tool is needed in that case.

ServiceLoader support is being turned in an OSGi Specification scheduled to be part of the upcoming OSGi Enterprise Release (due early 2012). You can read more details about it in RFC 167 in the Early Access Draft here: http://www.osgi.org/download/osgi-early-draft-2011-05.pdf

Using OSGi in JBoss AS7

2010-11-11T07:20:00.000-08:00

Recently the first milestone of JBoss Application Server 7 has been released which now comes with OSGi support built-in!

OSGi Support in JBoss AS7 what does that mean?
It means that AS7 comes with an OSGi framework inside that you can use to deploy your OSGi bundles in. Besides providing standards-compliant OSGi functionality, the OSGi framework in AS7 will also provide integration with non-OSGi components (such as JavaEE ones) deployed in the Application Server. This is realized by the JBoss OSGi framework that was developed from the ground up with a specific focus to provide the best possible integration with AS7. The framework is mostly the work of Thomas Diesler, although I have also been involved in writing bits of it over the past few months...

Let's give it a go!

So if you're curious, here's a quick demo on how to develop a 'hello world' bundle using Eclipse and deploy it into AS7. But first start by downloading an AS7 build from here: http://www.jboss.org/jbossas/downloads. Unzip it somewhere on disk and start it up by running:

  bin/standalone.sh 

Develop your OSGi bundle

You can develop your bundle any way you like. For little projects or simply to get started I always use the Eclipse PDE that's part of the 'classic' distribution. For more serious projects I normally use maven with the Felix Bundle Plugin.

Here I simply follow the PDE wizard to develop my new TestBundle. Go File | New Project | Plug-in Project, specifying a standard OSGi Framework as the target platform:

Select the 'Hello OSGi Bundle' template:

And after hitting 'Finish' you've got a simple OSGi bundle, which has an Activator provided by the template:

For this posting I'm not making any changes to the OSGi bundle code, so let's just simply File | Export it as a deployable plugin:

This puts the OSGi bundle in the directory that you specified in the wizard. In my case the bundle is called TestBundle_1.0.0.jar.

Now deploy in JBoss AS7!

I started AS7 somewhere using the bin/standalone.sh script and it has started up:
...
13:47:22,633 INFO [org.jboss.as.deployment] (pool-1-thread-2) Started FileSystemDeploymentService for directory /home/davidb/temp/as7/jboss-7.0.0.Alpha1/standalone/deployments
13:47:22,636 INFO [org.jboss.as.server] (pool-1-thread-1) JBoss AS 7.0.0.Alpha1 "Halloween" started in 3627ms. - Services [Total: 150, On-demand: 7. Started: 143]
Ok, so it has started up. A little bit higher in the log you'll also see:
13:47:21,007 INFO [osgi] Activating OSGi Subsystem

Good - OSGi is there.
Now I copy my TestBundle_1.0.0.jar into the .../jboss-7.0.0.Alpha1/standalone/deployments folder and a moment later I can see the following message in the log:
13:52:24,116 INFO [stdout] (pool-1-thread-2) Hello World!!
This is the message printed by our OSGi Bundle Activator running in AS7!

Control the framework through JMX

The OSGi Framework in JBoss AS7 does not currently come with a console application - you can control it through JMX. So let's fire up jconsole, use service:jmx:rmi:///jndi/rmi://127.0.0.1:1090/jmxrmi to connect to the Remote Process:

In the MBean tab you will find the org.osgi tree which allows us to control the OSGi framework through a JMX API standardized in the Enterprise Expert Group. You can find the listBundles() API in the bundleState operations:

Yes, our TestBundle is there, and it has bundle ID 6.

Let's now uninstall it using the framework MBean:

Hit the uninstallBundle JMX API with the bundle ID (6) and you'll see that the bundle gets stopped and then uninstalled.
14:14:00,041 INFO [stdout] (RMI TCP Connection(13)-127.0.0.1) Goodbye World!!
14:14:00,057 INFO [org.jboss.osgi.framework.bundle.AbstractBundle] (RMI TCP Connection(13)-127.0.0.1) Bundle uninstalled: TestBundle:1.0.0
You can also control the OSGi framework through a webconsole, which can currently get installed via a separate installer, we're working on making the webconsole available in the AS7 distributable in a future milestone.

Conclusion

OSGi support is now available from the very first milestone of JBoss AS7. Give it a try!
If you have any questions, reach us on the forums: http://community.jboss.org/en/jbossosgi or via JIRA: https://jira.jboss.org/browse/JBOSGI

More in-depth details can be found in Thomas Diesler's blog post: http://jbossosgi.blogspot.com/2010/11/jboss-as7-osgi-integration.html

Eek! Java reflection can cause NoClassDefFoundError after 15 calls

2010-06-17T02:34:00.000-07:00

Now this came as a bit of a surprise to me. Reading this thread on the Felix mailing list...

I can't believe this: when using reflection after about 15 calls, the internal behaviour of the Sun JRE changes. With the changed behaviour comes in a package dependency on the sun.reflect package! When running under OSGi you are very much in charge of what comes in via the classloader so a new package that comes in over the same code path after about 15 calls is bad news - you really want to know your package dependencies up front.
The solution is obviously to import that sun.reflect package in your affected bundle but how are you supposed to know? Let's say you have a test suite that properly tests your code and everything is looking nice. I would imagine that a testsuite doesn't test every method call 15 times to make sure its still ok. Moreover, how do we know it's 15? There could be another optimization lurking in the JRE that requires 300 calls in order for it to be exposed.
Another problem with importing sun.reflect is that it's implementation specific. Do other implementations have a similar issue?

Here's a message to all JVM implementors: optimize all you can but please don't change the package dependency graph after a number of invocations. At the very least make your implementation fail fast, not after x method calls.

There is a bug for this at sun: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6265952
Accepted in 2005 but still open...

Altering OSGi Service Lookups with Service Registry Hooks

2009-11-18T03:28:00.000-08:00

OSGi Services are great. They can solve many problems very elegantly. Take for instance SPI patterns. SPI patterns are used to make implementations pluggable. This type of pattern outside OSGi often comes with a bunch of external configuration (e.g. in META-INF/services) that makes it hard to manage and gives you a once-off chance to choose your implementation that you're stuck with for the rest of the VM lifecycle. OSGi Services provide a much more elegant way to solve this.

In many cases clients simply want to use some functionality and don't particularly care how that functionality was created. When using OSGi Services the functionality is looked up in the OSGi Service Registry which is a directory of these. Services can be looked up by implemented interface (like a phone book) or by provided attribute (like the yellow pages). How the service got there is not interesting to the client and he's not involved in that. What if the service gets replaced while the client is active? No problem, the OSGi Service Programming model is actually built around this dynamicity. You don't need to stop service consumers when you are replacing or updating the service, they are automatically rebound.

The Service Registry also provides us with an attribute based selection mechanism which is very nice if there are multiple implementations to choose from. As an example, take a system where every available printer is represented in the Service Registry as a Service that implements the org.acme.Printer interface. Each printer will have additional properties registered that help with the selection:

When looking for a Printer OSGi allows you to use an LDAP-style filter to select the printer you want. So if you want a printer that can do A3 you would use this LDAP filter:

(&(objectClass=org.acme.Printer)(paper-size=A3))

or if you want a printer in a location that starts with 'b' you do this:

(&(objectClass=org.acme.Printer)(location=b*))

This is all great and you can build your system on this using either plain OSGi code or component frameworks such as OSGi Blueprint or OSGi Declarative Services. However sometimes you may want to tweak the properties at a later date without having to rewrite your system. Assume that in your organisation all of a sudden the office numbering has changed, which would really mean that you'd have to update the 'location' attribute of all the printers. Or maybe you want to add some additional metadata to existing printers that influence the selection process, like their energy rating.

Service Registry Hooks provide the building blocks that make this possible. I wrote a little bundle called ServiceJockey that uses these to manipulate service registrations and how they are looked up. BTW you can find details at the end of this posting about getting ServiceJockey, which is open source and available freely under the Apache License.

Service Registry Hooks: what are they?

One of the driving factors for Service Registry Hooks (an OSGi standard introduced in the 4.2 Core specification) was the Remote Services (Distributed OSGi) work done in the OSGi Alliance. One of the things that implementations of the Remote Service spec need to know is what kind of services consumers are looking for so that they can go out to a discovery system to see if it might be available remotely. Eagerly registering all available remote services is clearly not scalable so we need a smarter mechanism to allow for transparent on demand discovery of remote services. This is what the ListenerHook and FindHook provide us. Together they allow us to find out what service consumers are requesting making us to only look in a remote Discovery system for those that are relevant to the current framework.

Another problem was frequently brought up. What if you have an existing bundle that doesn't know anything about Remote Services? What is the default behaviour? Should all service lookups all of a sudden always include remote services? The Remote Services spec does include a special property that you can add to your filter to influence this (service.imported) but you clearly don't want to break up all your existing service consumers to add this extra property to their lookup filters.

It turned out that there wasn't really a valid answer to that question applicable to all cases. If you're working on a system only using Remote Services for a select set of services it may make sense to default all other services consumers to not use Remote Services. On the other hand, maybe if you're building a Cloud-based infrastructure where services can move freely from one container to another the default behaviour could be that you do allow remote services for just about anything.

Since there isn't really a one-size fits-all here the Service Registry Hooks make it possible to provide a policy for this problem in a separate bundle. You can create EventHooks and FindHooks to influence what services consumers can see. This effectively allows you to add extra conditions to the lookup of service consumers without the need to modify those consumers.

Finally, EventHooks and FindHooks, together with the a ServiceListener, allow you to proxy Service Registrations, where you provide a second registration of the same object with modified properties, hiding the original.

The ability to provide an alternate registration on the fly and the possibility to impose extra conditions on existing service consumer lookups is what I built ServiceJockey around.

ServiceJockey doesn't replace the Service Object, it only effectively replaces the registration in the Service Registry plus it can put additional constraints on client lookups. But you could go further. If you wanted to actually replace or shadow the real service object and do some work when somebody uses it (maybe log it or something) you could put an additional object between the consumer and the original service object.

Because the Service Registry Hooks effectively modify some basic behaviour of the framework (the visibility of Services) they should really be started early in the Framework lifecycle. You can achieve that by giving them a low start level.

The Service Registry Hooks are available in Felix 1.8.0 and Equinox 3.5 and newer versions of these.

Altering Service Registrations

Let's start with a bundle that consumes (uses) a Printer. It isn't interested in which printer, as long as it can print A4:

  BundleContext context = ... // from Activator.start()

  Filter filter = context.createFilter(

  "(&(objectClass=org.acme.Printer)(paper-size=A4))");         

st = new ServiceTracker(context, filter, null) {

  public Object addingService(ServiceReference ref) {

    // print out some information on the printer

    // or use the printer

    return super.addingService(ref);

}

};

  st.open();

When I run this it's reporting both printers that I have in my system:

Printer:

objectClass: [org.acme.Printer]

service.id: 27

name: p1

location: b283

capabilities: [Double-sided]

paper-size: [A3, A4]

Printer:

objectClass: [org.acme.Printer]

service.id: 28

name: p7

location: a12

capabilities: [Colour, Staple]

paper-size: [A4, Letter]

So the client has arbitrary access to both of them. Now lets see if you can modify the client behaviour so that it only uses a printer with an Energy Rating < 50.

But hold on, we don't even know about energy ratings yet! We need to add this information to the Printer registration without changing the bundle(s) that register the Printers. Let's user Service Jockey to do this.

Service Jockey uses the OSGi Extender Model. This means that it is driven from a data file in a bundle.

So I've got a bundle (called PrinterJockey) that contains META-INF/sj.xml:

<service-jockey xmlns="http://www.coderthoughts.org/schemas/sj/v1.0.0">

 <proxy-registration>

  <rule>

   <service-filter>(&(objectClass=org.acme.Printer)(name=p1))</service-filter>

   <add-property key="energy-rating">75</add-property>

  </rule>

  <rule>

   <service-filter>(&(objectClass=org.acme.Printer)(name=p7))</service-filter>

        <add-property key="energy-rating">50</add-property>

  </rule>

 </proxy-registration>

</service-jockey>

I will have to tell the Service-Jockey that my bundle contains configuration for it, for that I'm adding a header to the Manifest:

Service-Jockey: META-INF/sj.xml

Lets run the Printer Consumer bundle together with Service Jockey and my 'Printer Jockey' configuration bundle that contains the sj.xml file:

id    State   Level Bundle

0 ACTIVE   0     org.eclipse.osgi_3.5.1.R35x_v20090827

1 ACTIVE   1     org.coderthoughts.servicejockey_0.0.1

2 ACTIVE   1     PrinterJockey_1.0.0

3 ACTIVE  10    Printers_1.0.0

In my case the Printers bundle both registers and consumes the Printers, but that's because its a little test bundle. It's not really relevant. Note that the Printers bundle has a higher start level than the Jockey ones and therefore starts later.

When I run my system again I can see that it's doing some work. Now my client reports these services:

Printer:

objectClass: [org.acme.Printer]

service.id: 30

name: p1

location: b283

capabilities: [Double-sided]

paper-size: [A3, A4]

energy-rating: 75

.ServiceJockey: Proxied

Printer:

objectClass: [org.acme.Printer]

service.id: 32

name: p7

location: a12

capabilities: [Colour, Staple]

paper-size: [A4, Letter]

energy-rating: 50

.ServiceJockey: Proxied

Aha! I've got my energy rating in there! That's good. Remember that these are alternative registrations. The original ones haven't changed, they're just hidden.

Altering Client Side Lookups

Now I want my client to only use the one that is rated <= 50.

That's done by adding another rule to the Service Jockey configuration. A rule that adds an extra filter to any matching lookup in a consumer.

<service-jockey xmlns="http://www.coderthoughts.org/schemas/sj/v1.0.0">

 <restrict-visibility>

     <rule>

   <bsn>.*</bsn>

   <service-filter>(objectClass=org.acme.Printer)</service-filter>  

   <add-filter>(energy-rating<=50)</add-filter>

  </rule>

 </restrict-visibility>

 <proxy-registration>

...

 </proxy-registration>

</service-jockey>

You can specify a regular expression to say what the Bundle Symbolic Name of the consumer bundle(s) is that the rule applies to. In my case .* matches anything. So it applies to any bundle that has OSGi Service consumers. By the way, there's also a tag.

You specify to what services the additional filter applies with the

tag and you specify the additional filter in

. In my case I'm adding the (energy-rating<=50) condition to any Printer Service returned to a consumer (the condition looks a bit dodgy because of the XML escaping of the '<' sign - you could use a CDATA section instead...).

Note that the Service Filter applies to any service returned to a service consumer, regardless of the filter used by the client itself looks like.

Let's run the client again:

Printer:

objectClass: [org.acme.Printer]

service.id: 32

name: p7

capabilities: [Colour, Staple]

location: a12

paper-size: [A4, Letter]

energy-rating: 50

.ServiceJockey: Proxied

Bingo - it only selects the service I wanted it to select. Based on additional properties and criteria listed in my Service Jockey configuration file. I did not have to modify the Printer Service registration bundle or the Consumer bundle...

Is it overkill?

Seems like a lot of work for just adding a service property, is there not a lighter way to at least add values to Service Registrations? Well at least that was my initial impression. However looking at the ServiceJockey bundle, it's only 14kb. Besides that there is currently no other way to achieve this...

Service Jockey - ride your service interactions

I didn't show any of the code to actually use the EventHook and the FindHook. Have a look at the HidingEventHook and HidingFindHook source code for that.

You can check out the source (Apache License) as an Eclipse Project from SVN here. If you fancy modifying the code, I would recommend you also get the tests project and add new ones. The test bundle depend on the Mockito bundle for its mock objects...

I'm sure the Service Jockey isn't perfect, because I only wrote it during an airplane flight so feel free to send patches :)

I also put a binary download of ServiceJockey here.

Also in the source tree, Eclipse projects for the Printers and PrinterJockey example bundles.

A single Calendar from Desktop to Phone

2009-10-06T17:33:00.000-07:00

This year I upgraded the really old smartphone I had to a Sony Ericsson C905. It almost looks like I'm one of the last people left who doesn't have an iPhone and doesn't want one (I'm resisting the voluntary lock-in ;).
I'm really happy with the C905 because it has all the features I wanted (Wifi, GPS, 8MP camera, loads of memory) but it still looks like a phone and it's not too big.

One problem I was starting to run into was that I had too many calendars going. I had the calendar in my email client (Thunderbird + Lightning), I had my Google Calendar and now the phone also has a calendar. I really wanted a single calendar that works on all of my devices. I had already been playing with the Google Calendar Provider for Lightning and quite liked it. If only there was something like that for the phone...

After a little digging I found out that Google supports the Exchange ActiveSync protocol. And my Sony Ericsson (and many other models too) supports it as well! It's not very well documented so I decided to write down the steps I took to get it all working.

1. Go to the Organiser and select Synchronization.

2. Create a new Account.

3. Select Exchange ActiveSync and give it a name, I called mine 'Google'.

4. In the General tab, enter as the server address: https://m.google.com also provide your Google user name and password. I didn't provide any other information here.

5. On the applications tab I only selected Calendar, although the Contancts and Email integration probably also work.

6. Now it asks me to delete all my contacts and calendar items. Very annoying because I don't want to sync my contacts, but anyway, for best results you have to clean them all. So I backed up my contacts previously and select clean here. I can restore my contacts later...

Ok - I'm all set up!
Let's create a calendar event on the phone:

Synchronize with Google...

I think you can even do this automatically, but I haven't enabled that because I don't want my phone bill to explode :)

Let's have a look at Google mail:

Yep, it's there!
Now let's see in Thunderbird Lightning:

And in all three modes I have read/write access. Aaaaah, nice!

An IRC alerter written using Apache Camel and Java

2009-07-07T02:38:00.000-07:00

IRC is a great tool if you have a bunch of people working on the same thing in different locations. One of the biggest problems I found with it is that I sometimes miss pings. I don’t always have the IRC window open, and although the client I use most (Trillian) has nice popups in the corner, I still sometimes miss a ping, simply because I’m away from my computer. I know there are some other IRC clients that can send you special alerts, but I just happen to like Trillian, so I didn’t want to abandon it just because I was missing one little thing...

While being annoyed with this for a little while, I came across the Apache Camel IRC component. Great - that should sort me out. I’ll just simply write a Camel route that connects to the IRC channel and sends me an alert of some sort when someone pings me.

After running the Camel Maven archetype for Java projects:
>mvn archetype:generate -DarchetypeGroupId=org.apache.camel.archetypes -DarchetypeArtifactId=camel-archetype-java -DarchetypeVersion=2.0-SNAPSHOT -DarchetypeRepository=https://repository.apache.org/content/groups/snapshots-group
I was able to get this going fairly quickly! The archetype creates a Maven Camel project for you with an example route in it. You can run it now with:

mvn install camel:run

Next step: write a Camel route that reads my IRC messages for me.
For this I’m simply using a URL in Camel to connect to the IRC server, e.g. I’m using the following to connect to the CXF IRC channel at irc.codehaus.org:
irc://my_user@irc.codehaus.org/#cxf?nickname=my_user
Once logged in you simply receive all the messages sent over the channel, in the process() method of your Camel route, so I can configure my route something like this:
public void configure() {
  String url =
    "irc://my_user@irc.codehaus.org/#cxf?nickname=my_user";
  from(url).process(new Processor() {
    public void process(Exchange ex) throws Exception {
      String msg = ex.getIn().toString();
      System.out.println(msg);
    }
  }
}
The process() callback gets called whenever a message is sent over the IRC channel, so now we can do something with it!

This is where I started experimenting :) I wrote a little generic IRC client that can connect to any number of IRC channels and searches any messages coming in for a certain text string.
If there is a match, it opens a dialog on my screen that will stay there until I click it away, now I don’t miss it any more! I’m also using the new Java 6 tray icon functionality, to show a little popup balloon.

I put the code (under the Apache license) in SVN over here: http://coderthoughts.googlecode.com/svn/trunk/irc-alerter. No built executables yet, but you can simply check it out and run it with the following command:
mvn install camel:run -Dirc.alerter.channels=c:\path\to\channels.xml
since it’s pure java, it should work on mac, linux etc too :) I used Maven 2.0.9 and Java 6.

The format of the channels.xml file is as follows:

<channels>
<match string="davidb"/> 
<password file="c:/davidb/etc/camel-irc.pwd"/>
<channel name="cxf" user="my_user" nick="my_user" host="irc.codehaus.org"/>
<channel name="somechannel" user="davidb" nick="davidb_camel" host="some_host" port="6667"/>
</channels>
So this one matches any message that contains 'davidb'. Obviously you’d put your match in there.
For each channel you specify the channel name, username, nickname, hostname and optionally the port. You probably want to use a different nick name than the one you use from your ordinary IRC client.
The passwords for the various servers are in a separate file, which you can put in an encrypted directory in your file system. These are properties files, your camel-irc.pwd file could look like this:

my_user@irc.codehaus.org=somepass
someuser@somehost=someotherpass

This has been a lot of fun to do. I’m still amazed how easy it was to create this app using Camel, and I’ll never miss an IRC ping ever again! (Well, I hope).

Source code - can be checked out from Subversion here: http://coderthoughts.googlecode.com/svn/trunk/irc-alerter

Questions from the RFC 119 webinar

2009-05-08T02:33:00.000-07:00

There were quite a number of questions on the webinar and I didn't get to answer all of them, so I'm providing some answers here. For those of you who missed the webinar, you can still view the recorded version of the Distributed OSGi webinar here.
The source code of the GreeterDemo that I used during the demo session can be found in the Apache CXF demos SVN here: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/greeter for details on how to run the demo see the demo walkthrough page on the CXF Distributed OSGi website.

Below are the questions asked during the webinar, and an attempt to answer them. Feel free to send comments if I missed anything…

Q: Does it use RMI?
A: The wire-protocol and data-binding used are not specified by Distributed OSGi. They are up to the implementation. The Apache CXF implementation used in the demo uses SOAP/HTTP but other implementations could be written to use other wire protocols, such as RMI.

Q: Why don't you use JaxWS?
A: It's all about choice for the developer. JaxWS is a specific programming model for Web Services in Java. Distributed OSGi is for those who like the OSGi Services programming model, and would like to stay within that programming model to distribute OSGi services. It's not incompatible with other programming models, such as JaxWS, JaxRS, CORBA etc and can be used alongside them.

Q: Can I used Distributed OSGi services from Visual Basic?
A: Yes you can! Visual Basic programs can invoke on Web Services that are defined by WSDL. If you use the Apache CXF Distributed OSGi implementation it will generate a WSDL for your OSGi service which you can use from your Visual Basic client.

Q: Is there an implementation of Distributed OSGi available in Open Source?
A: Yes, the Apache CXF implementation, Eclipse ECF implementation and Apache Tuscany implementation are all open source and freely available to everyone.

Q: Can I use Distributed OSGi services from an Ajax browser client?
A: Yes, this is possible. I wrote an example of this on this blog a little while ago, see here: http://coderthoughts.blogspot.com/2009/02/distributed-osgi-powered-ajax-webapp.html.

Q: Can i use cxf with equinox, specifically pre 3.5?
A: The short answer is no, but you can use the current betas of Equinox 3.5 (builds since December). Any implementation of Distributed OSGi will need the Service Registry Hooks. Since this is a new API, it is only available in Equinox 3.5 and Apache Felix 1.4.1 or newer.

Q: Why is it based on webservices?
A: The design of Distributed OSGi is not based on Web Services and you may find implementations that don't use web services at all. The Apache CXF implementation is based on Web Services. Web services provide a standards-based approach for distributing software and allow integration with other (non-OSGi) systems. Although not required per se by an implementation of RFC 119, ‘Legacy integration' is one of the use cases that drove the design of Distributed OSGi, and by using Web Services this use-case can be addressed.

Q: You are using CXF for cross-jvm/osgi communication and mentioned that the rfc does not define the communication protocol. Does that mean that it is possible to implement a custom communication protocol?
A: Absolutely!

Q: How can I specify which protocol(s) to be used for some remote service?
A: Use intents mechanism for that. The intents mechanism allows you to declaratively specify constraints on the remote service. In short, you can put a property on the service that you want to remote that declares using what binding/protocol you the service should be exposed. So you could do something like: osgi.remote.requires.intents=JMS which means that the service should only be exposed over JMS. If the Distribution Software (DSW) can't satisfy the intent, it will not be able to expose the service. Although some values (for SOAP and JMS) have been defined in the RFC for this property, the actual possible values depend on the Distribution Software implementation used.

Q: How do I set the discovery service dynamically without the remote-services.xml?
A: Use an RFC 119 compliant Discovery implementation, these don't use the remote-services.xml files. We're currently working on one in the Apache CXF codebase that is based on Apache Zookeeper, but other implementations, such as the one based on SLP at Eclipse ECF are also available.

Q: Can I specify more than one protocol for one service? For example Web Services, CORBA, RMI, etc.
A: If you specify the protocol to use using the osgi.remote.requires.intents mechanism as described above, you can only specify one. E.g. specifying that the service has to satisfy both the RMI and CORBA intent at the same time is a contradiction, however you can register the same POJO twice with the OSGi Service registry, once for each protocol.

Q: Is Discovery service a separate OSGi container?
The RFC 119 Discovery architecture typically uses a centralized Discovery Server (potentially a replicated one, like we have with the CXF/Zookeeper implementation). The Discovery Service in the OSGi container provides a standardized interface to such a Discovery Server. The Distribution Software (DSW) interacts with the Discovery Service in the local OSGi service registry, which then communicates with the centralized Discovery Server.

Q: Service Registry hooks- limiting visibility of service-This is similar to access control the service- what are your views on this? Thanks
A: I forwarded this question to BJ Hargrave from IBM, the main author of the Service Registry Hooks RFC, he provided the following answer: "It is more about controlling visibility to services (but I guess some people may call that access [see http://blog.bjhargrave.com/2009/03/i-am-visible-but-am-i-accessible.html]). But the primary purpose is not to secure the system since without a Security Manager installed, there is no actual security. Its primary purpose is to control service visibility to influence the services a bundle sees so that it selects a desired service rather than another service. For example, the proxy for X rather than X. "

Q: so if they [OSGi Services] don't implement any signature, how are they identified within the registry?
A: OSGi Services don't need to implement a particular OSGi signature or interface, but they would typically implement an interface that relates to their purpose. In the presentation I mentioned a Purchase Service, which implements a foo.bar.PurchaseService interface. Normally services are looked up via their interface, but you can also look up services in the OSGi Service Registry purely on other properties that they might have.

Q: When you say 'Dynamically come and go', can you elaborate on that what does that really mean?
A: Example: A running OSGi system. A consumer in that system might be using a service that satisfies its lookup criteria. While the system is running another service that also satisfies these criteria could arrive (either because a new bundle that creates this service is installed or because an existing bundle registers a new service). Depending on the consumer's requirements, it could now potentially be using both services, the old and the new one. At a later time one of the services might go away at which point the consumer will only be using the one that's left. All (properly developed) OSGi service consumers are capable of dealing with dynamics like this.

Q: is the WSDL, automatically generated by some tool?
A: Yes, the Apache CXF DOSGi implementation does this dynamically at runtime, based to the Java interface of the service. You don't have to run a tool for this with CXF, although other implementations might require the running of a tool to achieve something like this.

Q: What does pojo mean?
A: Plain Old Java Object.

Q: I suppose that the parameters of the service exposed are of type 'primitive' like int, string, double, DateTime but cannot be of type by example SessionFactory ?
A: It depends on the DOSGi implementation used. If you want your code to be suitable for a number of DOSGi implementations it's best to restrict the use of data types to primitives, collections and arrays. The CXF DOSGi implementation also supports classes and interfaces as part of the signature as long as they in turn are defined in terms of the data types mentioned.

Q: Can you highlight the security capabilities that are provided by distributed OSGi?
and Q: Can we secure the communication between remote OSGI servers (and authenticate, authorise access to the services)?
A: Security is a Quality of Service that DOSGi implementations could provide. The intents mechanism can be used to only select remote services that are secure, e.g. by requesting the ‘confidentiality' intent, or by requesting a custom intent (such as AcmeSecurity) that your deployer defines with the appropriate security configuration. For more information see chapter 5.5.3 of RFC 119 (in http://www.osgi.org/download/osgi-4.2-early-draft3.pdf).

Q: How do local discovery services know about other discovery services?
A: The Discovery Service registered in the OSGi container communicates with a centralized Discovery Server. This Discovery Service needs to be configured with the connection details of the central discovery server.

Q: what implementations of distributed OSGi will be available in Equinox 3.5? Anything faster than webservices?
A: Equinox 3.5 does not have any Distributed OSGi implementations, but Eclipse 3.5 provides the ECF implementation. I guess I'm wondering what really the performance difference is that you are looking for. Have you measured the performance of CXF webservices versus something else? Could you provide us with some target figures? I know that CXF performs pretty well…

Q: Can you specify multiple instance endpoints for the same service?
A: On the server side one remote service typically maps to one remote endpoint (although exceptions are possible, especially in case the service implements multiple interfaces some implementations will give you one endpoint per interface). If you want more endpoints you can always register more OSGi services for the same POJO (although I'm not sure what the use-case for that would be).
On the consumer side, the consumer can be informed of multiple remote endpoints satisfying the OSGi Service Lookup criteria, in which case the consumer could get multiple client-side proxies to the remote services.

Q: Can you talk more about discovery with ZooKeeper and how that would work?
A: In the CXF project I started implementing the RFC 119 Discovery implementation using Apache ZooKeeper as the underlying Discovery mechanism. ZooKeeper is based on a replicated central server that holds a virtual file system. Consumers connecting to the ZooKeeper server will all view the same filesystem which holds the discovery information. The way this works in practice is that you have your cluster (1 or more) of zookeeper servers running. The CXF Discovery implementation bundle is installed in your OSGi containers. It will communicate with the ZooKeeper server for the discovery information. The net effect is that the remote service consumers will see the remote services without needing any extra configuration.

Q: Question : Is there other org.osgi.remote.type foreseen (e.g. POJO) ? What is the particularity of the type 'POJO' except its name ?
A: The POJO configuration type is specific to CXF. There is one configuration type specified in RFC 119: SCA, which is optional. Using SCA as a way to configure Distributed OSGi will give you portability on the configuration level. CXF does not implement the SCA configuration type yet…

Q: Question : Will it be possible using 'blueprint or spring DM' to integrate remote OSGI with Spring XML file ?
A: Yes - there is a demo that comes with the CXF DOSGi implementation that uses Spring-DM to create a remote OSGi service and on the consumer side it uses Spring-DM to create the consumer. Distributed OSGi is orthogonal to any OSGi component model and should therefore work fine with any of them. See here for the demo code: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/spring_dm

Q: is it possible to load balance across distributed OSGi bundles?
A: Yes – if multiple remote instances exist the consumer could be informed about all of them: the consumer will get a service proxy for each remote instance. The consumer can use these to implement some load-balancing algorithm.

Q: Why does a copy of the interface and class have to live on the client?
A: Only the interface needs to be available to the service consumer. The implementation class does not. I think this is a very natural thing to do. If I write a consumer to a service, like a PurchaseService, I need to know the interface to that PurchaseService, how am I otherwise going to use this in a meaningful way? The interface is the contract between the consumer and the service provider. This is also the case for local OSGi Services. They can't use an OSGi service if they don't have access to the interface.

Q: It seems like you are cheating if you get a copy of the interface and class in your client code prior to starting the demo.
A: No I'm not cheating, only the interface is present in the consumer. When I run the consumer I can see that the value I'm entering in the consumer running in Felix is echoed in the Equinox server window. The implementation resides in Equinox.

Q: Does distributed OSGi support multiple instance of the same service and thus software based failover rof services across remote services?
A: Yes, see above.

Q: for properties is there a namespace type construct to prevent collisions? or is it inherent based on the scope of a service?
A: No there isn't really a namespace (although the Distributed OSGi properties use the osgi. prefix), but it is good practice to qualify your own properties with your own organization just like a Java package, e.g. com.acme.myproperty to avoid clashes.

Q: Are there standard properties of services, like names, that every OSGI service should define?
A: No. Its absolutely fine to register a service with no properties. OSGi will typically add a few properties, like service.id and objectClass but as a service provider you don't need to worry about these.

Q: How does a Discovery service, choose if the service is deployed both locally and remotely?
A: If a good match is available locally, DOSGi will not look for remote services. If you want to influence your match, forcing either a local or a remote match you can refine your lookup by adding extra criteria in the LDAP filter.

Q: Can the contract by dynamically found and transmitted remotely?
A: You could write a system to do this, as bundles can be installed at any time in a running OSGi system, but the Distributed OSGi standard doesn't define this.

Q: With multiple instances of the same service, would they have the same endpoint, or could they have different endpoints? How would a discovery pick which one?
A: The will have different endpoints. Everything being equal Discovery will simply inform you about all of them. If your consumer only needs to use one of them, which one the consumer will get is undefined. If the consumer has a preference for a particular one it needs to refine its filter to only match that one. You can also have the consumer receive proxies to all remote instances.

Q: Can you turn existing code (POJOs) into distributed OSGI services just using configuration (i.e. no java re-coding)?
A: Yes, OSGi Blueprint/Spring-DM provide a lot of support here. You configure them with an XML file that will instantiate your POJO and register it as an OSGi service.

Q: Are you familiar with the Paremus fabric, distributed OSGi runtime that uses SCA as the "wire-up" model? If so, does it use SCA in the way you described?
A: I'm not really familiar with this product, but I have seen demos of it. The SCA part of RFC 119 is really about defining qualities of service for the distributed services. It's not used for wiring services, the OSGi Service registry is used for that.

Q: Will both local and remote services be returned from a search?
A: There is no difference in the API to use to obtain local services or remote services. However, if appropriate local matches are available, the system won't look for remote services. If you want all local and remote matches you can do lookups with and without the osgi.remote property. The osgi.remote property is only set on proxies for remote services.

Q: Are there any drawbacks, using OSGI distributed services that you may know of?
A: None :) Seriously, yes remote computing is not the same as local computing. There are additional failure scenarios, additional latency, additional security impacts and so on. So if your system works perfect with all your services locally, keep it that way. However, if you want to distribute your OSGi services, then RFC 119 is a good way to do it!

Webinar and demo of Distributed OSGi (RFC 119) on Tue April 28

2009-04-21T07:49:00.000-07:00

Just to let y'all know... I'll be doing a webinar and demo of Distributed Services OSGi on Tuesday April 28.
Distributed OSGi is a major new feature in the OSGi 4.2 specification. For more info, see http://fusesource.com/resources/video-archived-webinars

The demo will be done using the Apache CXF Reference Implementation.
Registration and attendance is free :)

A Distributed OSGi Powered AJAX WebApp

2009-02-09T08:00:00.000-08:00

A pretty cool new way of using Distributed OSGi services is from a non-OSGi environment. In the previous post I used an OSGi AuctionService from an OSGi AuctionConsumer in a different VM.
This posting is about using that OSGi service from a non-OSGi consumer. From an AJAX-powered web application that's running in the browser!

I'm writing the webapp using the Google Web Toolkit (GWT). Both the webapp as well as my OSGi service are hosted in the same OSGi container. With AJAX you can create a rich client application running inside the browser. So no Servlet/JSP style server-side execution. The webapp runs on the client machine. GWT makes creating the AJAX app easy: you write them in plain old Java. No need to sidestep into Javascript as GWT turns the Java into a Javascript AJAX app and all (well, almost) of the browser specific madness is taken care of by GWT too.

As with any website, the end-users users simply open it in their browser. No client install, no browser plugins. Any modern browser supports this stuff natively. Remote communication, such as obtaining a list of available items, listing a new item, placing a bid, is done via SOAP/HTTP calls directly on my OSGi AuctionService (the one from the previous posting) which has been remoted using the Apache CXF-based Distributed OSGi Reference Implementation.

To do all this I'm adding a component called Pax Web Extender to my OSGi container, on top of the Distributed OSGi bundles. Pax Web Extender turns the OSGi container into a web container which means that I can deploy a .WAR file into it. It internally leverages the OSGi HTTP Service. (Spring DM Server can do this too BTW).

My OSGi container is configured like this:

Besides the CXF Distributed OSGi and Pax-Web-Extender Bundles I have my AuctionService and AuctionInterface bundles deployed (see previous posting for the details), plus the .WAR file that holds my AJAX webapp.

The reason I can invoke my OSGi Service is because it was made available remotely over a standards-based mechanism. Distributed OSGi itself doesn't prescribe how you do your remoting, it only says that you can use standards-based wire protocols and bindings if you wish and that this can make OSGi Services available to non-OSGi consumers.

The CXF implementation of Distributed OSGi turns an OSGi Service into a Web Service that is accessible via SOAP/HTTP. GWT can already natively make remote HTTP invocations. I didn't actually even care looking for a SOAP library for GWT. SOAP is just XML and GWT has some pretty good XML parsing functionality.
The last thing that I need to worry about (besides security, but that's a separate topic) is the browser's single-origin policy. But hey: I've deployed my webapp in the same runtime as my OSGi Service so as long as I make them available over the same port there's no problem! I achieved this by sharing the same OSGi HTTP Service by both the Distributed OSGi runtime and the Pax Web Extender. Thanks to the OSGi Services architecture!

So I'm taking the AuctionService of the previous posting, but first of all I want to expose it on a slightly simpler URL. The default URL of http://localhost:9000/org/coderthoughts/auction/AuctionService is just a bit too long, let's use /auction as the context instead. I also want to make it use the OSGi HTTP Service (which CXF doesn't do by default). I can do both in one shot by setting the osgi.remote.configuration.pojo.httpservice.context service property to the value /auction. The HTTP Service that comes with CXF runs by default on port 8080 (you can change this by setting the org.osgi.service.http.port system property) so my Web Service now runs on http://localhost:8080/auction. Change the activator that registers the service to the following:
public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    props.put("osgi.remote.configuration.type", "pojo");
    props.put("osgi.remote.configuration.pojo.httpservice.context",
      "/auction");
    sr = context.registerService(AuctionService.class.getName(),
      new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}
To double check that the service is operational on the new address, simply open http://localhost:8080/auction?wsdl and see if the WSDL appears again (just like in the previous posting).

I won't go into all the detail of writing the GWT app here. Look at the code in the SVN project for all of that.
I constructed my GWT application the usual way by creating a starting point with the projectCreator and applicationCreator scripts that come with GWT. This creates a nice Eclipse project skeleton for GWT from which I can start hacking my AJAX site in Plain Old Java :)

All you need to do in the GWT app to invoke on the Distributed OSGi service is:

figure out where it's running, since the Web Service is running on the same host and port as where the AJAX app is served from on the /auction context, I can construct the Web Service URL as follows:
webServiceURL = "http://" + Window.Location.getHost() + "/auction/";
create a SOAP message, just a bit of XML really.
send it via a HTTP POST request to the WebService using a com.google.gwt.http.client.RequestBuilder that comes with GWT.
register a callback object to process the SOAP (XML) response.

All of this happens in the AuctionSite class which is the webapp's main class. There are a few more classes such as data classes and a few dialogs. Check out the project from SVN to see them all...

To prepare my webapp (.WAR file) for running with the Pax Web Extender I'm turning it into an OSGi bundle by adding the following META-INF/MANIFEST.MF file:
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-SymbolicName: AuctionSite
Bundle-Version: 1.0.0
Webapp-Context: auctionsite
The Webapp-Context is important as this tells the Pax Web Extender where to register the webapp.

I also have to add a WEB-INF/web.xml file to the .WAR as this is the signature file that the Pax-Web Extender needs in order to spot a bundle as a webapp. However, I don't really need to put anything in it. My webapp consists only of static files (no servlets etc.) as far as the webserver is concerned, so my web.xml is simply:
<web-app>
<display-name>AuctionSite</display-name>
</web-app>

GWT comes with scrips that you run to turn your Java into AJAX.
I wrote a tiny little ant script that calls the GWT script and then creates a .WAR file from my GWT project in Eclipse.

Now lets boot is all up! I'm using Equinox 3.5M5 with the Multi Bundle Distribution of CXF-DOSGi.

To install the Multi Bundle Distribution of CXF-DOSGi, download it from here. Then unzip it in your Felix/Equinox installation dir.
Take the contents of the felix.config.properties.append or equinox.config.ini.append and append it to the configuration file of your OSGi container.
This will automatically load all the DOSGi bundles when starting up the OSGi container.

Start Equinox with the DOSGi Multi Bundle configuration to make it load all the DOSGi bundles at startup.
Then install and start the PAX Web Extender from here and your AuctionIntf and AuctionImpl bundles.
Finally install and start the AuctionSite.war file, just like any other bundle in OSGi.

You should now have the following bundles running in Equinox:
osgi> ss
Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20090127-1630
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE org.apache.geronimo.specs.geronimo-annotation_1.0_spec_1.1.1
3 ACTIVE org.apache.geronimo.specs.geronimo-activation_1.1_spec_1.0.2
4 ACTIVE org.apache.geronimo.specs.geronimo-javamail_1.4_spec_1.2.0
5 ACTIVE org.apache.geronimo.specs.geronimo-ws-metadata_2.0_spec_1.1.2
6 ACTIVE com.springsource.org.apache.commons.logging_1.1.1
7 ACTIVE com.springsource.org.jdom_1.0.0
8 ACTIVE org.springframework.bundle.spring.core_2.5.5
9 ACTIVE org.springframework.bundle.spring.beans_2.5.5
10 ACTIVE org.springframework.bundle.spring.context_2.5.5
11 ACTIVE com.springsource.org.aopalliance_1.0.0
12 ACTIVE org.springframework.bundle.spring.aop_2.5.5
13 ACTIVE org.springframework.bundle.osgi.io_1.1.2
14 ACTIVE org.springframework.bundle.osgi.core_1.1.2
15 ACTIVE org.springframework.bundle.osgi.extender_1.1.2
16 ACTIVE org.ops4j.pax.web.service_0.5.1
17 ACTIVE org.apache.servicemix.specs.locator-1.1.1
18 ACTIVE org.apache.servicemix.bundles.jaxb-impl_2.1.6.1
19 ACTIVE org.apache.servicemix.bundles.wsdl4j_1.6.1.1
20 ACTIVE org.apache.servicemix.bundles.xmlsec_1.3.0.1
21 ACTIVE org.apache.servicemix.bundles.wss4j_1.5.4.1
22 ACTIVE org.apache.servicemix.bundles.xmlschema_1.4.2.1
23 ACTIVE org.apache.servicemix.bundles.asm_2.2.3.1
24 ACTIVE org.apache.servicemix.bundles.xmlresolver_1.2.0.1
25 ACTIVE org.apache.servicemix.bundles.neethi_2.0.4.1
26 ACTIVE org.apache.servicemix.bundles.woodstox_3.2.7.1
27 ACTIVE org.apache.cxf.cxf-bundle-minimal_2.2.0.SNAPSHOT
28 ACTIVE org.apache.servicemix.specs.saaj-api-1.3_1.1.1
29 ACTIVE org.apache.servicemix.specs.stax-api-1.0_1.1.1
30 ACTIVE org.apache.servicemix.specs.jaxb-api-2.1_1.1.1
31 ACTIVE org.apache.servicemix.specs.jaxws-api-2.1_1.1.1
32 ACTIVE cxf-dosgi-ri-discovery-local_1.0.0.SNAPSHOT
33 ACTIVE cxf-dosgi-ri-dsw-cxf_1.0.0.SNAPSHOT
34 ACTIVE org.ops4j.pax.web.extender.war_0.4.0
35 ACTIVE AuctionIntf_1.0.0
36 ACTIVE AuctionImpl_1.0.0
37 ACTIVE AuctionSite_1.0.0

Open the webapp at http://localhost:8080/auctionsite/AuctionSite.html. This kicks off the AJAX app in your browser. When you hit the reload button it makes a SOAP/HTTP invocation on the server to list all the items and get their details. The SOAP messages it sends map to the AuctionService.getItemIDs() and AuctionService.getItems() APIs. (At the bottom of the screen you see the SOAP message as it comes back form the server)

Let's list a new item...

After filling in the Javascript popup, hit OK.
It now sends the following SOAP message from the browser to the server:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
  <ns1:listNewItem xmlns:ns1="http://auction.coderthoughts.org/">
   <ns1:arg0>Joke</ns1:arg0>
   <ns1:arg1>1000</ns1:arg1>
   <ns1:arg2>2009-02-09T15:48:22-00:00</ns1:arg2>
  </ns1:listNewItem>
</soap:Body>
</soap:Envelope>

Once the remote invocation has completed, the list of items is updated again, which is the same as hitting the reload button.

I can still access the web service from my remote OSGi client that I developed in the previous post. The same remote OSGi service can obviously be shared among many consumers. When I run the AuctionClient bundle I'm also seeing my newly listed item.
Before I can do that I need to update the remote-services.xml file in the consumer bundle with the new location of the Web Service (when using Discovery this is not needed BTW):
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
<service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/auction
  </property>
</service-description>
</service-descriptions>

Now run the AuctionClient bundle:
Items available for auction:
1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
10:                 Joke $ 10.00 Mon Feb 09 15:48:12 GMT 2009
2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009
You can get all the code in this posting under the Apache license from the google code project. It's all in SVN at http://coderthoughts.googlecode.com/svn/trunk/gwt_auctionsite and http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite .
Since it's all Eclipse projects, the easiest way to get it by doing the SVN checkout of the projects directly in Eclipse. It will give you a workspace that looks like this:

Distributed OSGi – A Simple Example

2009-02-03T04:36:00.000-08:00

One of the exciting things to come out in 2009 is the new version of the OSGi Specification. It's called 4.2 but should really have been called 5.0 since there's a lot of new stuff in it. Distributed OSGi for example. (You can find a preview of the Distributed OSGi specification in this doc: http://www.osgi.org/download/osgi-4.2-early-draft2.pdf look for RFC 119 from page 196).
Distributed OSGi enables the distribution of OSGi Services across VMs and provides a standardized way of doing this. It doesn't specify what protocol or binding should be used on the wire for the inter-VM communication, that's up to the implementation. What it does specify is which standard properties you set on your OSGi Service to declare that it should be remoted.

The Service Implementation side
So here it is in its simplest form. Take an OSGi Auction Service that implements the following Java Interface:
public interface AuctionService {
  int listNewItem(String description, int startPriceInCents, Date expiryDate);
  void bid(int itemID, String user, int priceInCents)

  Collection<Integer> getItemIDs(String query);
  Collection<AuctionItem> getItems(int ... id);
}
With this you can list a new item, bid on an item, search for items and get item details given the item's IDs.
I've got a noddy little implementation of this service here: AuctionServiceImpl.java.
Note that I designed the interface to be suitable for distribution with the technology that I'll be using. This means that I don't rely on pass-by-reference for my method arguments and return types. In most cases distribution technologies will use pass-by-value so I made my design compatible with that.
I have put this interface in a separate bundle called AuctionIntf so that it can be used by both the service implementor and the consumer.

Now I want make this service available to my remote OSGi service consumers. The only thing I have to do for this is set a property on the OSGi service when I register it in the Activator with the local OSGi Service Registry: osgi.remote.interfaces=*. This means that I declare all the interfaces passed to bundleContext.registerService() are suitable for remoting. In my case I'm only passing one interface. (You could list individual interfaces instead of passing '*' if you only want to expose a subset).

public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    sr = context.registerService(AuctionService.class.getName(),
        new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}
I'm putting the service implementation and the Activator in a bundle called AuctionImpl which uses the AuctionService interface from the AuctionIntf bundle.

Any my Distributed Service is done!
Well... as long as I've got an implementation of Distributed OSGi running as well.

Lets do an experiment with the server side running on Equinox and the client side running on Felix. For Equinox you need version 3.5 built in December 2008 or later. For Felix you need version 1.4.1 or newer.
I'm using the Distributed OSGi DSW Reference Implementation which is based on Apache CXF and have the single-bundle version of that installed.

It's currently not yet released, but you can get a snapshot version via Maven from here (the bundle name is cxf-dosgi-ri-singlebundle-distribution).

Install this, plus its dependencies in Equinox and you'll get the following bundles:
osgi> ss

Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20081201-1815
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE javax.servlet_2.5.0.v200806031605
3 ACTIVE cxf-dosgi-ri-singlebundle-distribution
4 ACTIVE AuctionIntf_1.0.0
5 ACTIVE AuctionImpl_1.0.0
The distributed OSGi bundle depends on a few API's from the OSGi compendium spec, which are found in the org.eclipse.osgi.services bundle, which in turn depends on the javax.servlet bundle.

Hey, my service is now remoted using Web Services and available via SOAP over HTTP! CXF even automatically generates a WSDL: http://localhost:9000/org/coderthoughts/auction/AuctionService?wsdl should look like this:

This in itself is very useful. I can use this WSDL to invoke my OSGi service using any technology that can make Web Services invocations. So I could write a C++ or even a Visual Basic client for it! (In the next posting I'll use it from a remote AJAX Web client: calling a remote OSGi service straight from a browser!)

There is another way of finding out whether your service has been made available remotely, which works independently of the middleware used. Look in the OSGi Service Registry for services implementing the org.osgi.service.discovery.ServicePublication interface:

osgi> services (objectClass=*Publication)
{org.osgi.service.discovery.ServicePublication}={
  osgi.remote.endpoint.location=http://localhost:9000/org/coderthoughts/auction/AuctionService,
  service.properties={...},
  service.interface=[org.coderthoughts.auction.AuctionService],   service.id=34}
Registered by bundle: cxf-dosgi-ri-singlebundle-distribution [3]

When a Distributed OSGi implementation is remotely exposing a service, it will in turn register a service implementing the ServicePublication interface to advertise the remote service to any available Discovery mechanisms.

The Consumer side
My consumer code looks like a very ordinary OSGi Service Consumer: a little bit messy :) In real life I'd rather use something like the OSGi Blueprint (aka Spring-DM), iPOJO or Guice-OSGi, but in this posting I'd like to keep things a little focused so I'll go with writing plain OSGi service consumer code using a ServiceTracker.

public class Activator implements BundleActivator {
  private ServiceTracker st;

  public void start(final BundleContext bc) throws Exception {
    st = new ServiceTracker(bc, AuctionService.class.getName(), null) {
      @Override
      public Object addingService(ServiceReference reference) {
        Object svc = bc.getService(reference);
        if (svc instanceof AuctionService) {
          printServiceInfo((AuctionService) svc);
        }

        return super.addingService(reference);
      }
    };
    st.open();
  }

  protected void printServiceInfo(AuctionService auctionService) {
    System.out.println("Items available for auction:");
    Collection<Integer> ids = auctionService.getItemIDs(null);

    int[] idArray = toIntArray(ids);
    for (AuctionItem item : auctionService.getItems(idArray)) {
      System.out.printf("%3d: %25s $%,6.2f %tc\n",
        item.getID(),
        item.getDescription(),
        (item.getPriceInCents() / 100.0),
        item.getExpires());
    }
  }

  private int[] toIntArray(Collection<Integer> ids) {
    // details omitted
  }

  public void stop(BundleContext bc) throws Exception {
    st.close();
  }
}
As soon as the ServiceTracker gets a callback that the AuctionService is there, it prints out some details about the items available for auction.
Euh yeah, but where are the distribution bits in the consumer? Correct – there aren't any! A remote OSGi service is looked up exactly the same way as a local OSGi service. The simple fact that the client is expressing an interest in the service is enough for Distributed OSGi to go out and check for the existence of the service as a remote service. Behind the scenes it asynchronously checks any registered OSGi Remote Service Discovery services.
In this posting I'm not using a Discovery Service yet. I'm using a alternate, more static way, of specifying where my remote service is. Via a remote-services.xml file. This file sits by default in the OSGI-INF/remote-service directory of a bundle:
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
<service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/org/coderthoughts/auction/AuctionService
  </property>
</service-description>
</service-descriptions>
The format of this file is specified by Distributed OSGi. osgi.remote.interfaces is the same standard property that I set on the service. The osgi.remote.configuration.pojo.address is a CXF-specific one that simply denotes where my remote service is running.

So let's run the consumer in Felix.
-> ps
START LEVEL 1
ID State Level Name
[ 0] [Active ] [ 0] System Bundle (1.4.1)
[ 1] [Active ] [ 1] Apache Felix Shell Service (1.0.2)
[ 2] [Active ] [ 1] Apache Felix Shell TUI (1.0.2)
[ 3] [Active ] [ 1] Apache Felix Bundle Repository (1.2.1)
[ 4] [Active ] [ 1] OSGi R4 Compendium Bundle (4.1.0)
[ 5] [Active ] [ 1] Distributed OSGi Distribution Software Single-Bundle Distribution
[ 6] [Active ] [ 1] AuctionIntf (1.0.0)
[ 7] [Active ] [ 1] AuctionClient (1.0.0)
Besides the default Felix system, shell and bundle repository bundles, I've also installed the OSGi Compendium bundle which contains some OSGi standard interfaces that we need.

After the client bundle has started it will receive a callback that the (remote) AuctionService it wants to use is available and it will print out some information about the available items:
Items available for auction:
1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009

You can also see the remote service represented in the local service registry the same way a local service is. It is registered on demand by the DSW bundle:
-> services 5
...
objectClass = org.coderthoughts.auction.AuctionService
osgi.remote = true
osgi.remote.configuration.type = pojo
osgi.remote.configuration.pojo.address = http://localhost:9000/org/coderthoughts/auction/AuctionService
osgi.remote.interfaces = *
service.id = 30

There's our remote service! Just like an ordinary OSGi service. An extra property is set on client side proxies: osgi.remote. This can be used to find out whether a service that you are dealing with is remote or not. It can also be used to change the default lookup behaviour. By default remote services are always included in the service lookups, but this default can be tweaked by registering a FindHook/EventHook, which is a new feature of the Service Registry specified in RFC 126.

You can find the full implementation of the AuctionIntf, AuctionImpl and AuctionClient OSGi bundles as Eclipse projects in SVN: http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite

Next posting: an AJAX client UI written using the Google Web Toolkit (GWT).

Links
OSGi Specs: http://www.osgi.org/Specifications
Disributed OSGi Reference Implementation based on Apache CXF: http://cwiki.apache.org/confluence/display/CXF/Distributed+OSGi
Apache CXF Home page: http://cxf.apache.org

A Web Site Concentrator to ease AJAX development

2008-11-26T03:31:00.000-08:00

Every now and then I do a little AJAX development and there is one thing that I always find frustrating during the development of AJAX apps: The browser's single-origin policy.

Now I agree that for non-development cases this policy is absolutely justified, but during development it's a big PITA.
Let's say you are developing an AJAX app that calls into a back-end app over XML and you want to debug both. Like in my case I have a little GWT app that invokes on a Distributed OSGi Service running on the back end in CXF over port 9000. Ultimately you'd deploy it all over a single server, so there wouldn't really be a problem in production, but during development my GWT app is running on http://localhost:8888/org.coderthoughts.dinner.DinnerClient while my CXF back end runs on http://localhost:9000/org/apache/cxf/dosgi/samples/springdm/DinnerService

If I try to invoke my back-end app in this setup my AJAX app complains:

Since your front-end and back-end are probably written in different languages chances are that they have different development and debugging environments, which makes adhering to the single-origin hard to do. Besides, you don't want to go through the hassle of rebuilding your .WAR files (or whatever format you might use) for every single change. I know MSIE has some options to alleviate this, but they don't always work. Also using JSON as a data format gives you some options to bypass the problem, but that only works if your data format is JSON. In my case it's XML.

Enter ProxyServlet. There's a few of these around, but they're generally a bit raw. I particularly liked the one from Frank Spieleck because it has no dependencies, so is simple to use.
Typically these ProxyServlets forward requests from one context to another location, so to build the multi-site concentrator that you need to make the browser think both of them come from the same source you need at least two of them. In my case I need the following proxy settings:

org.coderthoughts.dinner.DinnerClient => http://localhost:8888/org.coderthoughts.dinner.DinnerClient

org => http://localhost:9000/org

To ease this task I wrote a little commandline wrapper around Frank's ProxyServlet that simply generates a .WAR file with the right name to set the root context, the correct bits in the web.xml etc... So I generate 2 .WAR files:

java -jar genwebproxy.jar \
  org.coderthoughts.dinner.DinnerClient \
  http://localhost:8888/org.coderthoughts.dinner.DinnerClient
  Created: org.coderthoughts.dinner.DinnerClient.war

java -jar genwebproxy.jar org http://localhost:9000/org
  Created: org.war
and simply deploy them into Tomcat by copying them into the webapps directory. These are standard .WAR files so you can deploy them into any Webapp container. I've also put the PAX-WEB headers in, in case you want to deploy them into that.

Et voilà. It works!

Note that I access my AJAX app using the Tomcat port of 8080 instead of the GWT debug port of 8888. I did have to change the port number at with my webapp makes an invocation from 9000 to 8080 as well, but that's easy.

With Frank's permission I put the ProxyServlet plus my wrapping code for the genwebproxy.jar file in SVN here: http://coderthoughts.googlecode.com/svn/trunk/GenWebProxy

You can download the binary from here, the command line is:
java -jar genwebproxy.jar [context_root] [real_location]

An OSGi Bundle dependency visualizer in GMF

2008-04-28T09:30:00.000-07:00

When working with larger OSGi-base systems you sometimes wonder: why are all these bundles there? What other bundles does my bundle depend on at runtime? Is there any duplication going on?
For a project that I was working on I started manually creating Bundle Dependency diagrams. This got me thinking: it should be really easy to create a tool that does this for me using EMF (Eclipse Modeling Framework) and GMF (the Graphical Modeling Framework from Eclipse).

As always in Model Driven Development, you start with a model. In my case a model of how OSGi Bundles relate to each other.
Having the model, you just fill it with data, then use some GMF goodness to display is graphically.

Sounds easy? It was easy! With well under 300 lines of manually written code (and a whole bunch of EMF/GMF generated stuff) the end result looks like this:

In this picture I ran the visualizer in the ServiceMix 4 Kernel to see what was in it.

Main Components

There are two components in this application:

A bundle that you drop into an OSGi container, when you start the bundle it will walk all the other bundles in the OSGi container and figure out what their dependencies are. The result is stored in an EMF model which is stored a file called save.bundles at the end of the process.
An Eclipse GMF application that can create a visual diagram from the save.bundles file and allows you to tweak the resulting picture any way you like.

How it's done
The model

1. Start with a simple model of a Framework (representing the OSGi Container) which contains bundles. Each bundle in turn could be importing stuff from other bundles:

Here are the model details:

After generating the genmodel file from my model file (I only modified the base package in the genmodel package properties) I generated the model code so I could start coding with it. (See here for an earlier post that outlines how to generate code from an EMF model).

The Introspector Bundle
To create a model instance with the bundle dependency data, I've written an self-contained OSGi bundle that I can drop on any OSGi runtime (well, at this point I only tried Equinox and Felix). When the bundle is started it walks all the active bundles in the container and figures out what other bundles each bundle depends on by using the bundle's classloader to resolve imported classes. (Since Package Imports are resolved dynamically this can only be done at runtime in an active OSGi container). The collected info is stored in an instance of my model. I stayed away from using any implementation-specific admin API's, instead I'm just using the bundle's classloader to resolve the packages listed in the Import-Package header to and figure out what bundle the OSGi runtime has linked them to.
Additionally, Require-Bundle dependencies are stored in the model too.

Finally, after all the bundles have been processed, I use the built-in EMF XMI serialization to save model instance to a file (currently always called save.bundles in your temp directory).

That's the hard bit done. You can see the code here in the BundleIntrospector and BundleIntrospectorTools classes which are invoked from my simple Activator.

The Visualizer
Believe it or not, the easy bit is in creating the graphical visualizer. GMF does all the hard work for me. I'm by no means a GMF connaisseur, so I used the GMF Cheat Sheet that comes with GMF in Eclipse. I ran all the wizards from the Cheat Sheet, and used all the defaults to create the .gmfgraph, .gmftool, .gmfmap and .gmfgen files. For some reason, there was one setting that I had to manually change: the Label Mapping, but the cheat sheet told me what to do.
Finally I hit 'generate diagram code' on the .gmfgen file and I'm done.

My Eclipse Workspace looks like this.

Running it

Running the introspector with Equinox
This is very easy from within Eclipse. To try this out, create a new OSGi Framework launch configuration with the introspector bundle and, to give it something to look at, I also include the org.mortbay.jetty bundle and it's dependencies:

Make sure that the introspector bundle is not automatically started. When I launch Equinox I have the following bundles loaded:


osgi> ss
Framework is launched.

id   State    Bundle
0    ACTIVE   org.eclipse.osgi_3.3.1.R33x_v20070828
1    ACTIVE   org.mortbay.jetty_5.1.11.v200706111724
2    ACTIVE   org.apache.commons.logging_1.0.4.v200706111724
3    ACTIVE   javax.servlet_2.4.0.v200706111738
4    RESOLVED osgi_bundles.introspector.bundle_0.0.3.qualifier

When I start the introspector bundle it starts analyzing the other bundles:


osgi> start 4
Initializing data from bundle: org.eclipse.osgi
Initializing data from bundle: org.mortbay.jetty
Initializing data from bundle: org.apache.commons.logging
...
Saving... Done dependency model saved to: c:\temp\save.bundles
Finished introspecting, stopping bundle.

Running the Visualizer
Now that I have a data file, I can run the visualizer. Just right-click on one of your projects in Eclipse and select Run As Eclipse Application.
In the Eclipse Application that gets started, create a new Project and copy the save.bundles file in there.

At this point the save.bundles file is purely an EMF serialization into XMI, but GMF can initialize a picture from it, right-click 'initialize bundles_diagram file', and you'll get one:

Tadaa, a nice visualization of the bundles and their dependencies. The Jetty bundle uses the commons logging and servlet bundles (the system bundle is ommitted).
Now you can use the GMF features to rearrange your diagram, choose colours, line smoothness etc to make your diagram the way you want it (which is quite useful once you have a lot of bundles). You can even export the picture as a PNG, JPG or SVG file.

Running the introspector in ServiceMix 4 Kernel
The ServiceMix 4 Kernel is an OSGi based system built using Apache Felix. If you want to run the introspector in the ServiceMix 4 Kernel you can follow these steps:
1. Get hold of a SMX 4 Kernel distribution. I created mine by checking out the code from http://svn.apache.org/repos/asf/servicemix/smx4/kernel/trunk and running mvn install to build it. You will find the distribution in assembly/target subdirectory.
2. Run bin/servicemix


 ____                  _          __  __ _
/ ___|  ___ _ ____   _(_) ___ ___|  \/  (_)_  __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
 ___) |  __/ |   \ V /| | (_|  __/ |  | | |>  <
|____/ \___|_|    \_/ |_|\___\___|_|  |_|_/_/\_\

 ServiceMix (1.0-m3-SNAPSHOT)

Type 'help' for more information.
------------------------------------------------
servicemix>

Move to the osgi shell and install the introspector, I'm doing this straight from the Google Code project downloads section:


servicemix> osgi
servicemix osgi> install http://coderthoughts.googlecode.com/files/osgi_bundles.
introspector.bundle_0.0.3.200804202025.jar
Bundle ID: 28
servicemix osgi>

When I start bundle 28 it will do the introspection and save this to the save.bundles file. When loaded in the visualizer, you'll get a picture like the first one in this posting.

Getting your environment set up
I used Eclipse 3.3.1.1 with the EMF/GMF plugins loaded. See this posting for details on setting it up.

Getting the code
All the code used here is available under the Apache License from the Google Code project via SVN. The Eclipse Projects are accessible from here: http://coderthoughts.googlecode.com/svn/trunk/osgi_visualizer/

Getting the distribution
If you just want to use or play with the visualizer, you can download the introspector bundle and visualizer plugins from the the download area: http://code.google.com/p/coderthoughts/downloads/list

Improvements are welcomed!
The project is by no means finished. I'm still in the process of learning more about GMF and any suggestions or improvements are appreciated.

Using Jersey (JSR311) inside OSGi with Spring-DM

2008-03-31T11:54:00.000-07:00

Or: using REST & OSGi to create a number guessing game

Jersey is an implementation of JSR 311 also know as JAX-RS. It makes writing a REST service quite simple with Java Annotations. So I wanted to use Jersey inside my OSGi bundle with the OSGi HTTP Service. However, as I found out, Jersey is not written to run in an OSGi container...

Fortunately, the problematic code in Jersey is pluggable. In this posting I'm writing a little Jersey REST Application that runs in my OSGi container. I'm using Spring-DM (aka Spring-OSGi) to do the OSGi Service dependency injection which keeps my code really lean & clean (Spring Clean ;) .

To illustrate I wrote a little number guessing game. The very first program I ever wrote was a game like this. This was on the Commodore Pet computer and I was about 10 years old. So beware...

Development Environment Setup
The Jersey version I got is 0.7-ea, which I downloaded from the Jersey download site.

I'm using the Equinox OSGi runtime in Eclipse 3.3.2 (get it here) with Spring-DM added in.

I downloaded Spring-DM 1.0.2 from here and copied all the jars from the dist and the lib directory of the download in my eclipse/plugins directory. The jars in the Spring-DM lib directory have been OSGi-ified by the Spring guys so we can nicely drop them in as OSGi bundles.

Bundle Dependencies
The general dependency picture is like the image below. I'm writing a Jersey app in a bundle that depends on a library bundle which contains the Jersey jars. This way multiple bundles can use the Jersey functionality to implement REST services while sharing the same Jersey Library bundle.

The Jersey Library Bundle
This is a simple wrapper around the Jersey libraries. At the moment I'm only depending on jersey.jar, jsr311-api.jar and asm-3.1.jar. So I'm simply creating an OSGi Bundle Project in Eclipse (Plug-in Project - see this posting for some details on how to do this), add these three jars from the Jersey install to the root of the bundle and then put them on the Bundle-Classpath. Finally simply export all the classes defined by these bundles. I guess a subset of the classes would do too, anyone fancy figuring out what the minimum Export-Package would be?

My MANIFEST.MF looks like this:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: JerseyLibBundle
Bundle-SymbolicName: JerseyLibBundle
Bundle-Version: 1.0.0
Bundle-ClassPath: asm-3.1.jar,
jersey.jar,
jsr311-api.jar,
.
Import-Package: javax.servlet,
javax.servlet.http
Export-Package: ...,
... export all the packages in these jars ...

That's the Jersey Library Bundle done. My Eclipse screen looks like this (click on it to see all the details):

The REST Bundle
This is the interesting bit. Here's where we'll have our REST Guessing Game implemented.

The REST application will be available on:

  http://localhost/fun/numbers
Lists all the numbers available in XML format.

  http://localhost/fun/numbers/{number}
Returns information about the requested number in text format. The information says what number it is (the same one you asked for) and whether or not you guessed the right number.

I have a jersey_osgi.rest.Number class:
package jersey_osgi.rest;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class Number {
  enum Location {LESS, FOUND, MORE};
  private static final int MAX_NUM = 10;

  static List<Number> NUMBERS;
  static {
    // Initialize with numbers 1 to 10,
    // and pick a random on to guess.
    NUMBERS = new ArrayList<Number>();

    Random random =
      new Random(System.currentTimeMillis());
    int theOne = random.nextInt(MAX_NUM) + 1;

    for (int i=1; i <= MAX_NUM; i++) {
      Location location;
      if (i == theOne) {
        location = Location.FOUND;
      } else if (i < theOne) {
        location = Location.LESS;
      } else {
        location = Location.MORE;
      }
      NUMBERS.add(new Number(i, location));
    }
  };

  private final int number;
  private final Location location;
  public Number(int num, Location l) {
    number = num;
    location = l;
  }

  public int getNumber() {
    return number;
  }

  @Override
  public String toString() {
    StringBuilder sb = new StringBuilder();
    sb.append("I'm number ");
    sb.append(number);
    sb.append(". ");
    switch (location) {
    case FOUND:
      sb.append("You guessed right!");
      break;
    case LESS:
      sb.append("You need to guess higher.");
      break;
    case MORE:
      sb.append("You need to guess lower.");
      break;
    }
    return sb.toString();
  }
}

Next thing we need is a JSR311 compliant class that binds our Number class to a REST API:
package jersey_osgi.rest;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.ProduceMime;

@Path("/numbers")
public class NumberResource {
  @GET
  @ProduceMime("text/xml")
  public String listNumbers() {
    StringBuilder sb = new StringBuilder();

    sb.append("");
    sb.append("\n");
    sb.append("<numbers>");
    sb.append("\n");
    for (Number number : Number.NUMBERS) {
      sb.append("<number>");
      sb.append(number.getNumber());
      sb.append("</number>");
      sb.append("\n");
    }
    sb.append("</numbers>");
    return sb.toString();
  }

  @GET
  @Path("/{id}")
  @ProduceMime("text/plain")
  public String getNumber(@PathParam("id") String id) {
    try {
      int i = Integer.parseInt(id);
      Number n = Number.NUMBERS.get(i-1);
      return n.toString();
    } catch (Throwable th) {
      return "Sorry that number is out of range";
    }
  }
}
That's our REST implementation done. Only thing left to do is use the OSGi HTTPService to launch the Jersey Servlet and tell it what our REST Resource are, right? You wish! You would think that the following piece does the trick:

HttpService httpService = ...; // The OSGi HTTP Service
Hashtable initParams = new Hashtable();
initParams.put(
  // The following are standard Jersey properties
  "com.sun.ws.rest.config.property.resourceConfigClass",
  "com.sun.ws.rest.api.core.PackagesResourceConfig");
initParams.put(
  "com.sun.ws.rest.config.property.packages",
  "jersey_osgi.rest");

Servlet jerseyServlet =
  new com.sun.ws.rest.spi.container.servlet.ServletContainer();
httpService.registerServlet("/fun", jerseyServlet, initParams, null);

But hey! I'm getting an ugly exception:
com.sun.ws.rest.api.container.ContainerException: The ResourceConfig instance does not contain any root resource classes.

This is because the com.sun.ws.rest.api.core.PackagesResourceConfig class contains OSGi unfriendly classloader code (it tries to treat classes as resources, which at least does'nt work on Equinox). Luckily this class is plugged into Jersey, so I can write an OSGi-friendly version. PackagesResourceConfig searches the specified packages for any classes with the JSR311 annotations.
My alternative implementation is called jersey_osgi.common.OSGiResourceConfig. The source is slightly too big for the posting, so just click on the link if you want to see it.

For my OSGi version I have to actually specify the classes that are annotated in a property called jersey_osgi.classnames. So I change to properties like this:
HttpService httpService = ...; // The OSGi HTTP Service
Hashtable initParams = new Hashtable();
initParams.put(
  "com.sun.ws.rest.config.property.resourceConfigClass",
  "jersey_osgi.common.OSGiResourceConfig");
initParams.put(
  "jersey_osgi.classnames",
  "jersey_osgi.rest.NumberResource");

Servlet jerseyServlet =
  new com.sun.ws.rest.spi.container.servlet.ServletContainer();
httpService.registerServlet("/fun", jerseyServlet, initParams, null);
And run it. Now I'm getting:
com.sun.ws.rest.api.container.ContainerException: No WebApplication provider is present aaaargh! After some debugging I found out that Jersey heavily depends on the META-INF/services SPI architecture which is quite non-OSGi friendly. There are only two possible solutions to this that I can think of:

Export-Package META-INF.services in the Jersey Library Bundle. This works but is kinda nasty because you can only do it once in a single OSGi runtime. If you have multiple bundles exporting the same package, the OSGi runtime will only link you to only one of them for this package. If multiple bundles export META-INF.services you will get unexpected behaviour.

Copy the META-INF/services directory from the jersey.jar into our own bundle. Other projects with OSGi bundles seem to be taking this approach too.

So I'm taking approach 2 as well and copy all the META-INF/services files of jersey.jar into my bundle. And now it works! I can nicely build my Jersey apps as OSGi bundles that all depend on shared Jersey code.

Where's the Spring stuff?
Spring-DM makes it really easy for me to interact with the OSGi HTTP Service as it simply injects it in my JerseyServletFactory class which then registers the Jersey Servlet. To trigger this, I put a jersey_osgi_spring.xml file in the META-INF/spring directory of my bundle:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns:osgi="http://www.springframework.org/schema/osgi"
  xsi:schemaLocation="...">

  <osgi:reference id="httpServiceRef"
    interface="org.osgi.service.http.HttpService"/>

  <bean name="jerseyServletFactory"
    class="jersey_osgi.common.JerseyServletFactory" >
    <property name="jerseyClassNames"
      value="jersey_osgi.rest.NumberResource"/>
    <property name="rootContext" value="/fun"/>
    <property name="httpService" ref="httpServiceRef"/>
  </bean>
</beans
The JerseyServletFactory registers the Jersey Servlet with the OSGi HTTP Service. For this it needs that HTTP Service in the first place. Those who had the pleasure of using the pure OSGi API's to get hold of an OSGi Service know that this is actually slightly tricky. So OSGi Declarative Services come to the rescue (which I used in the earlier article about writing a Card Game with OSGi Services). Spring-DM provides an alternative to OSGi-DS and brings with it all of the goodies that Spring has to offer. With the above Spring file, my JerseyServletFactory is nicely reusable (the Root Context and Jersey resource classes are also externalized into the Spring Configuration file) and no bigger than this:
package jersey_osgi.common;

import java.util.Hashtable;
import javax.servlet.Servlet;
import org.osgi.service.http.HttpService;
import com.sun.ws.rest.spi.container.servlet.ServletContainer;

public class JerseyServletFactory {
  private String classNames;
  private String rootContext;

  // Spring uses the setters to inject our stuff
  public void setJerseyClassNames(String names) {
    classNames = names;
  }

  public void setRootContext(String ctx) {
    rootContext = ctx;
  }

  public void setHttpService(HttpService httpService) throws Exception {
  Hashtable<String, String> initParams =
    new Hashtable<String, String>();
    initParams.put(
      "com.sun.ws.rest.config.property.resourceConfigClass",
      "jersey_osgi.common.OSGiResourceConfig");
    initParams.put(
      "jersey_osgi.classnames",
      classNames);

    Servlet jerseyServlet =
      new ServletContainer();
    httpService.registerServlet(rootContext,
      jerseyServlet, initParams, null);
  }
}

Running it all inside Eclipse
Running Spring-DM inside Equinox as shipped with Eclipse requires careful definition of the runtime environment. You set this up by going to Run | Open Run Dialog ... in Eclipse where you create a new OSGi Framework run configuration. Deselect the Target Platform tickbox and only select the following bundles:
JerseyLibBundle_1.0.0
JerseyOsgiSpringDM_1.0.0
javax.servlet_2.4.0.v200706111738
org.apache.commons.logging_1.0.4.v200706111724
org.eclipse.core.jobs_3.3.1.R33x_v20070709
org.eclipse.core.runtime.compatibility.registry_3.2.100.v20070316
org.eclipse.equinox.common_3.3.0.v20070426
org.eclipse.equinox.http.jetty_1.0.1.R33x_v20070816
org.eclipse.equinox.http.registry_1.0.1.R33x_v20071231
org.eclipse.equinox.http.servlet_1.0.1.R33x_v20070816
org.eclipse.equinox.registry_3.3.1.R33x_v20070802
org.eclipse.osgi.services_3.1.200.v20070605
org.eclipse.osgi_3.3.2.R33x_v20080105
org.mortbay.jetty_5.1.11.v200706111724
org.springframework.bundle.osgi.core_1.0.2
org.springframework.bundle.osgi.extender_1.0.2
org.springframework.bundle.osgi.io_1.0.2
org.springframework.bundle.spring.aop_2.5.1
org.springframework.bundle.spring.beans_2.5.1
org.springframework.bundle.spring.context_2.5.1
org.springframework.bundle.spring.core_2.5.1
org.springframework.osgi.aopalliance.osgi_1.0.0.SNAPSHOT
Like this (I edited the unselected bundles out of the screenshot):

I found that if I don't do it exactly like this that I'm getting problems with the correct logger being instantiated, but this configuration works. Just hit 'Run' and you're off. You can now access the REST application from http://localhost/fun/numbers

I can guess a number: http://localhost/fun/numbers/5

... and try again: http://localhost/fun/numbers/3

Yay!

Get the code
You can get all the code (Apache License) as usual from the google code site. It's at available as 2 Eclipse projects at http://coderthoughts.googlecode.com/svn/trunk/jersey_osgi/. The easiest way to get them into your Eclipse installation is by using an SVN client inside Eclipse and check them out as projects. This is described in more detail in this posting (just search for the 'subclipse' in there).

OSGi Services for Dynamic Applications (IV)

2007-12-03T11:00:00.000-08:00

Or: use OSGi to write a dueling card game - part 4
In the previous postings I created the basics of a dueling card game using OSGi services. In this last posting of this series I'm going to add some special spell cards. Finally I'll use the capabilities of OSGi to dynamically add a bundle with new cards to the ongoing games without having to interrupt them.

Adding Spell Cards
Spells make the game really interesting as they can influence the game beyond the standard rules. They can do things like change the power of a monster, inflict damage on a player or another card or generally change the game state. Sometimes spells can behave like monsters or monsters can have additional spell-like behaviour too. My SpellDeck bundle adds a number of them, each of which has special abilities and a special implementation.

In my Activator, I'm registering my spell cards (5 of each):
public class Activator implements BundleActivator {
  public void start(BundleContext ctx) throws Exception {
    for (int i = 0; i < 5; i++) {
      registerCard(ctx, new ThunderBoltCard());
      registerCard(ctx, new ReflectCard());
      registerCard(ctx, new SkipBattlePhaseCard());
      registerCard(ctx, new DoubleAttackCard());
      registerCard(ctx, new MedicinCard());
      registerCard(ctx, new ExplosionCard());
      registerCard(ctx, new LightningBoltCard());
      registerCard(ctx, new DoubleStrengthCard());
      registerCard(ctx, new HalfStrengthCard());
    }
  }

  private void registerCard(BundleContext ctx, Card c) {
    ctx.registerService(Card.class.getName(), c,
        new Hashtable());
  }

Let's zoom in on a few of them: ThunderBoltCard, SkipBattlePhaseCard, ReflectCard and DoubleStrengthCard.

ThunderBoltCard
This spell card can be played during a player's Standby or Recover phase and deals 20 damage directly to the opponent. This functionality is implemented in the execute() of this Command Card.
public class ThunderBoltCard extends AbstractCommandCard implements Card, Command {
  public ThunderBoltCard() {
    super(CardType.SPELL, "Thunderbolt",
        "Thunderbolt deals 20 damage to your opponent.");
  }

  public boolean execute(Game game) {
    Player p = game.toGrave(this); // discard this card
    Player q = game.getOtherPlayer();
    q.setScore(q.getScore() - 20);
    printBoth(game, p + " plays Thunderbolt, " +
        q + " loses 20 life");

    return true;
  }
}

SkipBattlePhaseCard
This card can be played by a player who is being attacked during battle. It will end the current Battle phase aborting any pending attacks and move to the Recover phase. Here's the code:
public class SkipBattlePhaseCard extends AbstractCommandCard implements Card, Command {
  protected SkipBattlePhaseCard() {
    super(CardType.SPELL, "Battle Skipper",
      "With battle skipper, your opponent must drop " +
      "the current attack and skip the battle phase.");
  }

  public boolean execute(Game game) {
    Player p = game.getOwner(this);
    if (game.getPlayer() == p) {
      p.getUI().print("Can only play this card during " +
        "your opponent's battle phase when attacked.");
      return false;
    } else {
      game.setAttacking(new ArrayList<Fighter>());
      game.setPhase(Phase.RECOVER);
      game.toGrave(this);
      printBoth(game, p + " plays Battle Skipper, " +
        which causes the current battle phase to end.");
      return true;
    }
  }
}

ReflectCard
This spell can also be played by a player who is being attacked during battle. It will reflect the same amount of defense back to the attacker as was the original attack. Typically this will destroy the attacking monster.
public class ReflectCard extends AbstractCard implements Card, Fighter {
  private int defense;

  public ReflectCard() {
    super(CardType.SPELL, "Reflect",
      "Reflects an attack back to the attacker as " +
      "defense, will normally destroy the attacker.");
  }

  public void damage(Game game, int amount) {}

  public int getAttack() {
    return 0;
  }

  public int getDefense() {
    return defense;
  }

  public boolean responding(Game game) {
    defense = 0;
    // calculate the defense
    for (Fighter f : game.getAttacking()) {
      defense += game.getAttack(f);
    }
    return true;
  }
}

DoubleStrengthCard
This is an attached card, which means that it is attached to another Fighter card on the field. It doubles the attack of the fighter card. The attachment behaviour of this card is implemented in the AbstractAttachedCard base class. This card implements the Modifier interface, which is taken into account when a monster's strength is calculated.
public class DoubleStrengthCard extends AbstractAttachedCard implements Card, Modifier {
  protected DoubleStrengthCard() {
    // This card can be attached to a Monster card on
    // the table of type Fighter
    super("Double Strength",
      Area.TABLE, Fighter.class, CardType.MONSTER,
      "Target creature gets double the attack.");
  }

  public int addToAttack(Game game, Fighter fighter) {
    if (fighter instanceof Card) {
      Player owner = game.getOwner((Card) fighter);
      if (owner != null &&
          owner.getTable().contains(fighter)) {
        return fighter.getAttack();
      }
    }
    return 0;
  }

  public int addToDefense(Game game, Fighter fighter) {
    return 0;
  }
}

Get the code of the other spells by checking out the SpellDeck Bundle from the usual location: http://coderthoughts.googlecode.com/svn/trunk/duel/.

Adding More Monsters while playing
Last but not least, I'm going to add a whole bunch of extra monsters to my game, and I'm going to do this while a game is being played. In a development scenario this is not really normal, but it will be needed once our application is successful and in production. Let's say our application is used by many users and we don't want to disturb our users by the mere action of adding a few new cards to the game. We just want those cards to appear in any ongoing games and in any new games from now on. This is where OSGi will really help us. It allows us to add and remove bundles without the need to restart the container. If those new bundles contain new Card service my game controller will be notified of them straight away. Similarly if a bundle that contains cards is being removed, we will be told about it too. If our application (bundles) interact with other bundles via OSGi Services, we will be able to patch our entire application without taking it down. To show this functionality in action I'm going to add the MoreMonsters bundle to the OSGi runtime after the game has started, but first of all I need to slightly modify the addCard() method in my Controller class to shuffle newly arrived cards into the decks of any ongoing games.

public void addCard(Card card) {
  synchronized (cards) {
    cards.add(card);

    // add the new card to any game deck in
    // progress in a random place
    for (Game g : games) {
      System.out.println(
        "Shuffling new card into existing game " + card);
      List<card> deck = g.getDeck();
      deck.add(rand.nextInt(deck.size()), card);
    }
  }
}

To get started, first check out the MoreMonsters bundle from the usual SVN location: http://coderthoughts.googlecode.com/svn/trunk/duel. This bundle only contains an Activator in which additional monsters (instances of the MonsterCard) are created and registered as Card services (courtesy to my 9 and 11 year old kids, who created these monsters).

I want the MoreMonsters bundle to be added while the game is in progress, so I need to make sure that it's not automatically started with the other bundles. For this, the launch confiration that I'm using in Eclipse needs to be tweaked, I'm setting the 'Auto-Start' of MoreMonsters to false:

You may also need to select the 'Clear the configuration area before launching' option in the 'Settings' tab of the launch configuration, because Equinox remembers your first settings and sometimes doesn't pick up changes like this without clearning the configuration area.

Hit 'Run' and the game will start as usual, with the original three monsters and our spell cards.

If you go into the Equinox console and type 'ss', you'll see that the MoreMonsters bundle is there, but it's just not started:

osgi> ss

Framework is launched.

id State Bundle
0 ACTIVE org.eclipse.osgi_3.3.1.R33x_v20070828
1 ACTIVE org.eclipse.osgi.services_3.1.200.v20070605
3 RESOLVED MoreMonsters_1.0.0
4 ACTIVE DuelInterfaces_1.0.0
7 ACTIVE javax.servlet_2.4.0.v200706111738
8 ACTIVE org.eclipse.equinox.ds_1.0.0.v20070226
9 ACTIVE SpellDeck_1.0.0
10 ACTIVE Duel_1.0.0
11 ACTIVE SimpleUI_1.0.0
12 ACTIVE MonsterDeck_1.0.0

So let's start the bundle!

osgi> start 3

And we'll see our new cards arriving:

  Shuffling new card into existing game Thundergirl
  Shuffling new card into existing game DB Machine
  Shuffling new card into existing game Devil Spider
  <...and so on...>

Soon you'll see your new cards arriving in your current game:

In this case the MoreMonsters Bundles was automatically installed by Eclipse for me, it was just not started. A more realistic real-world scenario would be where the OSGi container is running outside of Eclipse. In that case I would first have to install the bundle (Equinox has the install command for that) before I can actually start it.

Dynamically removing services
The same mechanism can be used to dynamically remove services from the system. I'll leave the implementation of that to the reader, but I hope you can see how powerful this mechanism can be. If you need to patch the an OSGi Service-based system, just install the patch bundle first, which will registered the patched services, then remove the old bundle, the old services will be unregistered which can make clients to switch to the patched services. Together with OSGi's strong versioning support, this is a great feature for mission critical systems that need to be up and running 24/7 (like game servers ;).

Equinox Specifics
In this posting I did use some Equinox-specific commands to control the OSGi container, other OSGi containers have other commands for achieving the same. All of the code in these postings should be 100% portable across the various OSGi containers.

Download the code
All the code for all these postings is available under the Apache License in the google code Subversion Repository.

OSGi Services for Dynamic Applications (III)

2007-12-02T11:00:00.000-08:00

Or: use OSGi to write a dueling card game, part 3

Adding Monster cards to the game and a simple UI

In the previous posting I've created the hardest bit of the game. The actual game controller. In this posting I'm going to create a small bundle that contains a few Monster cards and a simple UI bundle so that we can actually start playing.

Creating the Monster Deck bundle
Every card in my Dueling game is an OSGi Service.
In the Duel Bundle I used OSGi Declarative Services to activate the bundle and consume the services, and you can use DS also to register services, but in this case I'm going to use the plain OSGi API's for this. Registering a service is so simple in OSGi that in this case I'm just going to go API.

My monster cards instances are realized by a class called MonsterCard, which implements the Card, Fighter and Command interfaces. It's nice and simple, also because it extends AbstractCommandCard, which means that Command.execute() for the Standby and Recover phase are implemented as placing the card on the table. Other basic Card requirements such as accessors to the name, ID and description and proper implementations of equals(), hashcode() and toString() are handled by AbstractCard, the superclass of AbstractCommandCard. My MonsterCard class basically adds the implementation of the Fighter interface into the mix. It also specifies when the card is enabled in conditions() - in the Standby phase and the Recover phase, and also in the Battle phase, but only when it's on the table.
public class MonsterCard extends AbstractCommandCard implements Fighter, Card, Command {
  private int attack;
  private int defense;

  public MonsterCard(String name, String desc,
      int att, int def) {
    super(CardType.MONSTER, name, desc);

    attack = att;
    defense = def;
  }

  public void damage(Game game, int amount) {
    // nothing to do on damage
    // the battle code will discard when hit
  }

  public int getAttack() {
    return attack;
  }

  public int getDefense() {
    return defense;
  }

  public Condition [] conditions() {
    return new Condition [] {
      new Condition(Phase.STANDBY),
      new Condition(Area.TABLE, Phase.BATTLE),
      new Condition(Phase.RECOVER)
    };
  }

  public boolean responding(Game game) {
    return true; // can respond to any attacker
  }
}

To create any straight monster without any special abilities, I can simply use this class. I'm going to create 3 different monster cards, Super Hero, Floppie and Diamond Buster:
  new MonsterCard("Super Hero",
    "This is the invincible super hero.", 60, 35));
  new MonsterCard("Floppie",
    "Floppie joppie.", 15, 25));
  new MonsterCard("Diamond Buster",
    "Diamond buster is as strong as a rock.", 40, 55));
To turn these POJO's into OSGi services, I need to register them as services with the BundleContext.registerService(), defined as:
  registerService(java.lang.String clazz,
    java.lang.Object service,
    java.util.Dictionary properties)
Simply pass in the name of the class as the first argument, the actual object and optionally a map of properties. The properties can be used when selecting services as a service consumer, but I'm not using them here.

So to get 5 of each card registered when the MonsterDeck bundle is started, a small OSGi Activator class does it all:
package monsterdeck;

import game.model.Card;
import java.util.Hashtable;
import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {
  public void start(BundleContext ctx) throws Exception {
    for (int i = 0; i < 5; i++) {
      registerCard(ctx, new MonsterCard("Super Hero",
          "This is the invincible super hero.", 60, 35));
      registerCard(ctx, new MonsterCard("Floppie",
          "Floppie joppie.", 15, 25));
      registerCard(ctx, new MonsterCard("Diamond Buster",
          "As strong as a rock.", 40, 55));
    }
  }

  private void registerCard(BundleContext ctx, Card c) {
    ctx.registerService(Card.class.getName(), c,
        new Hashtable());
  }

  public void stop(BundleContext context)
      throws Exception { }
}
The only thing left to do it create the bundle-manifest in the META-INF/MANIFEST.MF file:
  Bundle-Name: MonsterDeck Bundle
  Bundle-SymbolicName: MonsterDeck
  Bundle-Version: 1.0.0
  Bundle-Activator: monsterdeck.Activator
  Import-Package: game.model,
   org.osgi.framework
  Export-Package: monsterdeck
I have to import the org.osgi.framework package since in my Activator I'm using the BundleActivator and BundleContext classes from this package. I'm exporting the monsterdeck package from this bundle so that I can use the MonsterCard class later in another bundle.

Adding a simple UI
I've tried to create the simplest UI possible. Its written using SWT and simply implements UserInterface. I just wanted to keep this part as simple as possible, but (as always) GUI classes tend to become relatively big. My GUI looks like this:

At the top you see the cards your opponent has on the table.
Below that is a status area where messages are displayed.
Then the cards you have on the table are shown and at the bottom are the cards that you have in your hand.
On the right hand side there is a status panel where each player's life points are shown, the current game phase is displayed and buttons are present to draw a card, end the current phase and take the damage of a fight.
Monster cards are shown red and spell cards are green. To select a card, you simply click on it. Generally only cards and actions that make sense in the current context are enabled. Attacking monsters have their names shown in italics and card descriptions can be read by hovering with the mouse over a card.

The UI bundles comes with a little factory that is launched as soon as the bundles is started. Type your name in the UI Factory and it will create a User Interface for you which is then registered as a UserInterface Service with OSGi.

You can get this UI implementation by checking it out the SimpleUI bundle from the usual SVN location: http://coderthoughts.googlecode.com/svn/trunk/duel/.

Running the game
This is really easy if you're using Eclipse. What you should have is 4 OSGi Bundle Projects:

Select the Run -> Open Run Dialog... menu and double-click on the OSGi Framework item in the tree. This will create a new OSGi launch configuration for you. By default it will have all the OSGi bundles in the system selected, which is waaaay too much, so I normally deselect the root 'Target Platform' node and then click 'Add Required Bundles':

Hit 'Run' and you're off. The UI Controller will show up and after creating two UI's the game will start.
The game controller prints the names of the users as soon as they register their UI services and when two users are in, it will shuffle the cards and start the game. My console shows:
  Adding user interface for: Pinky
  Adding user interface for: Brain
  Shuffling deck...

I can Draw a card:

Play a monster (e.g. Super Hero) on the table:

and engage in battle: my opponent attacks me with Diamond Buster, I can choose to defend with Super Hero, but he will die, alternatively I can take the damage myself and lose life points.

Before I forget: closing all the windows will not actually exit the system. The OSGi container will keep running. The command to stop the OSGi container is implementation-specific, if you're using Equinox just type in exit in the console.

With only 3 different cards in the game it is a little bit boring to say the least. In the next posting I'll add some cool spell cards and a bunch more monsters too!

OSGi Services for Dynamic Applications (II)

2007-11-30T11:50:00.000-08:00

Or: use OSGi to write a dueling card game, part 2: the Game Controller OSGi Services client
In this posting I'm going to write the Game Controller which is a client to the User Interface OSGi Services and also a client to the Card OSGi services.

The following diagram from the first posting outlines the game architecture:

The yellow triangles represent the OSGi services, the red rectangles are bundles that provide service implementations. The Game Controller is a client to the services and lives in an OSGi bundle too.

I am using OSGi Declarative Services to handle the client side of the OSGi services. I'm doing this because DS is easier than using the OSGi ServiceTracker API, and it's part of the OSGi spec.

I always find the easiest way to develop an OSGi bundle by using the Eclipse tooling that you get with the Eclipse Plugin Development Environment (download from here). With that you simply go: File -> New Project... -> Plug-in project. In the following window you simply select 'OSGi Platform' as the target platform. If you specify 'standard' it means that you won't be using any Equinox specific bits.

On the next page you can specify some details around the plugin ID, name and version. The defaults are generally fine but for this posting you won't need an activator as Declarative Services will take care of the stuff that is typically done in an activator for us, so you can deselect that option. Finally, you don't need to use any of the templates that come with Eclipse for this posting.
Hitting 'Finish' will get you an Eclipse project that is marked as a PDE project (which is an OSGi project) with a META-INF/MANIFEST.MF file which defines the attributes of the OSGi bundle.

My game bundle has two dependencies:

It depends on the DuelInterfaces bundle from the previous posting. All of its classes are in the game.model package, so I will be importing that package, which gives me the freedom to rename or reversion the DuelInterfaces bundle if needed.
Since I'm using Declarative Services it depends on a DS implementation. I'm using the one from Equinox / Eclipse, which doesn't come with Eclipse by default, you have to manually download and install it. To get it, go to the Equinox download page, select your Eclipse version (I took 3.3.1.1) and then look for the org.eclipse.equinox.ds_xxx.jar download. I downloaded this one. Then simply drop the downloaded jar file in the plugins directory of your Eclipse installation.
Because I'm not using any specific classes from the DS implementation, I cannot use the 'Import Package' mechanism, so I've specified this bundle as a direct dependency in 'Required Plugins'.

I've entered the dependencies in my manifest, using the Eclipse manifest editor, which now looks like this.

What I want DS to do for me is tell the Game Controller when someone has registered any new Card Services in the game or when a new User Interface is registered. When a UI is added, the controller will look for another UI that is not playing a game, if there is one it will hook them up and start the game for both players. If there is no free other UI, the newly added UI will be held until another UI presents itself. This is implemented in the Controller class which contains an addUI() method like this:

public synchronized void addUI(UserInterface ui) {
  System.out.println("Adding UI for: " +
      ui.getPlayerName());

  if (freeUIs.size() > 0) {
    UserInterface otherUI = freeUIs.remove(0);

    Player p1 = new Player(otherUI, 250);
    Player p2 = new Player(ui, 250);

    // Init both UI's with players and static commands
    ui.initializeGame(p2, p1, new DrawCommand(),
        new NextPhaseCommand(), new SelfFighter(p2));
    otherUI.initializeGame(p1, p2, new DrawCommand(),
        new NextPhaseCommand(), new SelfFighter(p1));

    // Create a new game with all the cards available
    final Game g =
        new GameImpl(new ArrayList(cards), p1, p2);

    new Thread(new Runnable() {
      public void run() {
        g.play();
      }
    }).start(); // play, don't block the OSGi thread
    games.add(g);
  } else {
    freeUIs.add(ui);
  }
}

A similar, but even simpler method is called when new cards are registered, my addCard() looks like this:
public void addCard(Card card) {
  synchronized (cards) {
    cards.add(card);
  }
}

But what do I do to get my methods called when these services get registered. This is where DS comes in. I have added a DS configuration file that instantiates my Controller component and gets DS to call me back when this happens. This pattern is called Inversion of Control (IOC). I have added a DS configuration file in my Duel bundle project in OSGI-INF/controlcomponent.xml. I could really place this file anywhere in the bundle, but putting it in OSGI-INF seems to be the convention. It contains the following XML:
<component name="controller">
  <implementation class="game.Controller"/>

  <reference name="UI"
    interface="game.model.UserInterface"
    bind="addUI"
    unbind="removeUI"
    cardinality="0..n"
    policy="dynamic"/>
  <reference name="CARD"
    interface="game.model.Card"
    bind="addCard"
    unbind="removeCard"
    cardinality="1..n"
    policy="dynamic"/>
</component>

This is what tells DS to call the Controller class when a game.model.UserInterface and a game.model.Card service is registered. It defines a component called 'controller', specifies its class and service dependencies. I've set the policy to 'dynamic' so I get notified every time one of these services gets bound or unbound. Matter of fact, this component configuration file actually causes my game.Controller class to be instantiated in the first place so that's why I don't need an OSGi Activator here. DS does it for me.
There's one last handy little nugget in here. You can see that the cardinality of the service dependencies is specified. For user interfaces it's 0 or more, but for cards it's 1 or more. The latter is handy, because it makes sure that my game Controller doesn't get created before there are a few cards registered. There's no point offering a card game to play without any cards!
So as soon as there's a few cards, my game.Controller class gets created and players can make themselves known by registering a UI.

We will need to tell DS about our OSGI-INF/controlcomponent.xml file. For this I have to reference it in the META-INF/MANIFEST.MF file. Mine looks like this:
  Bundle-Name: Duel Bundle
  Bundle-SymbolicName: Duel
  Bundle-Version: 1.0.0
  Import-Package: game.model
  Require-Bundle: org.eclipse.equinox.ds
  Service-Component: OSGI-INF/controlcomponent.xml

I won't go into too much detail on the game.GameImpl class. This is an implementation of the game.model.Game interface. It's not very complicated but it's just a bit too long for a blog post; so just look at the code :)
You can get the Game Implementation class with the Duel bundle from Subversion here: http://coderthoughts.googlecode.com/svn/trunk/duel/.

We can't start the game just yet, we first need to add a Bundle with a few cards. That's what the next posting is about.

OSGi Services for Dynamic Applications (I)

2007-11-29T23:00:00.000-08:00

Or: use OSGi to write a dueling card game

Surprise is one of the interesting things about dueling card games. You might know Magic the Gathering or Yi-Gi-Oh, which is quite popular with kids at the moment, many others exist too. They are interesting because there is a sheer endless number of cards so you never know what card your opponent will play. On the other hand, I never really liked the fact that whoever spends the most money on them always wins. I'd rather prefer a variant with a shared deck so that player's chances are even.

I've wanted to write a little framework for a game like this for a while now and OSGi is a great technology for doing this. Its bundle architecture brings some nice modularization and especially the OSGi Services Framework allows me to plug my services in and out at runtime without having to restart other bundles that make use of my services.
In my case, I will use this to add a new sets of cards to my game without interrupting my ongoing games, the new cards will even appear straight away in any ongoing games. I will do this in one of the next postings.

So the basics of my dueling game are similar to the games above and go like this. Each player starts with 250 points, whoever hits 0 first loses. Each player plays in turn:

Draw phase - generally just draw a card
Standby phase - here you can cast spells and a monster. Some spells are cast on your own area or attached to one of your monsters, other spells can affect the opponent.
Battle phase - now you can attack with your monsters that are on the table. The opponent can defend with any monsters that are on her side of the table. If the defence of the defender is higher than the attack of the attacker, the defender wins, and vice versa. If the defender does not have any monsters on the table, she has to take the damage, which will cause the attack amount to be deducted from her life points.
Recover phase - play spells and possibly a monster if you haven't already in this turn.

The very first battle phase is skipped (as it would be unfair to attack an opponent who didn't get to play any cards yet).

Monsters and spells often have special abilities that could be tied to a particular phase. They could change the behaviour of other cards or generally disturb the order of the game.
Another feature cards in these games generally have, is some stunning artwork which, unfortunately, I won't be able to provide.

I have come up with a little architecture that looks like this:

I'm using OSGi-style diagramming here. The yellow triangles are services, the red boxes are bundles, which provide some implementations of the services, the blue box is a component which itself lives inside a bundle.

The Game Controller is a consumer of OSGi services. It consumes any number of Card services, it also consumes 2 UI services, one UI per player.
So every card is a service of its own and every user interface is a separate service instance. As long as we adhere to the Card interface we can add any cards we can think of. Also using the service approach for the UI allows us to develop the UI separately from the game and gives us the possibility to implement any type of GUI, a simple one, a fancy one, an AI player (without a screen) or maybe even a remote UI without changing the actual game.

OSGi doesn't use the static wiring for services that you can see in some other Service Oriented Architectures, in OSGi a service consumer specifies the Java interface that it is interested in, the OSGi runtime will then dynamically bind this to an available service that implements this interface. This is completely dynamic, and services could even come and go during the life time of the system. BTW I'm not going to go too much into the OSGi basics. Neil Bartlett's blog has some excellent OSGi Basics tutorials.

Let's start by diving into the the core model of the game. (BTW I would really have loved to use EMF here again, but I wanted to keep this posting focused so I'm just using the EMF Class diagram editor for the picture, and am in this case handcoding the classes) - click on the image for a clearer picture:

Entities in italics are interfaces, others are real classes. Let's look at the interfaces that define the various types of cards first. I put a few example cards in the middle. The top interface is called Action, everything you can do and every card is an action. Cards generally implement both a subclass of Action as well as the Card interface. However there are certain actions that are not a card, such as the DrawCommand and NextPhaseCommand actions.
Commands are things that can executed, usually in a particular Phase of the game. Example command cards are the ThunderBoltCard spell which directly damages the opponent, or the SkipBattlePhaseCard which ends the current Battle phase.
Fighters are things that can take part in a battle, generally monsters, but also the player himself. Modifiers can change the attack and/or defense of a Fighter.

Then there are the following classes and interfaces:

Game: controls the game and provides information to the various entities about the game state. Provides accessors to change the game state too.
Player: data class that holds player information, such as the cards she holds in her hand, table and graveyard. Also holds a reference to the UI service that this player uses and the name of the player.
UserInterface: this API defines the interaction with the UI. The purpose of the UI is simply to display the game state and get any input decisions (actions) from the player. The UI can also display a line of text to the user.

Since my dueling game revolves around these classes, I have put them in a separate OSGi bundle. Other bundles can participate in the game by importing these, without having to depend on any implementation bundles. This OSGi bundle is available in Subversion here. If you're using Eclipse, the easiest way to get it into your Eclipse IDE is with a Subversion client. I always use subclipse, which is extremely easy to install; instructions here.
Once you have subclipse, open the SVN Repository Exploring perspective and add the following SVN repository URL: http://coderthoughts.googlecode.com/svn/trunk/duel/. There's no need to log in, if you expand the root node, you will get the following repositories:

Just check out the DuelInterfaces project and you're done. You will then have the OSGi bundle in your local Eclipse workspace, it will be the basis for the other pieces of the game that I'll be writing next.

For more details on the DuelInterfaces classes, read their Javadoc here.

In the following few postings I will be implementing my dueling game using more OSGi bundles and services. I'll be using a mixture of OSGi Declarative Services (DS) and direct OSGi API calls for this. Besides DS and the plain API approach (which are both part of the OSGi spec) there exist a number of other technologies to work with OSGi services: Spring-OSGi, iPojo and Guice-OSGi are some of them. The nice thing is that it doesn't matter what toolkit you use as the actual service registered with OSGi conforms to the OSGi Service Spec, so you can mix-n-match all these technologies if you like.

Besides the game logic bundle, I will be writing a bundle with monster cards and another with spell cards and I'll also provide a basic UI implementation written using SWT, which looks like this:

The good thing is, it allows me to play the game and I can write a nicer GUI later if I want, without changing the rest of the code. I might at some point even want to do a GUI that runs remotely, which allows me to use my OSGi-based application as a game server that can be used by people playing the game in various locations from a browser or something like that.

As usual I'll be using the Eclipse IDE and Equinox OSGi implementation, but nothing in the code is tied to these so please use your IDE and OSGi implementation of choice.

Superfast Model Driven Development with EMF (III)

2007-10-20T15:53:00.000-07:00

Load and Save

In the previous postings (part I and part II) I've created an EMF model to hold the definition of an adventure game. I've used the EMF-generated editors to create an adventure and wrote a little bit of code to run the game.

In this posting I'm going to look at how we can use the EMF built-in serialization as a very easy way to load and save the game state. At the end I will also show how to run the program from the command line outside of Eclipse.

To implement Load and Save, I'm adding two new actions to every location in the game (a player can load and save in every game location): SaveAction and LoadAction.

The SaveAction creates a new SaveGame instance and stores the current location and the current inventory in it.

Looking at the model again, you can see that the SaveGame contains an Inventory object but only has a relation to the Location. This has an impact wrt to the content of the file that we're saving to. The contained items are actually inlined in the file where the referenced items are stored as hrefs.

A sample adventure.save file would look like this. The format of this XML file is called XMI, but as you can see, it reads as very typical XML. You can see that the inventory is actually contained in the file but the location is pointed to. When we save the file, it is in the same Resource Set as the My.adventure resource, so that's how they know about each other. The href URL also explains why we need an ID attribute. It is used in there. The ID attribute value cannot have a space in it so that's why it's generally handy to use a separate attribute for it.
<adv:savegame version="2.0" xmi="http://www.omg.org/XMI" adv="http://adventure">
  <inventory>
    <items name="Spade" id="spade"/ >
  </inventory>
  <location
    href="file:/C:/EMF_Adventure/bin/My.adventure#desert"/>
</adv:savegame>

The adventure.save file is created using the following code:
SaveGame sg =
  AdventureFactory.eINSTANCE.createSaveGame();
sg.setLocation(Status.location);
sg.setInventory(Status.inventory);

String fileName = System.getProperty("java.io.tmpdir") +
  "adventure.save";
Resource resource = Status.resourceSet.createResource(
  URI.createFileURI(fileName));
resource.getContents().add(sg);
try {
  resource.save(Collections.emptyMap());
  System.out.println("Game saved to " + fileName);
  return true;
} catch (IOException e) {
  e.printStackTrace();
} finally {
  Status.resourceSet.getResources().remove(resource);
}

First we create and fill the SaveGame object.
Then we add a new Resource, stored in a file called adventure.save in the temp dir, to the ResourceSet used by the game.
Add the SaveGame object to the new Resource.
call resource.save()
Finally remove the new resource from the game's ResourceSet again as we don't want it to change after this.

And we're done.

Loading data from an XMI file is very similar. The LoadAction does the following:
String fileName = System.getProperty("java.io.tmpdir") +
  "adventure.save";
Resource resource = Status.resourceSet.getResource(
  URI.createFileURI(fileName), true);
try {
  SaveGame sg = (SaveGame) EcoreUtil.getObjectByType(
  resource.getContents(),
  AdventurePackage.eINSTANCE.getSaveGame());
  if (sg != null) {
    Status.location = sg.getLocation();
    Status.inventory = sg.getInventory();

    System.out.println("Game loaded from " + fileName);
    return true;
  }
} finally {
  Status.resourceSet.getResources().remove(resource);
}

Create and add a resource that points to the adventure.save file. In this case we use ResourceSet.getResource() instead of ResourceSet.createResource().
Then we look up all of the contents of the file and find the SaveGame object in it. Since the adventure.save file really should only contain a single SaveGame object, resource.getContents().iterator().next() would also do the job.
Once we have our SaveGame object, simply change the current location of the game to be the one loaded and change the inventory to be the one from the savegame.
Finally, remove the adventure.save resource from the resourcset and were done.

Well, not quite, there is actually a little problem with the above code in that we should actually revisit all the locations of the adventure game and reset the items that belong there, taking into account the items that we've just loaded, but I'll leave that for another day :)

Oh, and I'm adding the following 2 lines to Status.initializeActions() so that my load and save actions are available from every location in the game:

actions.add(new SaveAction());
actions.add(new LoadAction());

Running the program from the command line

We're not writing an Eclipse application here, we're only using Eclipse and EMF as a tool to build our app. So we just want to run everything from the commandline. To do this, add to the following jars to your CLASSPATH or copy them into your project. For EMF that comes with Eclipse 3.3.1 the needed libraries are the following, but the actual number could be slightly different if you are using a different version of Eclipse (see the EMF FAQ for more info):

org.eclipse.emf.common_2.3.0.v200709252135.jar
org.eclipse.emf.ecore_2.3.1.v200709252135.jar
org.eclipse.emf.ecore.change_2.3.0.v200709252135.jar
org.eclipse.emf.ecore.xmi_2.3.1.v200709252135.jar

Now I can run my adventure game from the command line with:


set CLASSPATH=.;
  org.eclipse.emf.common_2.3.0.v200709252135.jar;
  org.eclipse.emf.ecore_2.3.1.v200709252135.jar;
  org.eclipse.emf.ecore.change_2.3.0.v200709252135.jar;
  org.eclipse.emf.ecore.xmi_2.3.1.v200709252135.jar
java game.Main

Welcome to Adventure Island. This is the start of the adventure game. You are at the house of a Viking.
Hints: (q)uit, (i)nventory, (w)est, (e)ast, save, load, you see a Spade
Please enter command:
All the code above is available in the Subversions repository at http://coderthoughts.googlecode.com/svn/trunk/emf_adventure for more info on how to check it out look here. Want all the projects including the generated code, get it here: http://coderthoughts.googlecode.com/svn/trunk/emf_adventure_full

Superfast Model Driven Development with EMF (II)

2007-10-20T13:36:00.000-07:00

Creating the actual adventure game
In the previous posting I used EMF to create a model and the generated editor to create a file (My.adventure) containing the definition of my text adventure game. In this posting I'm going to create the actual game.

Copy and paste the My.adventure file created in the previous posting into the src directory of the EMF_Adventure project. Also create a game package where we'll do our development. One of the reasons to do the actual development in a different place or different package than the code generated by EMF is that this way, if you make a change to your EMF code you can simply delete all the generated code and regenerate it, without having to watch out for your own files. So the EMF-generated code is in the adventure package and our code is in the game package.

The little text adventure that we'll be writing allows you to move from one location to another. It allows you to pick up items you may find or perform actions in order to find hidden items. Certain items can only be obtained once you have other items first. For example, in the sample adventure game definition (which is based on a game I wrote with my kids, get it here) you need a spade in order to dig up the treasure.

To give you an idea, here's the output from a sample running of it:
Welcome to Adventure Island. This is the start of the adventure game. You are at the house of a Viking.
Hints: (q)uit, (i)nventory, (w)est, (e)ast, save, load, you see a Spade

Please enter command:

> pick up spade
You pick up Spade...

Viking's house.
Hints: (q)uit, (i)nventory, (w)est, (e)ast, save, load
Please enter command:

> w
You're going west

You are at the Viking's boat. Will you help the Viking with his fishing?
Hints: (q)uit, (i)nventory, (e)ast, save, load
Please enter command:
etc...

I know, it's text only, which is slightly boring, but the point of this posting is about EMF. We've added images to the locations and that's actually quite easy to do, but I'll leave it up to yourselves.

The first class we'll make is the Main class which kicks off the game and holds the game loop:
package game;

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class Main {
  public static void main(String[] args) throws Exception {
    Status.intializeGame();

    for (;;) {
      Status.describeLocation();
      String command = new BufferedReader(
        new InputStreamReader(System.in)).readLine();
      Status.executeCommand(command);
    }
  }
}
The Status class holds the game state and controls the state transitions. The first thing it does it read the EMF model instance we've created that defines the game:
public class Status {
  static Adventure adventure;
  static Location location;
  static Inventory inventory =
    AdventureFactory.eINSTANCE.createInventory();

  static Map<Location, List<Action>> actionMap =
    new HashMap<Location, List<Action>>();
  static ResourceSet resourceSet = new ResourceSetImpl();

  public static void intializeGame() throws IOException {
    resourceSet.getResourceFactoryRegistry().
      getExtensionToFactoryMap().
      put(Resource.Factory.Registry.DEFAULT_EXTENSION,
      new XMIResourceFactoryImpl());
    resourceSet.getPackageRegistry().
      put(AdventurePackage.eNS_URI,
      AdventurePackage.eINSTANCE);

    URI gameDefURI = URI.createURI(Status.class.
      getResource("/My.adventure").toString());
    Resource resource = resourceSet.getResource(
      gameDefURI, true);
    adventure = (Adventure) EcoreUtil.getObjectByType(
      resource.getContents(),
      AdventurePackage.eINSTANCE.getAdventure());
    location = adventure.getStartLocation();
    initializeActions();
  }

In EMF objects are persisted in a resource, which could take many forms. I am using the XMIResource which basically is an XML file in the XMI format. Not all the objects in an EMF model have to be in the same file, objects could be spread over multiple files. To associate these files with each other EMF puts them in a ResourceSet.

The first two lines of initializeGame() set up the ResourceSet to use XMI resources and also informs the EMF package registry about our Adventure model.
Another thing you might notice in the class is the creation of the inventory object. Instead of going inventory = new InventoryImpl() (which would actually work) EMF prefers you to use the Factory pattern. So we create our inventory object by calling createInventory() on the factory instead.
Next, I'm loading the game definition from the My.adventure file. EMF wants to have a URI to the resource (rather than a simple file path).
After this I use the EcoreUtil.getObjectByType() to find my single Adventure object in the Resource that has loaded the file. There are other ways to get to the objects in a resource, the simplest way is just calling resource.getContents() which returns a list of the objects in the resource. The AdventurePackage.eINSTANCE.getAdventure() returns an EClass object that represents the Adventure EMF entity. EMF has its own metamodel which is richer than the Java reflection metamodel. It basically allows you to find out all the things that you have specified in your model, over and above what can be reflected in the Java signatures. But in a way AdventurePackage.eINSTANCE.getAdventure() in EMF-speak is similar to what would be Adventure.class in Java-speak.

We now finally have our adventure object and life becomes much easier. At this point can pretty much use the EMF model instance by utilizing Pojo bean-conventions. So we set our initial location to be the location that was marked as the start location in our Adventure definition file.
Last but not least we'll initialize the actionMap.

The actionMap is a map from a location to a list of Actions that are available on that location. These actions can be added statically (e.g. the 'quit' action is available everywhere) or dynamically, such as actions that relate to routes you can take from a particular place or actions associated with items. All actions in the game implement the game.Action interface:
public interface Action {
  /** A hint to the user how to execute this action, or
  * null if there is no direct hint.
  * @return A hint such as "(q)uit" or "You see a Spade"
  */
  String hint();

  /** Called on the action to see does it want to execute
  * the command. The command is passed in as an array of
  * strings. If the action does not want to execute the
  * command it can just return false.
  * @param commands The commands in an Array e.g.
  * ["pick", "up", "spade"]
  * @return true if the action was performed,
  * false otherwise.
  */
  boolean execute(String[] commands);

  /** The minimum number of words needed for this action
  * to qualify. So if your action is triggered by "pick
  * up spade" it would return 3. The action will not be
  * visited if the number of commands is less.
  * @return The minimum number for words required to
  * visit this action
  */
  int minLength();

  /** Whether or not to remove the action after
  * performing it. An item action would typically be
  * removed after you've picked it up (you can't pick
  * it up twice) but other actions normally remain.
  * @return Whether or not to remove the action after
  * successful execution.
  */
  boolean removeAfter();
}

Status.initializeActions() initializes the map of all the locations in the game with the actions that are available there.

Status.describeLocation() simply prints out the description of the current location and also prints out the hints for all the commands available here. Status.executeCommand() parses the string typed in by the user and then visits the actions registered with the current location to see whether any one action wants to execute the command. See here for the rest of the Status class, apart from the initializeGame() it's fairly simple Java. There are also a number of actions provided, such as InventoryAction, ItemAction, QuitAction and RouteAction which all take care of their particular things.

Finally, because we're using the XMI functionality to load the model, we need to make sure to specify this dependency in the bundle that holds our code. EMF generates code as OSGi bundles/eclipse plugins which use the META-INF/MANIFEST.MF file to describe their dependencies, for those who want to run this as a simple Java app we'll see later in this posting how to get these jar files on the classpath. For now just open the MANIFEST.MF file and in the Dependencies tab add 'org.eclipse.emf.ecore.xmi' to the required plugins.

Now we can actually run our game. To do this open the Main class, right-click in the editor and select 'Run As -> Java Application'. The game will start:

Welcome to Adventure Island. This is the start of the adventure game. You are at the house of a Viking.
Hints: (q)uit, (i)nventory, (w)est, (e)ast, you see a Spade
Please enter command:

All the code above is available in the Subversions repository at http://coderthoughts.googlecode.com/svn/trunk/emf_adventure for more info on how to check it out look here. Want all the projects including the generated code, get it here: http://coderthoughts.googlecode.com/svn/trunk/emf_adventure_full

In the next posting I'll use EMF to provide savegame/loadgame functionality and run the game as a plain Java App that uses EMF outside of Eclipse.

Superfast Model Driven Development with EMF (I)

2007-10-18T23:59:00.000-07:00

Or: write a text adventure game in no time

One of the technologies that I use quite a lot is EMF. It has helped me increase my development speed enormously, and I use it pretty much every time I need a little data model.

What is lesser known is that you don't actually need to be writing Eclipse plugins to take advantage of EMF. You can use EMF in any old Java program. Another thing Eclipse gives you is tooling around EMF and tooling for your own model, which makes it super handy.

So with the Eclipse tooling you get a nice UML editor on your EMF class model. Eclipse generates all the EMF runtime code for you, and it also generates a model instance editor, so basically you get a data entry tool for your model as well, which can be quite useful.
In addition to the modeling stuff, the EMF runtime also gives you a loading and saving of your model (to an XML file format called XMI). All of this stuff is available for free as open source.

So I've used it to write a little text adventure game framework (add your own graphics later ;) You can define the actual text adventure in the EMF model. EMF is also used to provide the load and save game functionality.

So I started off with Eclipse 3.3.1 with EMF and GMF installed (see here how that's done). We need GMF to do the graphical UML editing. I've posted a little how-to on this, in case you need help.

In this posting I'm creating the model and will generate a data entry editor for my model. The next posting will focus on the actual game and how you run it as a simple Java app.

1. Create the model
Create a project called EMF_Adventure.model and add an Ecore Model file to it (in Eclipse go to File | New | Ecore Model) and call it Adventure.ecore.

Then right-click on the Adventure.ecore file in the tree, and select 'Initialize ecore_diagram diagram file' from the popup menu. Double-click the Adventure.ecore_diagram that has just been generated and you're in the UML editor (click on the image for a clearer picture):

So now we actually create our model. My little model for the adventure game looks like this:

My model contains the definition of the structure of an Adventure.

The Adventure entity contains a number of Locations (containment is visualized using the diamond UML notation). One of these locations is to be the start location of the game.
Locations in turn have routes to other locations, they also possibly contain items. Every location has a name and a description and an ID.
A Route object describes how to get from one location to another.
Items are things you can find on a location. Certain items are visible, you just see them and can pick them up. Other items are a hidden, for instance, if you want to find the treasure you will need to dig, otherwise you don't see it. In this case there might be a precondition on another item that you need to have in your inventory to perform this action. In order to dig you may need a spade.

Besides the adventure definition, the model also contains a representation of the Inventory which contains all the items you have picked up during the game. EMF is quite nice in that it make sure the model is consistent. This means that contained objects can only have one container. So by adding an Item to your Inventory EMF will automatically remove it from the Location.

Finally the definition contains an entity used for saving and loading games. The SaveGame entity has a reference to the location where you saved the game and also holds the inventory that you had when you saved.

Another way to look at the same model is by opening the Adventure.ecore file, which gives you a tree-type view on the model.

You can either edit the model using the diagram editor or using the ecore editor. They generally keep each other in sync.

Find the Adventure.ecore file that I created here .

2. Generate a data entry tool for your model
The first thing you will need to do in order to use your EMF model is to generate code from it.

To generate your code you need to create a .genmodel file. Do this by going File -> New -> Other and then select 'Eclipse Modeling Framework | EMF Model'.

Just call it Adventure.genmodel, select 'Ecore model' as the Model Importer on the next page and then pick your Adventure.ecore file from your workspace. Then finish the wizard.

The Adventure.genmodel file contains a whole lot of settings that influence how the EMF code generation is done. I normally change at least one setting. By default EMF generates code in the project that holds the .genmodel file. I always prefer to have the generated code in another project, so I typically change the following property:
Model Directory: change from /EMF_Adventure.model/src to /EMF_Adventure/src
Then right-click on the top-level Adventure node in the Aventure.genmodel file and select 'Generate All':

The model generation gives us a couple of options:

Generate Model Code - this is the code that we will be using at runtime. Its an implementation of the model that we defined.
Generate Editor Code - this generates an Eclipse editor to fill your model with data. It can be really handy to populate your model. The Edit Code that is also generated is used internally by the editor.
Generate Test Code - this will create a plugin that contains test templates and some sample code that utilizes your model.

After running the generation you will get 4 new projects in your workspace that all compile without errors:

In this screenshot (again click on it for a high-res version) you can see some of the generated code. Currently open is the Item class which contains getters and setters for all the properties we defined in the model. Both interface and implementation are generated for the model, our implementation is generated in the adventure.impl package.

Now we can run it to create our adventure content! Just right-click any of the newly generated projects and select 'Run As -> Eclipse Application'.
This starts up a separate Eclipse Runtime that contains your newly generated editor. Create a new project in it and add a new adventure model file by going 'File -> New -> Example EMF Creation Wizards -> Adventure Model':

Call it My.adventure and select 'Adventure' as the Model Object in the last wizard page.

Once you have your file, you can double-click it and voilà there is your editor where you can define your adventure. It will allow you to create all your locations, items and routes. It also has a properties view where various properties are edited. And this all without writing a single line of code!

The actual file edited is an XML file. I created one based on a little adventure I made with my kids a little while ago, you can find it here .

In the next post, we'll create the adventure runtime.

How to get the EMF UML editor bits into Eclipse

2007-10-18T13:28:00.000-07:00

This is not really a blog entry per se, its more a little how-to on getting your tools set up for the next blog entry.

All of the tools required here are open source.

Well, first you have to get Eclipse: I've downloaded the 'classic' distribution of Eclipse 3.3.1 from www.eclipse.org.

Then use the Eclipse update mechanism to get the EMF bits.

Start Eclipse and select the menu 'Help -> Software Updates -> Find and Install' (click on the image for a better picture)

Select 'Search for new features to install', then select the 'Europa Discovery Site':

In the following wizard page (called 'Search Results') select 'Graphical Modeling Framework' in the 'Models and Model Development' section. Then hit the 'Select Required' button.

I'm selecting GMF because that thechnology provides the UML-style class diagram editor for EMF. Since GMF depends on EMF, you will get all the needed EMF bits too by hitting the 'Select Required' button.

Hit next 'Next' a couple of times and then hit Finish.

Eclipse will now download and install EMF.

First posting

2007-10-04T14:15:00.000-07:00

Le Code pour le Code - or: Just for the fun of it :)