Altering OSGi Service Lookups with Service Registry Hooks

OSGi Services are great. They can solve many problems very elegantly. Take for instance SPI patterns. SPI patterns are used to make implementations pluggable. This type of pattern outside OSGi often comes with a bunch of external configuration (e.g. in META-INF/services) that makes it hard to manage and gives you a once-off chance to choose your implementation that you're stuck with for the rest of the VM lifecycle. OSGi Services provide a much more elegant way to solve this. 
 

In many cases clients simply want to use some functionality and don't particularly care how that functionality was created. When using OSGi Services the functionality is looked up in the OSGi Service Registry which is a directory of these. Services can be looked up by implemented interface (like a phone book) or by provided attribute (like the yellow pages). How the service got there is not interesting to the client and he's not involved in that. What if the service gets replaced while the client is active? No problem, the OSGi Service Programming model is actually built around this dynamicity. You don't need to stop service consumers when you are replacing or updating the service, they are automatically rebound.

The Service Registry also provides us with an attribute based selection mechanism which is very nice if there are multiple implementations to choose from. As an example, take a system where every available printer is represented in the Service Registry as a Service that implements the org.acme.Printer interface. Each printer will have additional properties registered that help with the selection:

When looking for a Printer OSGi allows you to use an LDAP-style filter to select the printer you want. So if you want a printer that can do A3 you would use this LDAP filter:

 
(&(objectClass=org.acme.Printer)(paper-size=A3))

or if you want a printer in a location that starts with 'b' you do this:

 
(&(objectClass=org.acme.Printer)(location=b*))

This is all great and you can build your system on this using either plain OSGi code or component frameworks such as OSGi Blueprint or OSGi Declarative Services. However sometimes you may want to tweak the properties at a later date without having to rewrite your system. Assume that in your organisation all of a sudden the office numbering has changed, which would really mean that you'd have to update the 'location' attribute of all the printers. Or maybe you want to add some additional metadata to existing printers that influence the selection process, like their energy rating.

Service Registry Hooks provide the building blocks that make this possible. I wrote a little bundle called ServiceJockey that uses these to manipulate service registrations and how they are looked up. BTW you can find details at the end of this posting about getting ServiceJockey, which is open source and available freely under the Apache License.

Service Registry Hooks: what are they?

One of the driving factors for Service Registry Hooks (an OSGi standard introduced in the 4.2 Core specification) was the Remote Services (Distributed OSGi) work done in the OSGi Alliance. One of the things that implementations of the Remote Service spec need to know is what kind of services consumers are looking for so that they can go out to a discovery system to see if it might be available remotely. Eagerly registering all available remote services is clearly not scalable so we need a smarter mechanism to allow for transparent on demand discovery of remote services. This is what the ListenerHook and FindHook provide us. Together they allow us to find out what service consumers are requesting making us to only look in a remote Discovery system for those that are relevant to the current framework.

Another problem was frequently brought up. What if you have an existing bundle that doesn't know anything about Remote Services? What is the default behaviour? Should all service lookups all of a sudden always include remote services? The Remote Services spec does include a special property that you can add to your filter to influence this (service.imported) but you clearly don't want to break up all your existing service consumers to add this extra property to their lookup filters.

It turned out that there wasn't really a valid answer to that question applicable to all cases. If you're working on a system only using Remote Services for a select set of services it may make sense to default all other services consumers to not use Remote Services. On the other hand, maybe if you're building a Cloud-based infrastructure where services can move freely from one container to another the default behaviour could be that you do allow remote services for just about anything.

Since there isn't really a one-size fits-all here the Service Registry Hooks make it possible to provide a policy for this problem in a separate bundle. You can create EventHooks and FindHooks to influence what services consumers can see. This effectively allows you to add extra conditions to the lookup of service consumers without the need to modify those consumers.

Finally, EventHooks and FindHooks, together with the a ServiceListener, allow you to proxy Service Registrations, where you provide a second registration of the same object with modified properties, hiding the original.

The ability to provide an alternate registration on the fly and the possibility to impose extra conditions on existing service consumer lookups is what I built ServiceJockey around.

ServiceJockey doesn't replace the Service Object, it only effectively replaces the registration in the Service Registry plus it can put additional constraints on client lookups. But you could go further. If you wanted to actually replace or shadow the real service object and do some work when somebody uses it (maybe log it or something) you could put an additional object between the consumer and the original service object.

Because the Service Registry Hooks effectively modify some basic behaviour of the framework (the visibility of Services) they should really be started early in the Framework lifecycle. You can achieve that by giving them a low start level.

The Service Registry Hooks are available in Felix 1.8.0 and Equinox 3.5 and newer versions of these.

Altering Service Registrations

Let's start with a bundle that consumes (uses) a Printer. It isn't interested in which printer, as long as it can print A4:

BundleContext context = ... // from Activator.start()

Filter filter = context.createFilter(

  "(&(objectClass=org.acme.Printer)(paper-size=A4))");

st = new ServiceTracker(context, filter, null) {

  public Object addingService(ServiceReference ref) {

    // print out some information on the printer

    // or use the printer

    return super.addingService(ref);

  }

};

st.open();

When I run this it's reporting both printers that I have in my system:

Printer:

objectClass: [org.acme.Printer]

service.id: 27

name: p1

location: b283

capabilities: [Double-sided]

paper-size: [A3, A4]

Printer:

objectClass: [org.acme.Printer]

service.id: 28

name: p7

location: a12

capabilities: [Colour, Staple]

paper-size: [A4, Letter]

So the client has arbitrary access to both of them. Now lets see if you can modify the client behaviour so that it only uses a printer with an Energy Rating < 50.

But hold on, we don't even know about energy ratings yet! We need to add this information to the Printer registration without changing the bundle(s) that register the Printers. Let's user Service Jockey to do this.

Service Jockey uses the OSGi Extender Model. This means that it is driven from a data file in a bundle.

So I've got a bundle (called PrinterJockey) that contains META-INF/sj.xml:

<service-jockey xmlns="http://www.coderthoughts.org/schemas/sj/v1.0.0">

 <proxy-registration>

  <rule>

   <service-filter>(&(objectClass=org.acme.Printer)(name=p1))</service-filter>

   <add-property key="energy-rating">75</add-property>

  </rule>

  <rule>

   <service-filter>(&(objectClass=org.acme.Printer)(name=p7))</service-filter>

   <add-property key="energy-rating">50</add-property>

  </rule>

 </proxy-registration>

</service-jockey>

I will have to tell the Service-Jockey that my bundle contains configuration for it, for that I'm adding a header to the Manifest:

Service-Jockey: META-INF/sj.xml

Lets run the Printer Consumer bundle together with Service Jockey and my 'Printer Jockey' configuration bundle that contains the sj.xml file:

id State Level Bundle

0 ACTIVE  0 org.eclipse.osgi_3.5.1.R35x_v20090827

1 ACTIVE  1 org.coderthoughts.servicejockey_0.0.1

2 ACTIVE  1 PrinterJockey_1.0.0

3 ACTIVE 10 Printers_1.0.0

In my case the Printers bundle both registers and consumes the Printers, but that's because its a little test bundle. It's not really relevant. Note that the Printers bundle has a higher start level than the Jockey ones and therefore starts later.

When I run my system again I can see that it's doing some work. Now my client reports these services:

Printer:

objectClass: [org.acme.Printer]

service.id: 30

name: p1

location: b283

capabilities: [Double-sided]

paper-size: [A3, A4]

energy-rating: 75

.ServiceJockey: Proxied

Printer:

objectClass: [org.acme.Printer]

service.id: 32

name: p7

location: a12

capabilities: [Colour, Staple]

paper-size: [A4, Letter]

energy-rating: 50

.ServiceJockey: Proxied

Aha! I've got my energy rating in there! That's good. Remember that these are alternative registrations. The original ones haven't changed, they're just hidden.

Altering Client Side Lookups

Now I want my client to only use the one that is rated <= 50.

That's done by adding another rule to the Service Jockey configuration. A rule that adds an extra filter to any matching lookup in a consumer.

<service-jockey xmlns="http://www.coderthoughts.org/schemas/sj/v1.0.0">

 <restrict-visibility>

  <rule>

   <bsn>.*</bsn>

   <service-filter>(objectClass=org.acme.Printer)</service-filter>

   <add-filter>(energy-rating<=50)</add-filter>

  </rule>

 </restrict-visibility>

 <proxy-registration>

  ...

 </proxy-registration>

</service-jockey>

You can specify a regular expression to say what the Bundle Symbolic Name of the consumer bundle(s) is that the rule applies to. In my case .* matches anything. So it applies to any bundle that has OSGi Service consumers. By the way, there's also a tag.

You specify to what services the additional filter applies with the tag and you specify the additional filter in . In my case I'm adding the (energy-rating<=50) condition to any Printer Service returned to a consumer (the condition looks a bit dodgy because of the XML escaping of the '<' sign - you could use a CDATA section instead...).

Note that the Service Filter applies to any service returned to a service consumer, regardless of the filter used by the client itself looks like.

Let's run the client again:

Printer:

objectClass: [org.acme.Printer]

service.id: 32

name: p7

capabilities: [Colour, Staple]

location: a12

paper-size: [A4, Letter]

energy-rating: 50

.ServiceJockey: Proxied

Bingo - it only selects the service I wanted it to select. Based on additional properties and criteria listed in my Service Jockey configuration file. I did not have to modify the Printer Service registration bundle or the Consumer bundle...

Is it overkill?

Seems like a lot of work for just adding a service property, is there not a lighter way to at least add values to Service Registrations? Well at least that was my initial impression. However looking at the ServiceJockey bundle, it's only 14kb. Besides that there is currently no other way to achieve this...

Service Jockey - ride your service interactions

I didn't show any of the code to actually use the EventHook and the FindHook. Have a look at the HidingEventHook and HidingFindHook source code for that.

You can check out the source (Apache License) as an Eclipse Project from SVN here. If you fancy modifying the code, I would recommend you also get the tests project and add new ones. The test bundle depend on the Mockito bundle for its mock objects...

I'm sure the Service Jockey isn't perfect, because I only wrote it during an airplane flight so feel free to send patches :)

I also put a binary download of ServiceJockey here.

Also in the source tree, Eclipse projects for the Printers and PrinterJockey example bundles.

A single Calendar from Desktop to Phone

This year I upgraded the really old smartphone I had to a Sony Ericsson C905. It almost looks like I'm one of the last people left who doesn't have an iPhone and doesn't want one (I'm resisting the voluntary lock-in ;).
I'm really happy with the C905 because it has all the features I wanted (Wifi, GPS, 8MP camera, loads of memory) but it still looks like a phone and it's not too big.

One problem I was starting to run into was that I had too many calendars going. I had the calendar in my email client (Thunderbird + Lightning), I had my Google Calendar and now the phone also has a calendar. I really wanted a single calendar that works on all of my devices. I had already been playing with the Google Calendar Provider for Lightning and quite liked it. If only there was something like that for the phone...

After a little digging I found out that Google supports the Exchange ActiveSync protocol. And my Sony Ericsson (and many other models too) supports it as well! It's not very well documented so I decided to write down the steps I took to get it all working.

1. Go to the Organiser and select Synchronization.

2. Create a new Account.

3. Select Exchange ActiveSync and give it a name, I called mine 'Google'.

4. In the General tab, enter as the server address: https://m.google.com also provide your Google user name and password. I didn't provide any other information here.

5. On the applications tab I only selected Calendar, although the Contancts and Email integration probably also work.

6. Now it asks me to delete all my contacts and calendar items. Very annoying because I don't want to sync my contacts, but anyway, for best results you have to clean them all. So I backed up my contacts previously and select clean here. I can restore my contacts later...

Ok - I'm all set up!
Let's create a calendar event on the phone:

Synchronize with Google...

I think you can even do this automatically, but I haven't enabled that because I don't want my phone bill to explode :)

Let's have a look at Google mail:

Yep, it's there!
Now let's see in Thunderbird Lightning:

And in all three modes I have read/write access. Aaaaah, nice!

An IRC alerter written using Apache Camel and Java

IRC is a great tool if you have a bunch of people working on the same thing in different locations. One of the biggest problems I found with it is that I sometimes miss pings. I don’t always have the IRC window open, and although the client I use most (Trillian) has nice popups in the corner, I still sometimes miss a ping, simply because I’m away from my computer. I know there are some other IRC clients that can send you special alerts, but I just happen to like Trillian, so I didn’t want to abandon it just because I was missing one little thing...

While being annoyed with this for a little while, I came across the Apache Camel IRC component. Great - that should sort me out. I’ll just simply write a Camel route that connects to the IRC channel and sends me an alert of some sort when someone pings me.

After running the Camel Maven archetype for Java projects:
>mvn archetype:generate -DarchetypeGroupId=org.apache.camel.archetypes -DarchetypeArtifactId=camel-archetype-java -DarchetypeVersion=2.0-SNAPSHOT -DarchetypeRepository=https://repository.apache.org/content/groups/snapshots-group
I was able to get this going fairly quickly! The archetype creates a Maven Camel project for you with an example route in it. You can run it now with:

mvn install camel:run

Next step: write a Camel route that reads my IRC messages for me.
For this I’m simply using a URL in Camel to connect to the IRC server, e.g. I’m using the following to connect to the CXF IRC channel at irc.codehaus.org:
irc://my_user@irc.codehaus.org/#cxf?nickname=my_user
Once logged in you simply receive all the messages sent over the channel, in the process() method of your Camel route, so I can configure my route something like this:
public void configure() {
  String url =
    "irc://my_user@irc.codehaus.org/#cxf?nickname=my_user";
  from(url).process(new Processor() {
    public void process(Exchange ex) throws Exception {
      String msg = ex.getIn().toString();
      System.out.println(msg);
    }
  }
}
The process() callback gets called whenever a message is sent over the IRC channel, so now we can do something with it!

This is where I started experimenting :) I wrote a little generic IRC client that can connect to any number of IRC channels and searches any messages coming in for a certain text string.
If there is a match, it opens a dialog on my screen that will stay there until I click it away, now I don’t miss it any more! I’m also using the new Java 6 tray icon functionality, to show a little popup balloon.




I put the code (under the Apache license) in SVN over here: http://coderthoughts.googlecode.com/svn/trunk/irc-alerter. No built executables yet, but you can simply check it out and run it with the following command:
mvn install camel:run -Dirc.alerter.channels=c:\path\to\channels.xml
since it’s pure java, it should work on mac, linux etc too :) I used Maven 2.0.9 and Java 6.

The format of the channels.xml file is as follows:

<channels>
 <match string="davidb"/> <!-- the text you want to match -->
 <password file="c:/davidb/etc/camel-irc.pwd"/>
 <channel name="cxf" user="my_user" nick="my_user" host="irc.codehaus.org"/>
 <channel name="somechannel" user="davidb" nick="davidb_camel" host="some_host" port="6667"/>
</channels>
So this one matches any message that contains 'davidb'. Obviously you’d put your match in there.
For each channel you specify the channel name, username, nickname, hostname and optionally the port. You probably want to use a different nick name than the one you use from your ordinary IRC client.
The passwords for the various servers are in a separate file, which you can put in an encrypted directory in your file system. These are properties files, your camel-irc.pwd file could look like this:

my_user@irc.codehaus.org=somepass
someuser@somehost=someotherpass

This has been a lot of fun to do. I’m still amazed how easy it was to create this app using Camel, and I’ll never miss an IRC ping ever again! (Well, I hope).

Source code - can be checked out from Subversion here: http://coderthoughts.googlecode.com/svn/trunk/irc-alerter

Questions from the RFC 119 webinar

There were quite a number of questions on the webinar and I didn't get to answer all of them, so I'm providing some answers here. For those of you who missed the webinar, you can still view the recorded version of the Distributed OSGi webinar here.
The source code of the GreeterDemo that I used during the demo session can be found in the Apache CXF demos SVN here: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/greeter for details on how to run the demo see the demo walkthrough page on the CXF Distributed OSGi website.

Below are the questions asked during the webinar, and an attempt to answer them. Feel free to send comments if I missed anything…

Q: Does it use RMI?
A: The wire-protocol and data-binding used are not specified by Distributed OSGi. They are up to the implementation. The Apache CXF implementation used in the demo uses SOAP/HTTP but other implementations could be written to use other wire protocols, such as RMI.

Q: Why don't you use JaxWS?
A: It's all about choice for the developer. JaxWS is a specific programming model for Web Services in Java. Distributed OSGi is for those who like the OSGi Services programming model, and would like to stay within that programming model to distribute OSGi services. It's not incompatible with other programming models, such as JaxWS, JaxRS, CORBA etc and can be used alongside them.

Q: Can I used Distributed OSGi services from Visual Basic?
A: Yes you can! Visual Basic programs can invoke on Web Services that are defined by WSDL. If you use the Apache CXF Distributed OSGi implementation it will generate a WSDL for your OSGi service which you can use from your Visual Basic client.

Q: Is there an implementation of Distributed OSGi available in Open Source?
A: Yes, the Apache CXF implementation, Eclipse ECF implementation and Apache Tuscany implementation are all open source and freely available to everyone.

Q: Can I use Distributed OSGi services from an Ajax browser client?
A: Yes, this is possible. I wrote an example of this on this blog a little while ago, see here: http://coderthoughts.blogspot.com/2009/02/distributed-osgi-powered-ajax-webapp.html.

Q: Can i use cxf with equinox, specifically pre 3.5?
A: The short answer is no, but you can use the current betas of Equinox 3.5 (builds since December). Any implementation of Distributed OSGi will need the Service Registry Hooks. Since this is a new API, it is only available in Equinox 3.5 and Apache Felix 1.4.1 or newer.

Q: Why is it based on webservices?
A: The design of Distributed OSGi is not based on Web Services and you may find implementations that don't use web services at all. The Apache CXF implementation is based on Web Services. Web services provide a standards-based approach for distributing software and allow integration with other (non-OSGi) systems. Although not required per se by an implementation of RFC 119, ‘Legacy integration' is one of the use cases that drove the design of Distributed OSGi, and by using Web Services this use-case can be addressed.

Q: You are using CXF for cross-jvm/osgi communication and mentioned that the rfc does not define the communication protocol. Does that mean that it is possible to implement a custom communication protocol?
A: Absolutely!

Q: How can I specify which protocol(s) to be used for some remote service?
A: Use intents mechanism for that. The intents mechanism allows you to declaratively specify constraints on the remote service. In short, you can put a property on the service that you want to remote that declares using what binding/protocol you the service should be exposed. So you could do something like: osgi.remote.requires.intents=JMS which means that the service should only be exposed over JMS. If the Distribution Software (DSW) can't satisfy the intent, it will not be able to expose the service. Although some values (for SOAP and JMS) have been defined in the RFC for this property, the actual possible values depend on the Distribution Software implementation used.

Q: How do I set the discovery service dynamically without the remote-services.xml?
A: Use an RFC 119 compliant Discovery implementation, these don't use the remote-services.xml files. We're currently working on one in the Apache CXF codebase that is based on Apache Zookeeper, but other implementations, such as the one based on SLP at Eclipse ECF are also available.

Q: Can I specify more than one protocol for one service? For example Web Services, CORBA, RMI, etc.
A: If you specify the protocol to use using the osgi.remote.requires.intents mechanism as described above, you can only specify one. E.g. specifying that the service has to satisfy both the RMI and CORBA intent at the same time is a contradiction, however you can register the same POJO twice with the OSGi Service registry, once for each protocol.

Q: Is Discovery service a separate OSGi container?
The RFC 119 Discovery architecture typically uses a centralized Discovery Server (potentially a replicated one, like we have with the CXF/Zookeeper implementation). The Discovery Service in the OSGi container provides a standardized interface to such a Discovery Server. The Distribution Software (DSW) interacts with the Discovery Service in the local OSGi service registry, which then communicates with the centralized Discovery Server.

Q: Service Registry hooks- limiting visibility of service-This is similar to access control the service- what are your views on this? Thanks
A: I forwarded this question to BJ Hargrave from IBM, the main author of the Service Registry Hooks RFC, he provided the following answer: "It is more about controlling visibility to services (but I guess some people may call that access [see http://blog.bjhargrave.com/2009/03/i-am-visible-but-am-i-accessible.html]). But the primary purpose is not to secure the system since without a Security Manager installed, there is no actual security. Its primary purpose is to control service visibility to influence the services a bundle sees so that it selects a desired service rather than another service. For example, the proxy for X rather than X. "

Q: so if they [OSGi Services] don't implement any signature, how are they identified within the registry?
A: OSGi Services don't need to implement a particular OSGi signature or interface, but they would typically implement an interface that relates to their purpose. In the presentation I mentioned a Purchase Service, which implements a foo.bar.PurchaseService interface. Normally services are looked up via their interface, but you can also look up services in the OSGi Service Registry purely on other properties that they might have.

Q: When you say 'Dynamically come and go', can you elaborate on that what does that really mean?
A: Example: A running OSGi system. A consumer in that system might be using a service that satisfies its lookup criteria. While the system is running another service that also satisfies these criteria could arrive (either because a new bundle that creates this service is installed or because an existing bundle registers a new service). Depending on the consumer's requirements, it could now potentially be using both services, the old and the new one. At a later time one of the services might go away at which point the consumer will only be using the one that's left. All (properly developed) OSGi service consumers are capable of dealing with dynamics like this.

Q: is the WSDL, automatically generated by some tool?
A: Yes, the Apache CXF DOSGi implementation does this dynamically at runtime, based to the Java interface of the service. You don't have to run a tool for this with CXF, although other implementations might require the running of a tool to achieve something like this.

Q: What does pojo mean?
A: Plain Old Java Object.

Q: I suppose that the parameters of the service exposed are of type 'primitive' like int, string, double, DateTime but cannot be of type by example SessionFactory ?
A: It depends on the DOSGi implementation used. If you want your code to be suitable for a number of DOSGi implementations it's best to restrict the use of data types to primitives, collections and arrays. The CXF DOSGi implementation also supports classes and interfaces as part of the signature as long as they in turn are defined in terms of the data types mentioned.

Q: Can you highlight the security capabilities that are provided by distributed OSGi?
and Q: Can we secure the communication between remote OSGI servers (and authenticate, authorise access to the services)?
A: Security is a Quality of Service that DOSGi implementations could provide. The intents mechanism can be used to only select remote services that are secure, e.g. by requesting the ‘confidentiality' intent, or by requesting a custom intent (such as AcmeSecurity) that your deployer defines with the appropriate security configuration. For more information see chapter 5.5.3 of RFC 119 (in http://www.osgi.org/download/osgi-4.2-early-draft3.pdf).

Q: How do local discovery services know about other discovery services?
A: The Discovery Service registered in the OSGi container communicates with a centralized Discovery Server. This Discovery Service needs to be configured with the connection details of the central discovery server.

Q: what implementations of distributed OSGi will be available in Equinox 3.5? Anything faster than webservices?
A: Equinox 3.5 does not have any Distributed OSGi implementations, but Eclipse 3.5 provides the ECF implementation. I guess I'm wondering what really the performance difference is that you are looking for. Have you measured the performance of CXF webservices versus something else? Could you provide us with some target figures? I know that CXF performs pretty well…

Q: Can you specify multiple instance endpoints for the same service?
A: On the server side one remote service typically maps to one remote endpoint (although exceptions are possible, especially in case the service implements multiple interfaces some implementations will give you one endpoint per interface). If you want more endpoints you can always register more OSGi services for the same POJO (although I'm not sure what the use-case for that would be).
On the consumer side, the consumer can be informed of multiple remote endpoints satisfying the OSGi Service Lookup criteria, in which case the consumer could get multiple client-side proxies to the remote services.

Q: Can you talk more about discovery with ZooKeeper and how that would work?
A: In the CXF project I started implementing the RFC 119 Discovery implementation using Apache ZooKeeper as the underlying Discovery mechanism. ZooKeeper is based on a replicated central server that holds a virtual file system. Consumers connecting to the ZooKeeper server will all view the same filesystem which holds the discovery information. The way this works in practice is that you have your cluster (1 or more) of zookeeper servers running. The CXF Discovery implementation bundle is installed in your OSGi containers. It will communicate with the ZooKeeper server for the discovery information. The net effect is that the remote service consumers will see the remote services without needing any extra configuration.

Q: Question : Is there other org.osgi.remote.type foreseen (e.g. POJO) ? What is the particularity of the type 'POJO' except its name ?
A: The POJO configuration type is specific to CXF. There is one configuration type specified in RFC 119: SCA, which is optional. Using SCA as a way to configure Distributed OSGi will give you portability on the configuration level. CXF does not implement the SCA configuration type yet…

Q: Question : Will it be possible using 'blueprint or spring DM' to integrate remote OSGI with Spring XML file ?
A: Yes - there is a demo that comes with the CXF DOSGi implementation that uses Spring-DM to create a remote OSGi service and on the consumer side it uses Spring-DM to create the consumer. Distributed OSGi is orthogonal to any OSGi component model and should therefore work fine with any of them. See here for the demo code: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/samples/spring_dm

Q: is it possible to load balance across distributed OSGi bundles?
A: Yes – if multiple remote instances exist the consumer could be informed about all of them: the consumer will get a service proxy for each remote instance. The consumer can use these to implement some load-balancing algorithm.

Q: Why does a copy of the interface and class have to live on the client?
A: Only the interface needs to be available to the service consumer. The implementation class does not. I think this is a very natural thing to do. If I write a consumer to a service, like a PurchaseService, I need to know the interface to that PurchaseService, how am I otherwise going to use this in a meaningful way? The interface is the contract between the consumer and the service provider. This is also the case for local OSGi Services. They can't use an OSGi service if they don't have access to the interface.

Q: It seems like you are cheating if you get a copy of the interface and class in your client code prior to starting the demo.
A: No I'm not cheating, only the interface is present in the consumer. When I run the consumer I can see that the value I'm entering in the consumer running in Felix is echoed in the Equinox server window. The implementation resides in Equinox.

Q: Does distributed OSGi support multiple instance of the same service and thus software based failover rof services across remote services?
A: Yes, see above.

Q: for properties is there a namespace type construct to prevent collisions? or is it inherent based on the scope of a service?
A: No there isn't really a namespace (although the Distributed OSGi properties use the osgi. prefix), but it is good practice to qualify your own properties with your own organization just like a Java package, e.g. com.acme.myproperty to avoid clashes.

Q: Are there standard properties of services, like names, that every OSGI service should define?
A: No. Its absolutely fine to register a service with no properties. OSGi will typically add a few properties, like service.id and objectClass but as a service provider you don't need to worry about these.

Q: How does a Discovery service, choose if the service is deployed both locally and remotely?
A: If a good match is available locally, DOSGi will not look for remote services. If you want to influence your match, forcing either a local or a remote match you can refine your lookup by adding extra criteria in the LDAP filter.

Q: Can the contract by dynamically found and transmitted remotely?
A: You could write a system to do this, as bundles can be installed at any time in a running OSGi system, but the Distributed OSGi standard doesn't define this.

Q: With multiple instances of the same service, would they have the same endpoint, or could they have different endpoints? How would a discovery pick which one?
A: The will have different endpoints. Everything being equal Discovery will simply inform you about all of them. If your consumer only needs to use one of them, which one the consumer will get is undefined. If the consumer has a preference for a particular one it needs to refine its filter to only match that one. You can also have the consumer receive proxies to all remote instances.

Q: Can you turn existing code (POJOs) into distributed OSGI services just using configuration (i.e. no java re-coding)?
A: Yes, OSGi Blueprint/Spring-DM provide a lot of support here. You configure them with an XML file that will instantiate your POJO and register it as an OSGi service.

Q: Are you familiar with the Paremus fabric, distributed OSGi runtime that uses SCA as the "wire-up" model? If so, does it use SCA in the way you described?
A: I'm not really familiar with this product, but I have seen demos of it. The SCA part of RFC 119 is really about defining qualities of service for the distributed services. It's not used for wiring services, the OSGi Service registry is used for that.

Q: Will both local and remote services be returned from a search?
A: There is no difference in the API to use to obtain local services or remote services. However, if appropriate local matches are available, the system won't look for remote services. If you want all local and remote matches you can do lookups with and without the osgi.remote property. The osgi.remote property is only set on proxies for remote services.

Q: Are there any drawbacks, using OSGI distributed services that you may know of?
A: None :) Seriously, yes remote computing is not the same as local computing. There are additional failure scenarios, additional latency, additional security impacts and so on. So if your system works perfect with all your services locally, keep it that way. However, if you want to distribute your OSGi services, then RFC 119 is a good way to do it!

Webinar and demo of Distributed OSGi (RFC 119) on Tue April 28

Just to let y'all know... I'll be doing a webinar and demo of Distributed Services OSGi on Tuesday April 28.
Distributed OSGi is a major new feature in the OSGi 4.2 specification. For more info, see http://fusesource.com/resources/video-archived-webinars

The demo will be done using the Apache CXF Reference Implementation.
Registration and attendance is free :)

A Distributed OSGi Powered AJAX WebApp

A pretty cool new way of using Distributed OSGi services is from a non-OSGi environment. In the previous post I used an OSGi AuctionService from an OSGi AuctionConsumer in a different VM.
This posting is about using that OSGi service from a non-OSGi consumer. From an AJAX-powered web application that's running in the browser!

I'm writing the webapp using the Google Web Toolkit (GWT). Both the webapp as well as my OSGi service are hosted in the same OSGi container. With AJAX you can create a rich client application running inside the browser. So no Servlet/JSP style server-side execution. The webapp runs on the client machine. GWT makes creating the AJAX app easy: you write them in plain old Java. No need to sidestep into Javascript as GWT turns the Java into a Javascript AJAX app and all (well, almost) of the browser specific madness is taken care of by GWT too.

As with any website, the end-users users simply open it in their browser. No client install, no browser plugins. Any modern browser supports this stuff natively. Remote communication, such as obtaining a list of available items, listing a new item, placing a bid, is done via SOAP/HTTP calls directly on my OSGi AuctionService (the one from the previous posting) which has been remoted using the Apache CXF-based Distributed OSGi Reference Implementation.

To do all this I'm adding a component called Pax Web Extender to my OSGi container, on top of the Distributed OSGi bundles. Pax Web Extender turns the OSGi container into a web container which means that I can deploy a .WAR file into it. It internally leverages the OSGi HTTP Service. (Spring DM Server can do this too BTW).

My OSGi container is configured like this:

Besides the CXF Distributed OSGi and Pax-Web-Extender Bundles I have my AuctionService and AuctionInterface bundles deployed (see previous posting for the details), plus the .WAR file that holds my AJAX webapp.

The reason I can invoke my OSGi Service is because it was made available remotely over a standards-based mechanism. Distributed OSGi itself doesn't prescribe how you do your remoting, it only says that you can use standards-based wire protocols and bindings if you wish and that this can make OSGi Services available to non-OSGi consumers.

The CXF implementation of Distributed OSGi turns an OSGi Service into a Web Service that is accessible via SOAP/HTTP. GWT can already natively make remote HTTP invocations. I didn't actually even care looking for a SOAP library for GWT. SOAP is just XML and GWT has some pretty good XML parsing functionality.
The last thing that I need to worry about (besides security, but that's a separate topic) is the browser's single-origin policy. But hey: I've deployed my webapp in the same runtime as my OSGi Service so as long as I make them available over the same port there's no problem! I achieved this by sharing the same OSGi HTTP Service by both the Distributed OSGi runtime and the Pax Web Extender. Thanks to the OSGi Services architecture!

So I'm taking the AuctionService of the previous posting, but first of all I want to expose it on a slightly simpler URL. The default URL of http://localhost:9000/org/coderthoughts/auction/AuctionService is just a bit too long, let's use /auction as the context instead. I also want to make it use the OSGi HTTP Service (which CXF doesn't do by default). I can do both in one shot by setting the osgi.remote.configuration.pojo.httpservice.context service property to the value /auction. The HTTP Service that comes with CXF runs by default on port 8080 (you can change this by setting the org.osgi.service.http.port system property) so my Web Service now runs on http://localhost:8080/auction. Change the activator that registers the service to the following:
public class Activator implements BundleActivator {
  private ServiceRegistration sr;

  public void start(BundleContext context) throws Exception {
    Dictionary props = new Hashtable();
    props.put("osgi.remote.interfaces", "*");
    props.put("osgi.remote.configuration.type", "pojo");
    props.put("osgi.remote.configuration.pojo.httpservice.context",
      "/auction");
    sr = context.registerService(AuctionService.class.getName(),
      new AuctionServiceImpl(), props);
  }

  public void stop(BundleContext context) throws Exception {
    sr.unregister();
  }
}
To double check that the service is operational on the new address, simply open http://localhost:8080/auction?wsdl and see if the WSDL appears again (just like in the previous posting).

I won't go into all the detail of writing the GWT app here. Look at the code in the SVN project for all of that.
I constructed my GWT application the usual way by creating a starting point with the projectCreator and applicationCreator scripts that come with GWT. This creates a nice Eclipse project skeleton for GWT from which I can start hacking my AJAX site in Plain Old Java :)

All you need to do in the GWT app to invoke on the Distributed OSGi service is:

• figure out where it's running, since the Web Service is running on the same host and port as where the AJAX app is served from on the /auction context, I can construct the Web Service URL as follows:
  webServiceURL = "http://" + Window.Location.getHost() + "/auction/";
  
• create a SOAP message, just a bit of XML really.
  
• send it via a HTTP POST request to the WebService using a com.google.gwt.http.client.RequestBuilder that comes with GWT.
  
• register a callback object to process the SOAP (XML) response.
  

All of this happens in the AuctionSite class which is the webapp's main class. There are a few more classes such as data classes and a few dialogs. Check out the project from SVN to see them all...

To prepare my webapp (.WAR file) for running with the Pax Web Extender I'm turning it into an OSGi bundle by adding the following META-INF/MANIFEST.MF file:
Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-SymbolicName: AuctionSite
Bundle-Version: 1.0.0
Webapp-Context: auctionsite
The Webapp-Context is important as this tells the Pax Web Extender where to register the webapp.

I also have to add a WEB-INF/web.xml file to the .WAR as this is the signature file that the Pax-Web Extender needs in order to spot a bundle as a webapp. However, I don't really need to put anything in it. My webapp consists only of static files (no servlets etc.) as far as the webserver is concerned, so my web.xml is simply:
<web-app>
  <display-name>AuctionSite</display-name>
</web-app>

GWT comes with scrips that you run to turn your Java into AJAX.
I wrote a tiny little ant script that calls the GWT script and then creates a .WAR file from my GWT project in Eclipse.

Now lets boot is all up! I'm using Equinox 3.5M5 with the Multi Bundle Distribution of CXF-DOSGi.

To install the Multi Bundle Distribution of CXF-DOSGi, download it from here. Then unzip it in your Felix/Equinox installation dir.
Take the contents of the felix.config.properties.append or equinox.config.ini.append and append it to the configuration file of your OSGi container.
This will automatically load all the DOSGi bundles when starting up the OSGi container.

• Start Equinox with the DOSGi Multi Bundle configuration to make it load all the DOSGi bundles at startup.
  
• Then install and start the PAX Web Extender from here and your AuctionIntf and AuctionImpl bundles.
  
• Finally install and start the AuctionSite.war file, just like any other bundle in OSGi.

You should now have the following bundles running in Equinox:
osgi> ss
Framework is launched.
id State Bundle
0 ACTIVE org.eclipse.osgi_3.5.0.v20090127-1630
1 ACTIVE org.eclipse.osgi.services_3.2.0.v20081205-1800
2 ACTIVE org.apache.geronimo.specs.geronimo-annotation_1.0_spec_1.1.1
3 ACTIVE org.apache.geronimo.specs.geronimo-activation_1.1_spec_1.0.2
4 ACTIVE org.apache.geronimo.specs.geronimo-javamail_1.4_spec_1.2.0
5 ACTIVE org.apache.geronimo.specs.geronimo-ws-metadata_2.0_spec_1.1.2
6 ACTIVE com.springsource.org.apache.commons.logging_1.1.1
7 ACTIVE com.springsource.org.jdom_1.0.0
8 ACTIVE org.springframework.bundle.spring.core_2.5.5
9 ACTIVE org.springframework.bundle.spring.beans_2.5.5
10 ACTIVE org.springframework.bundle.spring.context_2.5.5
11 ACTIVE com.springsource.org.aopalliance_1.0.0
12 ACTIVE org.springframework.bundle.spring.aop_2.5.5
13 ACTIVE org.springframework.bundle.osgi.io_1.1.2
14 ACTIVE org.springframework.bundle.osgi.core_1.1.2
15 ACTIVE org.springframework.bundle.osgi.extender_1.1.2
16 ACTIVE org.ops4j.pax.web.service_0.5.1
17 ACTIVE org.apache.servicemix.specs.locator-1.1.1
18 ACTIVE org.apache.servicemix.bundles.jaxb-impl_2.1.6.1
19 ACTIVE org.apache.servicemix.bundles.wsdl4j_1.6.1.1
20 ACTIVE org.apache.servicemix.bundles.xmlsec_1.3.0.1
21 ACTIVE org.apache.servicemix.bundles.wss4j_1.5.4.1
22 ACTIVE org.apache.servicemix.bundles.xmlschema_1.4.2.1
23 ACTIVE org.apache.servicemix.bundles.asm_2.2.3.1
24 ACTIVE org.apache.servicemix.bundles.xmlresolver_1.2.0.1
25 ACTIVE org.apache.servicemix.bundles.neethi_2.0.4.1
26 ACTIVE org.apache.servicemix.bundles.woodstox_3.2.7.1
27 ACTIVE org.apache.cxf.cxf-bundle-minimal_2.2.0.SNAPSHOT
28 ACTIVE org.apache.servicemix.specs.saaj-api-1.3_1.1.1
29 ACTIVE org.apache.servicemix.specs.stax-api-1.0_1.1.1
30 ACTIVE org.apache.servicemix.specs.jaxb-api-2.1_1.1.1
31 ACTIVE org.apache.servicemix.specs.jaxws-api-2.1_1.1.1
32 ACTIVE cxf-dosgi-ri-discovery-local_1.0.0.SNAPSHOT
33 ACTIVE cxf-dosgi-ri-dsw-cxf_1.0.0.SNAPSHOT
34 ACTIVE org.ops4j.pax.web.extender.war_0.4.0
35 ACTIVE AuctionIntf_1.0.0
36 ACTIVE AuctionImpl_1.0.0
37 ACTIVE AuctionSite_1.0.0


Open the webapp at http://localhost:8080/auctionsite/AuctionSite.html. This kicks off the AJAX app in your browser. When you hit the reload button it makes a SOAP/HTTP invocation on the server to list all the items and get their details. The SOAP messages it sends map to the AuctionService.getItemIDs() and AuctionService.getItems() APIs. (At the bottom of the screen you see the SOAP message as it comes back form the server)


Let's list a new item...


After filling in the Javascript popup, hit OK.
It now sends the following SOAP message from the browser to the server:
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
 <soap:Body>
  <ns1:listNewItem xmlns:ns1="http://auction.coderthoughts.org/">
   <ns1:arg0>Joke</ns1:arg0>
   <ns1:arg1>1000</ns1:arg1>
   <ns1:arg2>2009-02-09T15:48:22-00:00</ns1:arg2>
  </ns1:listNewItem>
 </soap:Body>
</soap:Envelope>

Once the remote invocation has completed, the list of items is updated again, which is the same as hitting the reload button.


I can still access the web service from my remote OSGi client that I developed in the previous post. The same remote OSGi service can obviously be shared among many consumers. When I run the AuctionClient bundle I'm also seeing my newly listed item.
Before I can do that I need to update the remote-services.xml file in the consumer bundle with the new location of the Web Service (when using Discovery this is not needed BTW):
<service-descriptions xmlns="http://www.osgi.org/xmlns/sd/v1.0.0">
 <service-description>
  <provide interface="org.coderthoughts.auction.AuctionService"/>
  <property name="osgi.remote.interfaces">*</property>
  <property name="osgi.remote.configuration.type">pojo</property>
  <property name="osgi.remote.configuration.pojo.address">
   http://localhost:9000/auction
  </property>
 </service-description>
</service-descriptions>

Now run the AuctionClient bundle:
Items available for auction:
 1:                 Doll $  7.99 Sat Dec 05 08:00:00 GMT 2009
10:                 Joke $ 10.00 Mon Feb 09 15:48:12 GMT 2009
 2: Empty sheet of paper $ 12.00 Tue May 11 21:10:00 IST 2010
 3:            Bike shed $126.75 Tue Sep 15 08:12:00 IST 2009
You can get all the code in this posting under the Apache license from the google code project. It's all in SVN at http://coderthoughts.googlecode.com/svn/trunk/gwt_auctionsite and http://coderthoughts.googlecode.com/svn/trunk/osgi_auctionsite .
Since it's all Eclipse projects, the easiest way to get it by doing the SVN checkout of the projects directly in Eclipse. It will give you a workspace that looks like this: