Friday, April 19, 2013

Using OSGi Subsystems to deploy your Applications

One of the major new specs in the OSGi R5 Enterprise release is the Subsystem specification. While this spec itself is quite large and covers a wide number of angles and use-cases, I find the simplest way to explain Subsystems really as something like Application deployment for OSGi where an application is comprised of a number of bundles.

OSGi always encourages modular development and during development this is indeed great, because you can focus on the module at hand and have clear visibility of the impact of any changes that you make. However, once you want to deploy your application of 150 bundles this may become a little bit complicated - you certainly don't want to hand the person performing the deployment of 150 different files to deploy. You want to put them together in some way or form. This has caused many projects to come up with their own solution around this. Karaf and Eclipse have features, Eclipse Virgo has plans and Apache Aries has applications. The OSGi Subsystem specification now provides a standard for combining a number of bundles in a single deployable (an .esa file) which means that an .esa file can be deployed in any compliant Subsystem implementation.
I am really happy that the good people at Apache Aries have recently released version 1.0 of the Aries Subsystem implementation, so you can now use this without having to build an implementation yourself.

I'm going to look at how to create and use Subsystems later in this post, but first let's get an OSGi framework set up with Subsystems. The Aries implementation with its dependencies consists of 15 bundles. In my example below I'm using Equinox as the OSGi framework, but it should obviously work just as well on any other OSGi R5 compliant framework.

Setting up the Subsystem infrastructure
I'm going to be using the following:
  • Equinox 3.8.2 (which comes with Eclipse 4.2.2 or 3.8.2)
  • The gogo shell (which comes with Equinox and also with Felix)
  • The Aries Subsystem 1.0 implementation
  • dependencies of the above...
Let's start getting Equinox up and running with the shell, add the following bundles to your Equinox runtime. These all come shipped with Eclipse so if you're working in Eclipse you can simply select them in the 'OSGi Framework' launch configuration:

org.eclipse.osgi_3.8.2.v20130124-134944
org.eclipse.osgi.services_3.3.100.v20120522-1822
org.apache.felix.gogo.runtime_0.8.0.v201108120515
org.apache.felix.gogo.shell_0.8.0.v201110170705
org.eclipse.equinox.console_1.0.0.v20120522-1841

Now add the following bundles to install the Aries Subsystem implementation (the links below can be used to download them from Maven Central):

I'll leave it to the reader to find a convenient way to install all these (see also the comments section with a note about how to install this on a framework other than Equinox; you need an extra bundle). You can do it with a script, using a repository, etc... There is also a subsystem-bundle artifact that may help. In any case, this alone validates one of the key points why Subsystems were designed in the first place. If you have an application that is formed by a number of bundles you'd really want a nice and convenient way to deploy them. Once we have a subsystem implementation in place we can do this and start deploying large applications that consist of many bundles by simply deploying a single Subsystem archive file.

With the above bundles started the Subsystem Service is registered and ready to deploy subsystems:
osgi> services (objectClass=*Subsystem)
  {org.osgi.service.subsystem.Subsystem}=
  {subsystem.id=0, subsystem.state=ACTIVE, subsystem.version=1.0.0,
   subsystem.type=osgi.subsystem.application,
   subsystem.symbolicName=org.osgi.service.subsystem.root, ...}
  "Registered by bundle:" org.apache.aries.subsystem.core_1.0.0 [6]

There is one issue though - we have no tool yet that utilizes the subsystem service so we can interact with it. It would be really nice if we could add a command to the OSGi console to do this. Using the extensible Gogo command shell this is childs play. So let's add a few subsystem commands.

Add some Subsystem commands to Gogo
Gogo is becoming the de-facto standard for shell commands in an OSGi framework. It's used by Equinox, Felix, Karaf and other OSGi distributions these days. Gogo is extensible and adding a few new commands to it is as simple as registering an OSGi service.

I created a bundle with only a single class, the activator which provides the following commands:
  subsystem:list
  subsystem:install <url>
  subsystem:uninstall <id>
  subsystem:start <id>
  subsystem:stop <id>

import java.io.IOException;
import java.net.URL;
import java.util.*;
import org.osgi.framework.*;
import org.osgi.service.subsystem.Subsystem;

public class Activator implements BundleActivator {
  private BundleContext bundleContext;

  public void start(BundleContext context) throws Exception {
    bundleContext = context;
    Dictionary<String, Object> props = new Hashtable<String, Object>();
    props.put("osgi.command.function",
      new String [] {"install", "uninstall", "start", "stop", "list"});
    props.put("osgi.command.scope", "subsystem");
    context.registerService(getClass().getName(), this, props);
  }

  public void install(String url) throws IOException {
    Subsystem rootSubsystem = getSubsystem(0);
    Subsystem s = rootSubsystem.install(url, new URL(url).openStream());
    System.out.println("Subsystem successfully installed: " +
      s.getSymbolicName() + "; id: " + s.getSubsystemId());
  }

  public void uninstall(long id) {
    getSubsystem(id).uninstall();
  }

  public void start(long id) {
    getSubsystem(id).start();
  }

  public void stop(long id) {
    getSubsystem(id).stop();
  }

  public void list() throws InvalidSyntaxException {
    for (ServiceReference<Subsystem> ref :
         bundleContext.getServiceReferences(Subsystem.class, null)) {
      Subsystem s = bundleContext.getService(ref);
      if (s != null) {
        System.out.printf("%d\t%s\t%s\n", s.getSubsystemId(), s.getState(), s.getSymbolicName());
      }
    }
  }

  private Subsystem getSubsystem(long id) {
    try {
      for (ServiceReference<Subsystem> ref :
           bundleContext.getServiceReferences(Subsystem.class, "(subsystem.id=" + id + ")")) {
        Subsystem svc = bundleContext.getService(ref);
        if (svc != null)
          return svc;
      }
    } catch (InvalidSyntaxException e) {
      throw new RuntimeException(e);
    }
    throw new RuntimeException("Unable to find subsystem " + id);
  }

  public void stop(BundleContext context) throws Exception {}
}

I shared a bundle that contains this command, you can get it from here: http://coderthoughts.googlecode.com/files/subsystem-gogo-command-1.0.0.jar

Once the above bundles are installed and everything is started I have the following bundles in my framework:
0  ACTIVE org.eclipse.osgi_3.8.2.v20130124-134944
1  ACTIVE org.eclipse.osgi.services_3.3.100.v20120522-1822
2  ACTIVE org.apache.felix.gogo.runtime_0.8.0.v201108120515
3  ACTIVE org.apache.felix.gogo.shell_0.8.0.v201110170705
4  ACTIVE org.eclipse.equinox.console_1.0.0.v20120522-1841
5  ACTIVE org.apache.aries.subsystem.api_1.0.0
6  ACTIVE org.apache.aries.subsystem.core_1.0.0
7  ACTIVE org.apache.aries.subsystem.obr_1.0.0
8  ACTIVE org.apache.aries.application.api_1.0.0
9  ACTIVE org.apache.aries.application.modeller_1.0.0
10 ACTIVE org.apache.aries.application.utils_1.0.0
11 ACTIVE org.apache.aries.blueprint_1.1.0
12 ACTIVE org.apache.aries.proxy_1.0.1
13 ACTIVE org.apache.aries.util_1.1.0
14 ACTIVE org.apache.felix.bundlerepository_1.6.6
15 ACTIVE org.apache.felix.resolver_1.0.0
16 ACTIVE org.eclipse.equinox.coordinator_1.1.0.v20120522-1841
17 ACTIVE org.eclipse.equinox.region_1.1.0.v20120522-1841
18 ACTIVE slf4j.api_1.7.5, Fragments=19
19 RESOLV slf4j.simple_1.7.5, Master=18
20 ACTIVE org.osgi.service.subsystem.region.context.0_1.0.0
21 ACTIVE subsystem-gogo-command_1.0.0

Note that bundle 20 is a synthesized bundle created automatically by the subsystem implementation. We can safely ignore it.

Now I can start doing something. Let's list the available subsystems using our new command from the subsystem-gogo-command listing/bundle:
osgi> subsystem:list
0 ACTIVE org.osgi.service.subsystem.root 

At this point there is only a single subsystem: the root one.

Working with Subsystems
Let's create some sample subsystems to look at what you can do.

I'm going to create two basic subsystems that should allow us to play with it. The subsystem specification defines a number of different subsystem types. In this post I will be looking at the feature subsystem type which deploys all the bundles from the subsystem in a shared space. As if you were just installing all the bundles in a plain framework. (note: other subsystem types provide isolation for the subsystems.)
Subsystem archives typically use the .esa file extension. Both my example subsystems contain 3 bundles. The subsystem1.esa file contains Bundle A, Bundle B and a bundle called Shared Bundle. subsystem2.esa contains Bundle C, Bundle D and also the same Shared Bundle. Both subsystems package the Shared Bundle as they both have a dependency on it. So in order to get a fully working system for either subsystem I need that Shared Bundle. However since these are feature subsystems, where everything is shared I only need the Shared Bundle deployed once.

Creating a subsystem file is pretty easy. The .esa file is really just a zip file that contains the embedded bundles in the root. Additionally it contains a subsystem manifest. I created mine simply using the jar command, but you can also use tools such as the esa-maven-plugin. Here's what you'll find inside:

$ jar tvf subsystem1.esa
    99 Fri Apr 19 08:34:08 IST 2013 OSGI-INF/SUBSYSTEM.MF
  1181 Fri Apr 19 08:33:06 IST 2013 BundleA_1.0.0.jar
  1058 Fri Apr 19 08:33:06 IST 2013 BundleB_1.0.0.jar
   906 Fri Apr 19 08:33:06 IST 2013 SharedBundle_1.0.0.jar


As you can see the zip file contains the relevant bundles in the root plus a subsystem manifest. Here's what the SUBSYSTEM.MF file in subsystem1.esa looks like:
  Subsystem-SymbolicName: subsystem1
  Subsystem-Version: 1.0.0
  Subsystem-Type: osgi.subsystem.feature
It looks a bit like a Bundle Manifest. Most of the information in there is optional...

The subsystem2.esa file is very similar. You can download the sample subsystem files from here: subsystem1.esa and subsystem2.esa.

Let's deploy a subsystem:
  osgi> subsystem:install http://coderthoughts.googlecode.com/files/subsystem1.esa
  Subsystem successfully installed: subsystem1; id: 1

If we list the bundles the three bundles that were in subsystem1.esa are now added.
  22 INSTALLED SharedBundle_1.0.0
  23 INSTALLED BundleA_1.0.0
  24 INSTALLED BundleB_1.0.0

Now let's start the subsystem:
  osgi> subsystem:start 1
...
  22 ACTIVE SharedBundle_1.0.0
  23 ACTIVE BundleA_1.0.0
  24 ACTIVE BundleB_1.0.0
This is pretty handy: starting the subsystem will start all of the bundles that it contains!

Let's add the other subsystem:
  osgi> subsystem:install http://coderthoughts.googlecode.com/files/subsystem2.esa
  Subsystem successfully installed: subsystem2; id: 2
  osgi> subsystem:start 2

Now both subsystems are active:
  22 ACTIVE SharedBundle_1.0.0
  23 ACTIVE BundleA_1.0.0
  24 ACTIVE BundleB_1.0.0
  25 ACTIVE BundleC_1.0.0
  26 ACTIVE BundleD_1.0.0
And we can see that the SharedBundle was only deployed once, because it could be shared across subsystems.

You can also query the subsystems known in the system:
  osgi> subsystem:list
  0 ACTIVE org.osgi.service.subsystem.root
  1 ACTIVE subsystem1
  2 ACTIVE subsystem2

Another interesting aspect is how stopping and un-installation works. Especially in relation to the SharedBundle. I'll leave is as an exercise for the reader but you can see that the Subsystems implementation keeps track of the bundle sharing. If you only stop subsystem1, the SharedBundle will remain ACTIVE. Only when both subsystems that use the bundle are stopped the bundle will move to the RESOLVED state. Uninstalling works similarly. When you uninstall subsystem1, BundleA and BundleB will be uninstalled, but the SharedBundle won't as it is still being used by subsystem2. Only when subsystem2 is uninstalled as well all of the bundles associated with subsystem1 and subsystem2 are uninstalled.

There is a lot more to talk about in relation to subsystems. For example, subsystems don't have to actually embed their dependencies. They can also download them from an OSGi Repository service. In that case your .esa file can be limited to only contain a SUBSYSTEM.MF which lists what your root application bundles should be. The subsystem implementation can also use the Repository Service to automatically find transitive dependencies.

In my little example, the subsystems only contain 3 bundles each, but using .esa files can become really handy when your application becomes large and contains tens or hundreds of bundles. You can even nest them, so subsystems can contain other subsystems - becoming building blocks of higher-level subsystems.

OSGi Subsystems should make the distribution and deployment of larger OSGi applications much easier. The .esa file provides a portable format which allows you to hand your users a single artifact to deploy, regardless of how many bundles your application is made up of.

For more information about OSGi Subsystems see chapter 134 of the OSGi R5 Enterprise specification: http://www.osgi.org/Download/Release5
Posted by at 4 comments:
Labels: , , , , , , ,

Monday, March 11, 2013

HTML5 video fun

I started playing with HTML5 and found that it contains some very cool stuff. Take for example the native ability to play video with the <video> tag.

But your browser can do more natively these days - it can access your computer's webcam too! In fact, connecting to your webcam and streaming it out to a video tag in a html page can be done in only a few lines of HTML with some embedded JavaScript!

Take a look at the following small html page: 
<html>
  <head>
    <title>HTML5 Video with no plugins!</title>
    <script>
      function setup() {
        navigator.myGetMedia = ( navigator.getUserMedia || 
          navigator.webkitGetUserMedia ||
          navigator.mozGetUserMedia ||
          navigator.msGetUserMedia);

        navigator.myGetMedia({video: true}, connect, error); 
      }

      function connect(stream) {
        var video = document.getElementById("my_video");
        video.src = window.URL ? window.URL.createObjectURL(stream) : stream;
        video.play();
      }

      function error(e) { console.log(e); }

      addEventListener("load", setup);
    </script>
  </head>
  <body>
    <header><h1>HTML5 Video with no plugins!</h1></header>
    <video id="my_video"/>
  </body>
</html>

When the page is loaded (on the load event) the setup() function is executed. This connects to your webcam through the navigator.getUserMedia() API. Currently there are a few different browser variants with various prefixes of this. When the WebRTC spec is finalized they will most likely all be unified to getUserMedia (without any prefix), which will take another 5 lines out of the above code. Once connected to the webcam the script obtains a URL to this connection and sets it as the source for the video tag. That's it!
Audio works similarly too so you can also create a combined audio/video stream.

Running the webpage shows what your webcam sees in on the browser page:

And, it works on my Android phone too!

You can try it yourself or you can launch the page from here.

This opens some really interesting possibilities. For example the people from html5videoguide have used this to create a video conferencing system:
http://html5videoguide.net/presentations/LCA_2013_webrtc
powered by your browser and a few lines of serverside JavaScript. You can use these APIs to record video or audio. Or turn your browser into a camera app! Combine with the new canvas APIs and you can use it to do photo editing too! All right there in your browser, all without the need for additional plugins.

The code above still contains some variations to cater for slight differences that can be found in browsers, that should all be resolves as soon as the webrtc spec goes final.

Posted by at No comments:
Labels: , , , , ,

Thursday, January 3, 2013

A mobile device photo organizer (using OSGi)

When people think about OSGi applications they often think of complex server-side applications or embedded programs that are running as part of a set-top-box of some sort. Or they think of Eclipse-based RCP applications which are also based on OSGi.

When I started writing a little program to organize photos from the various mobile devices in my home it was unlike many of those applications. It was small, it had a (Swing-based) GUI and it wasn't running on the server side. Still using OSGi helped me enormously with the development. In this blog post I will discuss how.

The application
I'm sure I'm not the only person where the house is full of devices that can take photos. I like to store these pictures centrally on a NAS drive where they are neatly organized. Most devices come with a photo management solution of some sort, but they are often only working with a certain device and not with others and they mostly store the photos in different ways on disk. On top of that I want to keep the photos in a directory structure that looks like this: year/date-taken where I don't mean date-copied. All in all I couldn't find a solution that did this the way I wanted so I started a little tool for this over the holiday season.

While most of the actual work is done by headless processing logic, I needed a little GUI to kick it off. While there are tons of options available, I chose to create a simple Java Swing GUI, made to look nice with one of the awesome look and feels from the JTattoo guys.

While the project itself (which is available here) has the ASL2 license, I am using a variety of libraries to extract information from photo files, video files and for example to access my Android device, because I want to be able to copy photos directly from that.

I need modularization
Most of the libraries that I'm using have a license compatible with my ASL2 license, but I really wanted to access my Android phone directly. Android phones, as well as other mobile devices cannot easily be mounted as a directory on the file system. They need to be accessed using MTP. I found a library that allowed me to access my phone from Java: jusbpmp. It worked fine for the most part but there was one issue: it's GPL-licensed. I don't want to get into which open source license is better, but the viral effects of GPL are well known and I already decided that my application was to be ASL2-licensed. I didn't want to relicense my whole project because of the fact that one dependency has this other license.

If I could single out the functionality that uses this library and only license that piece GPL (as needed) and then plug it into the rest of my application that would limit the scope and the amount of code that is required to have the GPL license. Ideally I want that component to be loosely coupled so that it can be downloaded separately and plugged into the main application.

OSGi Bundles to the rescue. And in particular OSGi Services! OSGi Services use a contribution model where service implementations contribute them to the Service Registry. Consumers find them there. It all works by defining the Service APIs in a separate module that they all communicate through.

The MTP library allows me to access the files on my Android phone. So I started by defining an API module to create an Iterable that can give me entries that serve the photo information from anything any type of source. The PhotoIterable interface can represent a file directory-based photo store, but also one that comes from an MTP device or other device. It looks somewhat like this:

public interface PhotoIterable extends Iterable<PhotoIterable.Entry> {
  /**
   * Get a human readable identification of the location.
   * @return The location string.
   */
  String getLocationString();

  /**
   * This class represents a photo object that can be read.
   */
  public interface Entry {
    /**
     * Returns the file name to use for the photo.
     * @return The file name without path information, for example IMG_01429.JPG
     */
    String getName();

    /**
     * @return The stream to read the photo bytes from.
     */
    InputStream getInputStream();
  }
}

Now I need a way to obtain such a PhotoIterable. Typically the user wants to select a location on the device to copy the photos from (instead of getting all the image files on the device) and to do this  I defined the PhotoSource interface. This is what I will register in the OSGi Service Registry. For every supported type of source a corresponding service will be registered. The core bundle ships with one for loading photos from a file system directory, and the phototools.mtp bundle registers one to handle MTP devices.

public interface PhotoSource {
  /**
   * The label for this source, for example 'File System Directory' or 'Android'.
   * @return The label to use.
   */
  String getLabel();

  /**
   * Calling this method should open a selection window where the user can select where the
   * photos are to be copied or downloaded from.
   * @return A PhotoIterable to obtain photos from the selected location.
   */
  PhotoIterable getPhotoIterable();
}

At this stage I have 3 bundles: the API bundle, the core implementation bundle and the MTP implementation bundle.


This separation is nice because:
  • The API bundle is small. Anyone who wants to write support for another mobile device has only a very small number of interfaces to look at. No distracting implementation code to get in the way.
  • I can find all the PhotoSource implementations by looking them up in the OSGi Service Registry, for example with bundleContext.getServiceReferences(PhotoSource.class, null))
  • I can contribute support for additional devices without changing the rest of the code. Just add the bundle that contains the support (and registers the PhotoSource service) and it will appear (even the GUI will react to this).
  • I didn't have to write my own plugin mechanism. OSGi Bundles and Services provide that to me.
  • I could isolate the functionality that depends on a GPL library in a separate bundle. This means that the main application is still ASL2 licensed. The GPL-based bundle is optional and can be provided separately if I want to create a pure ASL2-based product.
The PhotoSource that the MTP provide bundle contributes is visible as a widget in the main screen (the Mobile Device via USB Radio Button). When I click Select I can see the custom MTP selection GUI in action:

More OSGi Bundles and Services to keep things clean
While the phototools.core bundle can deal with extracting metadata from some of the Photo formats (thanks to Drew's metadata-extractor) I also want my application to handle movie files. I found another nice library that could handle mp4 files for me: Sebastian's mp4parser. Although mp4parser is ASL2 licensed I didn't want to include it the core bundle because it was getting fairly heavy on embedded libraries and also there were still a number of photo/video formats unsupported. Getting the core bundle to support them all didn't seem the right thing to do. Allowing separate bundles to contribute a format handler did! So I defined an additional Service API:

public interface PhotoMetadataProvider {
  /**
   * Get metadata for a photo or video file
   * @param f The file to process.
   * @return The metadata found.
   */
  Metadata getMetaData(File f);

  public interface Metadata {
    /**
     * Obtain the date the photo was taken (not necessarily the file date).
     * @return The date taken.
     */
    Date getDateTaken();

    /**
     * Obtain a small preview file for the photo or movie. If available,
     * the preview file will always be a JPEG file.
     * @return The preview file.
     */
    File getPreviewFile();
  }
}

I can find an appropriate Metadata Provider for my photos by looking up one from the OSGi Service Registry that is registered for the relevant extension. This is done by using OSGi Service Registration properties. Each Photo Metadata Provider registers the formats it can handle with the format property. Then I can look one up by querying on the extension of the file I want to process e.g:
  ServiceReferences[] refs = bundleContext.getServiceReferences(
    PhotoMetadataProvider.class, "(format=.jpeg)");
When I find a service that can handle my format, get it to process the file:
  PhotoMetadataProvider p = bundleContext.getService(sref);
  Metadata metadata = p.getMetaData(myPhotoOrVideoFile);
  Date dateTaken = metadata.getDateTaken();

The mp4 handling code is now nicely separated in its own bundle. In addition, if I ever want to add support for other formats (.AVI for example) I can do this by simply adding another bundle, which drastically reduces the scope of my changes and also reduces the amount of code that I may have to look at.

Conclusion? Well my little project is not finished yet, it's still work in progress. But OSGi really helped me by providing a nice plug-in architecture and its modularity almost forced me to write nice interface-based components which will be easier to maintain in the long run. Because the bundles have a clear scope they tend to be quite small and when making changes the amount of code you have to look at as a developer is much smaller than if this was part of a monolithic application. This is great because I generally only sporadically have time to go back to my hobby projects and having less code to refresh my brain is good :) Oh, and the fact that I'm using OSGi is completely hidden to the end user. It's really just an architectural choice under the covers.

Happy new year, everyone.

Posted by at No comments:
Labels: , , , , , , ,

Tuesday, August 28, 2012

Dynamic Provisioning OSGi Cloud Ecosystems

In my recent posts I looked at building a cloud ecosystem using OSGi and adding extra control to remote service invocations in such ecosystems.

In this post I'm starting to look at the provisioning of such ecosystems. OSGi frameworks are dynamic which means they can dynamically be assigned a function and when the need arises they can be dynamically repurposed. This repurposing is interesting in particular because it saves you from sending the cloud vm image over the network again. And those images can be big. You can reuse the image that you started with, changing its function and role over time.

In my previous post I was using pre-configured VM images, each representing a different deployment in the ecosystem. In this post I will start out with a number of identical images. In addition, I will deploy one other special image: the provisioner. However, since that one is also running in an OSGi framework, even that one can be repurposed or moved if the need arises.

You'll see that, as in the previous posts, none of the OSGi bundles are statically configured to know where the remote services they use are located. They are all part of the dynamic OSGi Cloud Ecosystem which binds all services together via the OSGi Remote Services compliant Discovery mechanism.
I wrote a very simple demo provisioner which keeps an eye on what frameworks are up, provisions them with OSGi bundles, and if one goes down it redeploys services to an alternative.

Altogether, the setup is as follows:
Provisioning an ecosystem of initially identical OSGi frameworks
The ecosystem contains 5 OSGi frameworks, 4 of which are identical: osgi1, osgi2, osgi3 and web. The only difference wrt to the last one is that it has a well-known hostname because it's hosting the web component that will be accessed from the outside world. The 5th framework has the same baseline as the first 4, but it acts as the provisioner/management agent, so this one has some extra bundles deployed that can provision OSGi bundles in the other frameworks in the ecosystem.

To be able to work with such a dynamic ecosystem of OSGi frameworks, the provisioner needs to monitor the frameworks available in it. Since these frameworks are represented as OSGi services this monitoring can simply be done through OSGi service mechanics, in my case I'm using a simple ServiceTracker, but you could do the same with Blueprint or Declarative Services or any other OSGi services-based framework.

Filter filter = bundleContext.createFilter(
  "(&(objectClass=org.coderthoughts...api.OSGiFramework)(service.imported=*))");
frameworkTracker = new ServiceTracker(bundleContext, filter, null) {
  @Override
  public Object addingService(ServiceReference reference) {
    handleTopologyChange();
    return super.addingService(reference);
  }

  @Override
  public void removedService(ServiceReference reference, Object service) {
    handleTopologyChange();
    super.removedService(reference, service);
  }
};
My ServiceTracker listens for services with OSGiFramework as objectClass which have the service.imported property set. This means that it listens to remote frameworks only. Whenever a framework appears or disappears the handleTopologyChange() will be called.

For this posting I wrote a really simple demo provisioner that has the following rules that are evaluated when the topology changes (frameworks arrive and disappear in the ecosystem):
  • Deploy the web bundles to a framework with an ip address starting with 'web-'.
  • Deploy one instance of the service provider bundles to the framework with the largest amount of memory available. 
  • Ensure that if the vm hosting the service provider bundles disappears for some reason, they get redeployed elsewhere. Note that by virtue of using the OSGi Remote Services Discovery functionality the client will automatically find the moved service.
Obviously these aren't real production rules, but they give you an idea what you can do in a management/provisioning agent:
  • Decide what to deploy based on image characteristics, I used image name for the web component, but you can also use other metadata such as image type (Red Hat OpenShift/Amazon/Azure/etc), location (e.g. country where the VM is hosted) or other custom metadata.
  • Dynamically select a target image based on a runtime characteristic, in my case I'm using memory available.
  • Keep an eye on the topology. When images disappear or new ones arrive the provisioner gets notified - simply through OSGi Service dynamics - and changes can be made accordingly. For example a redeploy action can be taken when a framework disappears.
  • No hardcoded references to where the target services live. The service consumer use ordinary OSGi service APIs to look up and invoke the services in the cloud ecosystem. I'm not using a component system like Blueprint or DS here but you can also use those, of course. 
the demo provisioner code is here: DemoProvisioner.java

As before, I'm using Red Hat's free OpenShift cloud instances, so anyone can play with this stuff, just create a free account and go.

My basic provisioner knows the sets of bundles that need to be deployed in order to get a resolvable system. In the future I'll write some more about the OSGi Repository and Resolver services which can make this process more declarative and highly flexible.

However, the first problem I had was: what will I use to do the remote deployment to my OSGi frameworks? There are a number of OSGi specs that deal with remote management, most notably JMX. But remote JMX deployment doesn't really work that well in my cloud environment as it requires me to open an extra network port, which I don't have. Probably the best solution would be based on the REST API that is being worked on in the EEG (RFC 182) and hook that in with the OSGi HTTP service. For the moment, I decided to create a simple deployment service API that works well with OSGi Remote Services:

public interface RemoteDeployer {
    long getBundleID(String location);
    String getSymbolicName(long id);
    long installBundle(String location, byte [] base64Data);
    long [] listBundleIDs();
    void startBundle(long id);
    void stopBundle(long id);
    void uninstallBundle(long id);
}

 This gives me very basic management control over remote frameworks and allows me to find out what they have installed. It's a simple API that's designed to work well with Remote Services. Note that the installBundle() method takes the actual bundle content as base-64 encoded bytes. This saves me from having to host the bundles as downloadable URLs from my provisioner framework, and makes it really easy to send over the bytes.

 I'm registering my RemoteDeployer service in each framework in the ecosystem using the Remote Services based cloud ecosystem service distribution I described in my previous post. This way the provisioner can access the service in the remote frameworks and deploy bundles to each.

That's pretty much it! I've simplified the web servlet a little from the previous post to only show a single TestService invocation, let's try it out...

Try it out!
First thing I'm doing is start the Discovery Server in the ecosystem. This is done exactly as in my earlier post: Cloud Ecosystems with OSGi.

Having the discovery server running, I begin by launching my provisioner framework.  After that I'll be adding the empty OSGi frameworks one-by-one to see how it all behaves.
Finally I'll kill a VM to see redeployment at work.

The common OSGi Framework bits
The provisioner framework is created by first adding the baseline infrastructure to a new OpenShift DIY clone:
$ git clone ssh://blahblahblah@provisioner-coderthoughts.rhcloud.com/~/git/provisioner.git/
$ cd provisioner
$ git fetch https://github.com/bosschaert/osgi-cloud-infra.git
$ git merge -Xtheirs FETCH_HEAD
Don't forget to set up the Discovery SSH tunnel, which is also described in Cloud Ecosystems with OSGi.

The specific Provisioner bits
Specific to the provisioner vm are the provisioner bundles. The demo provisioner I wrote can be built and deployed from the osgi-cloud-prov-src github project:
$ git clone https://github.com/bosschaert/osgi-cloud-prov-src.git
$ cd osgi-cloud-prov-src
$ mvn install 
$ ./copy-to-provisioner.sh .../provisioner
finally add the changed configuration and provisioner bundle to git in the provisioner clone, then commit and push:
$ cd .../provisioner
$ git add osgi
$ git commit -m "Provisioner VM"
$ git push

The provisioner image will be uploaded to the cloud and if you have a console visible (which you'll get by default by doing a git push on OpenShift) you will eventually see the OSGi Framework coming up.

Adding the identical OSGi Frameworks
I'm launching 3 identical cloud images all containing an unprovisioned OSGi framework. As they become active my demo provisioner will exercise the RemoteDeployer API to provision them with bundles.

First add one called web-something. Mine is called web-coderthoughts. The demo provisioner looks for one with the 'web-' prefix and will deploy the servlet there, this will allow us to use a fixed URL in the browser to see what's going on. This is exactly the same as what is descibed above in the common OSGi Framework bits:
$ git clone ssh://blahblahblah@web-coderthoughts.rhcloud.com/~/git/web.git/
$ cd web
$ git fetch https://github.com/bosschaert/osgi-cloud-infra.git
$ git merge -Xtheirs FETCH_HEAD
And don't forget to set up the SSH tunnel.

Once the web framework is started, the provisioner starts reacting, if you have a console, you'll see messages appear such as:
remote: *** Found web framework
remote: *** Web framework bundles deployed

Great! Let's look at the servlet page:
So far so good. We are seeing 2 frameworks in the ecosystem: the web framework and the provisioner. No invocations where made on the TestService because it isn't deployed yet.

So let's deploy our two other frameworks. The steps are the same as for the web framework, except the vm names are osgi1 and osgi2 and reload the web page:
click to see full size
The two new frameworks are now visible in the list. Our TestService, which is invoked by the servlet/web vm, is apparently deployed to osgi1. You can see that by comparing the UUIDs.

 Reacting to failure
Let's kill the framework hosting the TestService and see what happens :)
OpenShift has the following command for this, if you're using a different cloud provider I'm sure they provide a means to stop a Cloud VM image.
$ rhc app stop -a osgi1

Give the system a few moments to react, then refresh the web page:
click to see full size
The osgi1 framework has disappeared from the list and the TestService invocation reports that it's now running in the osgi2 framework. The provisioner has reacted to the disappearance of osgi1 and reprovisioned the TestService bundles to osgi2. The service client (the servlet) was automatically reconfigured to use osgi2 throught the Remote Services Discovery mechanism. No static configuration or binding file, simply OSGi service dynamics at work across VMs in the cloud.

Repurposing
Obviously the provisioner in this posting was very simplistic and only really an example of how you could write one. Additionally, things will become even more interesting when you start using an OSGi Repository Service to provide the content to the provisioner. That will bring a highly declarative model to the provisioner and allows you do create a much more generic management agent. I'll write about that in a future post.

Another element not worked out in this post is the re-purposing of OSGi frameworks. However it's easy to see that with the primitives available this can be done. Initially the frameworks in the cloud ecosystem are identical, and although provisioner will fill them with different content, the content can be removed (the bundles can be uninstalled) and the framework can be given a different role by the provisioner when the need arises.

Note that all the code in the various github projects used in this post has been tagged 0.3.
Posted by at 1 comment:
Labels: , , , , , ,

Thursday, July 12, 2012

Controlling OSGi Services in Cloud Ecosystems

In my previous post I looked at the basics of setting up an OSGi Cloud Ecosystem. With that as a basis I'm going to look at an issue that was brought up during the second OSGi Cloud Workshop, held this year at EclipseCon in Washington, where this list of ideas was produced. One of topics on this list is about new ways in which services can fail in a cloud scenario. For example, a service invocation might fail because the service consumer's credit card has expired. Or the number of invocations allocated to a certain client has been used up, etc.

In this post I'll be looking at how to address this in the OSGi Cloud Ecosystem architecture that I've been writing about.

First lets look at the scenario where a cloud service grants a maximum number of invocations to a certain consumer.

An invocation policy for a remote OSGi service


From the consumer side, I'm simply invoking my remote demo TestService as usual:
  TestService ts = ... from the Service Registry ...
  String result = ts.doit();

On the Service Provider side we need to add additional control. As this control (normally) only applies to remote consumers from another framework in the cloud I've extended the OSGi Remote Services implementation to provide extra control points. I did this on a branch of the Apache CXF-DOSGi project, which is the Remote Services implementation I'm using. I came up with the idea of a RemoteServiceFactory. Conceptually a bit similar to a normal OSGi ServiceFactory. Where the normal OSGi ServiceFactory can provide a new service instance for each client bundle, a RemoteServiceFactory can provide a new service (and exert other control) for each remote client. I'm currently distinguishing each client by IP address, not completely sure whether this covers all the cases, maybe some sort of a security context would also make sense in here.
My initial version of the RemoteServiceFactory interface looks like this:
  public interface RemoteServiceFactory {
    public Object getService(String clientIP, ServiceReference reference);
    public void ungetService(String clientIP, ServiceReference reference, Object service);
  }

Now we can put the additional controls in place. I can limit service invocations to 3 per IP address:
public class TestServiceRSF implements RemoteServiceFactory, ... {
  ConcurrentMap<String, AtomicInteger> invocationCount = new ConcurrentHashMap<>();

  @Override
  public Object getService(String clientIP, ServiceReference reference) {
    AtomicInteger count = getCount(clientIP);
    int amount = count.incrementAndGet();
    if (amount > 3)
      throw new InvocationsExhaustedException("Maximum invocations reached for: " + clientIP);

    return new TestServiceImpl(); // or reuse an existing one
  } 

  private AtomicInteger getCount(String ipAddr) {
    AtomicInteger newCnt = new AtomicInteger();
    AtomicInteger oldCnt = invocationCount.putIfAbsent(ipAddr, newCnt);
    return oldCnt == null ? newCnt : oldCnt;
  }

A RemoteServiceFactory allows me to add other policies as well, for example it can prevent concurrent invocations from a single consumer, see here for an example, select provided functionality based on the client or even charge the customer per invocation.

To register the RemoteServiceFactory in the system I'm currently adding it as a service registration property:
public class Activator implements BundleActivator {
  // ...
  public void start(BundleContext context) throws Exception {
    TestService ts = new TestServiceImpl();
    Dictionary<String, Object> tsProps = new Hashtable<>();
    tsProps.put("service.exported.interfaces", "*");
    tsProps.put("service.exported.configs", "org.coderthoughts.configtype.cloud");
    RemoteServiceFactory tsControl = new TestServiceRSF(context);
    tsProps.put("org.coderthoughts.remote.service.factory", tsControl);
    tsReg = context.registerService(TestService.class.getName(), ts, tsProps);
   ...

More control to the client

Being able to control the service as described above is nice, but seeing an exception in the client when trying to invoke the service isn't great. It would be good if the client could prevent such a situation by asking the framework whether a service it's hosting will accept invocations. For this I added service variables to the OSGiFramework interface. You can ask frameworks in the ecosystem for metadata regarding the services it provides:
  public interface OSGiFramework {
    String getServiceVariable(long serviceID, String name);
  }

I implemented this idea using the OSGi Monitor Admin service (chapter 119 of the Compendium Specification)

Service Variables are accessed via OSGi Monitor Admin

From my client servlet I'm checking the service status before calling the service:
  fw.getServiceVariable(serviceID, OSGiFramework.SV_STATUS);
(note given a service reference, you can find out whether it's remote by checking for the service.imported property, you can find the hosting framework instance by matching their endpoint.framework.uuid properties and you can get the service ID of the service in that framework by looking up the endpoint.service.id of the remote service - see here for an example).
So I can ask the OSGiFramework whether I can invoke the service, it can respond with various return codes, as a starting point for possible return values, I took the following list, largely inspired by the HTTP response codes:
  SERVICE_STATUS_OK // HTTP 200
  SERVICE_STATUS_UNAUTHORIZED // HTTP 401
  SERVICE_STATUS_PAYMENT_NEEDED // HTTP 402
  SERVICE_STATUS_FORBIDDEN // HTTP 403
  SERVICE_STATUS_NOT_FOUND // HTTP 404
  SERVICE_STATUS_QUOTA_EXCEEDED // HTTP 413
  SERVICE_STATUS_SERVER_ERROR // HTTP 500
  SERVICE_STATUS_TEMPORARY_UNAVAILABLE // HTTP 503

It might also be worth adding some response codes around saying that things are still ok, but not for long, I was thinking of these responses:
  SERVICE_STATUS_OK_QUOTA_ALMOST_EXCEEDED
  SERVICE_STATUS_OK_PAYMENT_INFO_NEEDED_SOON
These need more thought but I think they can provide an interesting mechanism in preventing outages.

Under the hood I'm using the OSGi Monitor Admin Specification. I took an implementation from KnowHowLab (thanks guys!). It gives a nice Whiteboard pattern-based approach to providing metadata via a Monitorable service. As the RemoteServiceFactory is the place where I'm implementing the policies for my TestService, it provides me a natural place to publish the metadata too.

When the client calls OSGiFramework.getServiceVariable(id, SV_STATUS) the OSGiFramework service implementation in turn finds the matching Monitorable, which provides the status information. The Monitorable for my TestService is implemented by its RemoteServiceFactory:
public class TestServiceRSF implements ..., Monitorable {
  ConcurrentMap<String, AtomicInteger> invocationCount = new ConcurrentHashMap<>(); 

  // ...

  @Override
  public StatusVariable getStatusVariable(String var) throws IllegalArgumentException {
    String ip = getIPAddress(var);
    AtomicInteger count = invocationCount.get(ip);

    String status;
    if (count == null) {
      status = OSGiFramework.SERVICE_STATUS_NOT_FOUND;
    } else {
      if (count.get() < 3)
        status = OSGiFramework.SERVICE_STATUS_OK;
      else
        status = OSGiFramework.SERVICE_STATUS_QUOTA_EXCEEDED;
    }
    return new StatusVariable(var, StatusVariable.CM_SI, status);
  }

  private String getIPAddress(String var) {
    // The client IP is the suffix of the variable name
    if (!var.startsWith(OSGiFramework.SERVICE_STATUS_PREFIX))
      throw new IllegalArgumentException("Not a valid status variable: " + var);


    String ip = var.substring(OSGiFramework.SERVICE_STATUS_PREFIX.length());
    return ip;
  }

Using MonitorAdmin thought the OSGi ServiceRegistry gives me a nice, loosely coupled mechanism to provide remote service metadata. It fits nicely with the RemoteServiceFactory approach but can also be implemented otherwise.

When I use my web servlet again I can see it all in action. It invokes the test service 5 times:



In the Web UI you can see the 2 OSGi Frameworks in this cloud ecosystem. The local one (that also hosts the Web UI) and another one in a different cloud VM.
The Servlet hosting the webpage invokes the TestService 5 times. In this case there is only a remote instance available. After 3 invocations it reports that the invocation quota have been used up.

The Web UI servlet also invokes another remote service (the LongRunningService) twice concurrently. You can see the policy that prevents concurrent invocation in action where only one invocation succeds (it waits for a while then returns 42) and the other reports an error and does not invoke the actual service.

The demo simply displays the service status and the return value from the remote service, but given this information I can do some interesting things.
  • I can make the OSGi service consumer aware and avoid services that are not OK to invoke. Standard OSGi service mechanics allow me to switch to another service without much ado. 
  • I can go even further and add a mechanism in the OSGi framework that automatically hides services if they are not OK to invoke. I wrote a blog post a while ago on how that can be done: Altering OSGi Service Lookups with Service Registry Hooks - the standard OSGi Service Registry Hooks allow you to do things like this.

Do it yourself!

I have updated osgi-cloud-infra (and its source) with the changes to the OSGiFramework service. The bundles in this project are also updated to contain the Monitor Admin service implementation and the changes to CXF-DOSGi that I made on my branch to support the RemoteServiceFactory.

Additionally I updated the demo bundles in osgi-cloud-disco-demo to implement the RemoteServiceFactory, add Service Variables and update the webui as above.

There are no changes to the discovery server component.

Instructions to run it all are identical to what was described in the previous blog post - just follow the steps from 'try it out' in that post and you'll see it in action.
Note that it's possible that this code will be updated in the future. I've tagged this version as 0.2 in git.
Posted by at No comments:
Labels: , , , , ,
Subscribe to: Posts (Atom)