Monday, April 28, 2008

An OSGi Bundle dependency visualizer in GMF

When working with larger OSGi-base systems you sometimes wonder: why are all these bundles there? What other bundles does my bundle depend on at runtime? Is there any duplication going on?
For a project that I was working on I started manually creating Bundle Dependency diagrams. This got me thinking: it should be really easy to create a tool that does this for me using EMF (Eclipse Modeling Framework) and GMF (the Graphical Modeling Framework from Eclipse).

As always in Model Driven Development, you start with a model. In my case a model of how OSGi Bundles relate to each other.
Having the model, you just fill it with data, then use some GMF goodness to display is graphically.

Sounds easy? It was easy! With well under 300 lines of manually written code (and a whole bunch of EMF/GMF generated stuff) the end result looks like this:


In this picture I ran the visualizer in the ServiceMix 4 Kernel to see what was in it.

Main Components

There are two components in this application:
  1. A bundle that you drop into an OSGi container, when you start the bundle it will walk all the other bundles in the OSGi container and figure out what their dependencies are. The result is stored in an EMF model which is stored a file called save.bundles at the end of the process.
  2. An Eclipse GMF application that can create a visual diagram from the save.bundles file and allows you to tweak the resulting picture any way you like.
How it's done
The model

1. Start with a simple model of a Framework (representing the OSGi Container) which contains bundles. Each bundle in turn could be importing stuff from other bundles:

Here are the model details:


After generating the genmodel file from my model file (I only modified the base package in the genmodel package properties) I generated the model code so I could start coding with it. (See here for an earlier post that outlines how to generate code from an EMF model).

The Introspector Bundle
To create a model instance with the bundle dependency data, I've written an self-contained OSGi bundle that I can drop on any OSGi runtime (well, at this point I only tried Equinox and Felix). When the bundle is started it walks all the active bundles in the container and figures out what other bundles each bundle depends on by using the bundle's classloader to resolve imported classes. (Since Package Imports are resolved dynamically this can only be done at runtime in an active OSGi container). The collected info is stored in an instance of my model. I stayed away from using any implementation-specific admin API's, instead I'm just using the bundle's classloader to resolve the packages listed in the Import-Package header to and figure out what bundle the OSGi runtime has linked them to.
Additionally, Require-Bundle dependencies are stored in the model too.

Finally, after all the bundles have been processed, I use the built-in EMF XMI serialization to save model instance to a file (currently always called save.bundles in your temp directory).

That's the hard bit done. You can see the code here in the BundleIntrospector and BundleIntrospectorTools classes which are invoked from my simple Activator.

The Visualizer
Believe it or not, the easy bit is in creating the graphical visualizer. GMF does all the hard work for me. I'm by no means a GMF connaisseur, so I used the GMF Cheat Sheet that comes with GMF in Eclipse. I ran all the wizards from the Cheat Sheet, and used all the defaults to create the .gmfgraph, .gmftool, .gmfmap and .gmfgen files. For some reason, there was one setting that I had to manually change: the Label Mapping, but the cheat sheet told me what to do.
Finally I hit 'generate diagram code' on the .gmfgen file and I'm done.

My Eclipse Workspace looks like this.


Running it

Running the introspector with Equinox
This is very easy from within Eclipse. To try this out, create a new OSGi Framework launch configuration with the introspector bundle and, to give it something to look at, I also include the org.mortbay.jetty bundle and it's dependencies:


Make sure that the introspector bundle is not automatically started. When I launch Equinox I have the following bundles loaded:


osgi> ss
Framework is launched.

id   State    Bundle
0    ACTIVE   org.eclipse.osgi_3.3.1.R33x_v20070828
1    ACTIVE   org.mortbay.jetty_5.1.11.v200706111724
2    ACTIVE   org.apache.commons.logging_1.0.4.v200706111724
3    ACTIVE   javax.servlet_2.4.0.v200706111738
4    RESOLVED osgi_bundles.introspector.bundle_0.0.3.qualifier

When I start the introspector bundle it starts analyzing the other bundles:

osgi> start 4
Initializing data from bundle: org.eclipse.osgi
Initializing data from bundle: org.mortbay.jetty
Initializing data from bundle: org.apache.commons.logging
...
Saving... Done dependency model saved to: c:\temp\save.bundles
Finished introspecting, stopping bundle.


Running the Visualizer
Now that I have a data file, I can run the visualizer. Just right-click on one of your projects in Eclipse and select Run As Eclipse Application.
In the Eclipse Application that gets started, create a new Project and copy the save.bundles file in there.

At this point the save.bundles file is purely an EMF serialization into XMI, but GMF can initialize a picture from it, right-click 'initialize bundles_diagram file', and you'll get one:

Tadaa, a nice visualization of the bundles and their dependencies. The Jetty bundle uses the commons logging and servlet bundles (the system bundle is ommitted).
Now you can use the GMF features to rearrange your diagram, choose colours, line smoothness etc to make your diagram the way you want it (which is quite useful once you have a lot of bundles). You can even export the picture as a PNG, JPG or SVG file.

Running the introspector in ServiceMix 4 Kernel
The ServiceMix 4 Kernel is an OSGi based system built using Apache Felix. If you want to run the introspector in the ServiceMix 4 Kernel you can follow these steps:
1. Get hold of a SMX 4 Kernel distribution. I created mine by checking out the code from http://svn.apache.org/repos/asf/servicemix/smx4/kernel/trunk and running mvn install to build it. You will find the distribution in assembly/target subdirectory.
2. Run bin/servicemix


____ _ __ __ _
/ ___| ___ _ ____ _(_) ___ ___| \/ (_)_ __
\___ \ / _ \ '__\ \ / / |/ __/ _ \ |\/| | \ \/ /
___) | __/ | \ V /| | (_| __/ | | | |> <
|____/ \___|_| \_/ |_|\___\___|_| |_|_/_/\_\

ServiceMix (1.0-m3-SNAPSHOT)

Type 'help' for more information.
------------------------------------------------
servicemix>

Move to the osgi shell and install the introspector, I'm doing this straight from the Google Code project downloads section:

servicemix> osgi
servicemix osgi> install http://coderthoughts.googlecode.com/files/osgi_bundles.
introspector.bundle_0.0.3.200804202025.jar
Bundle ID: 28
servicemix osgi>

When I start bundle 28 it will do the introspection and save this to the save.bundles file. When loaded in the visualizer, you'll get a picture like the first one in this posting.

Getting your environment set up
I used Eclipse 3.3.1.1 with the EMF/GMF plugins loaded. See this posting for details on setting it up.

Getting the code
All the code used here is available under the Apache License from the Google Code project via SVN. The Eclipse Projects are accessible from here: http://coderthoughts.googlecode.com/svn/trunk/osgi_visualizer/

Getting the distribution
If you just want to use or play with the visualizer, you can download the introspector bundle and visualizer plugins from the the download area: http://code.google.com/p/coderthoughts/downloads/list

Improvements are welcomed!
The project is by no means finished. I'm still in the process of learning more about GMF and any suggestions or improvements are appreciated.

9 comments:

Steilpass said...

did you see: http://www.eclipse.org/pde/incubator/dependency-visualization/

davidb said...

Hi Steilpass, yeah and Prosyst's mConsole has similar features as well. The point here is really how easy it is to do something like this in EMF & GMF...

Ed Merks said...

David, indeed it's a good point and a point well taken!

Chris Aniszczyk (zx) said...

Nice, I love to see plug-ins used in interesting ways :)

lgrammel said...

This is a great idea and similar to what Ian Bull does with model driven visualization - he actually wrote the PDE plugin for dependency visualization. Good luck with your project!

davidb said...

See for an alternative visualisation mechanism (IBM AlphaWorks's Many Eyes) to create a very cool (and more scalable) visualisation. Slide 12 and further on http://people.apache.org/~lresende/presentations/felix%20goes%20to%20tuscany.pdf.

Junior said...

Hi, its a very interesting example. Could it be possible to synchronize the the GMF editor to simultaneously write 2 different textual files? And preserving the two files generated by default by the GMF editor?

And do you know if xtext has an authomatic way to generate the code to process an xmi model generated by the gmf editor? Like an alternative for not sincrozing both editors.

Junior said...

I'm sorry for the previous I was writing in two blogs simultaneosuly.

I couldn't configurate the plugins to work. Which eclipse version and intalled bundles are you using.

I am missing depencies. Using Eclipse 3.6 Modeling SR2. I found two projects with maven but they both end in failure.

davidb said...

Hi Junior, I wrote this blog back in 2008 so things may have changed a little. At the time I used Eclipse 3.3 with EMF/GMF that came with that. If there are changes to the instructions to make them apply to a newer version of Eclipse I'd love to hear :)