This is a brief note on new the filter support in Scalamodules 2.0. I humbly suggested this functionality to Heiko, and it’s coming in the new version. (It is also on github now, of course.)

What are OSGi filters? Well, they are a subset of LDAP filters, which you most likely aren’t familiar with, unless you’ve dabbled in various black arts of enterprise. They are expressions that can be matched against service registrations in OSGi. Registrations are usually decorated with simple service properties on the familiar name/value form. For instance:

foo=bar
zot=5

Very familiar, I’m sure. Incidentally, this is represented in OSGi with an old-school Java Dictionary. Don’t worry, non-masochistic Scalamodules users won’t ever see that. Here’s a filter that matches it:

(&(foo=bar)(zot<10))

You get the general idea. There is negation and or as well as the and, and the usual set of operators. See the OSGi spec for full details.

OK, so how is the filter represented in OSGi? Answer: It’s a String.

If you silently go “eew” inside now, read on. Again, we feel that Scalamodules users shouldn’t have to see it.

Sure, strings are fine, but this is still something that needs to be a well-formed expression. So why don’t we add programmatic support for it, to ensure both the well-formedness and possibly other constraints? It’s sort of a reverse parser – instead of getting the AST out of an expression, we build the AST and produce the expression as a String when it is to be used.

Here are some ways to build the filter in Scalamodules. First, we import the Filter object:

import org.scalamodules.core.Filter._

The below is one of the nicer constructs; it uses Scala-style API pimping to provide implicit conversion to a builder object, which supports the === and its peers:

("foo" === "bar") && ("zot" <== 10)

Direct use of the Filter methods instead would be:

and(set("foo", "bar"), lt("zot", 10))

or:

set("foo", "bar") and lt("zot", 10)
set("foo", "bar") && lt("zot", 10)

Note that arguments don’t have to be strings, they can be anything that turns into a reasonable string with toString, including primitives.

The set method, which becomes a (foo=x)-style equality filter, actually takes varargs. Passing no arguments indicates a present filter, which is a convention for asking if the variable is set:

Filter.set("foo")

This turns into a (foo=*) filter, which is interpreted by OSGi as any value. While the following is a multi-value filter:

Filter.set("foo", 5, 6)

This turns into (foo=[5,6]), which requires that foo is one of the two values.

So why all this brouhaha? The motivation for all this is simply to make filter construction a more reliable and verifiable process than simply putting together strings, which (at least when I do it) is error-prone and sometimes code-intensive. If it gets bad enough, I usually end up with small string-construction frameworks to do this for me, anyway. And we want to fail fast – detecting and reporting errors as early as possible. Let’s just sayt that not all OSGi implementations produce the best error messages on malformed filters.

But wait! What if you have a filter string? Suppose you got it as an input from somewhere. You don’t want to have to parse it and pick it apart, only to reconstruct it programmatically! The Filter.literal method handles the case where you get a valid filter from somewhere else, and want to use it verbatim. So if you have a string you’re happy with, you can turn it into a filter and we’ll take your word for its validity. However, it might fail later, when and if it is passed to OSGi and you’ve been less than diligent.

This post is an overview and not a reference manual, we (will) have scaladocs for that. But the above should give and outline of the general idea – I hope.

Here’s a tip if you need to access HBase from an OSGi environment. I tried packaging them into two separate Hadoop and HBase bundles, but I ran into an unexpected, and quite unseemly, issue. To make a short story sweet (or so): You will need to package them together in a single bundle, and write a manifest that exports the HBase and the Hadoop packages.

The issue, shockingly, is that HBase takes the liberty of declaring a class in a Hadoop package! Specifically, the org.apache.hadoop.ipc.HBaseClient class extends Hadoop’s Client class, defined in that same package. This is the kind of necessary evil that pops up from time to time between two projects, especially in early days – after all, they are in 0.17.1 and 0.2.0, respectively. It does break an important OSGi rule: A package can only be imported from a single bundle, it cannot contain classes contributed from multiple bundles. Which means that other bundles can only ever depend on one of them, never both.

I haven’t thought of any import/export acrobatics that could resolve this, and I’m not sure if anyone should. The HBase bundle needs to see that package exported from the Hadoop bundle, or it can’t extend its Client class. Given that, the HBase bundle then needs to export that same package. I can’t imagine any way that would work, except by accidents in the implementation. Even if it somehow is an edge case within the spec, it certainly isn’t something I would like to spend too much think-time concerning myself about, let alone maintain. The quick fix is to package them together. Do that, and your stuff works.

Feel free to ask for the manifest, if you’re interested. I didn’t have it at hand while writing this post.

Just to capture your buzz interest, this is DSL-related! Read on …

OSGi filters are under-modelled, because they’re strings. Strings are great per se, they’re portable and very expressive – after all, source code is strings. So, strings – great, love’em. Write’em every day myself, for money, donchaknow.

But OSGi filters represent a shift of responsibility. With specifications, there’s always a specifier, a vendor, and a user. Specifications can come with reference implementations and library support, to help vendors and users. (OSGi is pretty well off in this regard, compared to e.g. Corba in its day.) But the responsibility for filter representation has been passed through the chain and squarely onto the user. The specifier and the vendor have told the user: Just give us back a string, and we’ll look at it and possibly give you an InvalidFilterException back. And yes, it’s checked, because you should be prepared being a gal (or guy) who can’t append strings together correctly.

Fortunately for the users, they’re programmers. They will say: OK then, I’ll have a computer make sure this string is right every time, OK? Just watch me catch that checked exception of yours, and rethrow it as a ThisNeverHappensError.

(I may have forgotten the precise name of the actual exception, InvalidFilterException. That’s because I rarely ever think about it anymore.)

So we should make filter classes, build instances with code, and hang them in trees. Make them print themselves to a well-formed string. Yes, this is the good old shape-draw-yourself thing that we old-hand object modellers thought was a silly example of polymorphism.

I have a small OSGi library brewing, name of vanadis. I will be referring to that from now on. The Filter interface there looks like this, in a condensed version – look up the code for the whole shebang:

interface Filter {
    Filter and(Filter... exprs);
    Filter or(Filter... exprs);
    Filter not();
    String toFilterString();
}

Then there is the Filters factory class, a classic static factory with static methods all over it. But if it had an interface, it would be:

public interface Filters {
    Filter eq(String attribute, Object... values);
    Filter approx(String attribute, Object... values);
    Filter gt(String attribute, Object... values);
    Filter lt(String attribute, Object... values);
    Filter present(String attribute);
    Filter substring(String attribute, String substring);
    Filter objectClasses(Class... objectClasses);
}

So, to create a simple filter that works well for a WerewolfKiller service:

    Filters.objectClasses(SilverBullet.class).and(Filters.isTrue("fullMoon"));

Of course, the Filter interface can host methods that move us closer to DSL land, allowing us to express things with less typing:

   objectClasses(SilverBullet.class).and("fullMoon");

The HelplessFarmer service might want to query for an experienced, yet reasonably stable WerewolfKiller service:

   objectClasses(WerewolfKillerService.class).andGt("killCount", 5).unless("psychotic");

And so on. You might have noticed I pass class objects to the factory methods, not class names. Again, this is an attempt to keep myself DRY – calling getName() on class objects should also be done only once. In this case, the Filter object does it at construction time, so the class object never “hangs” on the tree. However, if making a large, distributed application, you might need some filter creation happening in an infrastructural capacity, which is likely to be in a context where the classes aren’t visible. In that case, using strings to create filters might be the only way to go. But my experience tells me that you will, more often than not, be constructing filters in the context where the classes are known.

Building filter classes and putting them in trees is really quite simple. In my last project, I made a support class that allowed us to build strings correctly and incrementally: Each filter object would hold a legal filter string, and appending stuff like e.g. this.and(that) would simply create a new filter object, with “&(” at the head, then the strings from this and that, ended by “)” at the tail. This might be perfect for your purposes, and very simple.

In vanadis, I have the whole structure represented as a tree. Not only does this save on string garbage, the resulting structure is more pick-apart-able, one can do automated analysis etc. This can be golden if you find that filter matching is becoming a bottleneck.

These are all just ideas, and a work in progress that I hope to make use of in my current job. I’m happy to take comments and feedback if you have any, especially on making the base classes more generally usable.

Look out, unbelievers. I’m on a crusade against the OSGi API’s these days. 

Admittedly, it’s not that the OSGi API’s are bad. They are quite complete and completely workable. The impression you get, though, is that they could do with a brush-up and a make-over, give them that coat of wax. Make them more… modern. They’re old-school. But then, OSGi comes out of the embedded space, and wants to be as portable as possible, so this makes sense. But couldn’t we use, say, the Collections API? 

Though I do like lots of the new-fangled Java features, I would argue against OSGi adopting them recklessly. Why? Because that’s not their job – they want to grow a mature, consistent platform. (Also, they’ve got IBM backing them – a virtual guarantee for glacial-at-best adoption of all the new and cool stuff.) It is a core technology, it can and should get away with being low-level and old-fashioned.

In short, OSGi targets a wide variety of execution environments, and tries to leave no (remotely relevant) configuration behind. It is you, the adopter, that gets to make the decisions that narrow this scope, and building good libraries on your platform of preference. Which, I suppose, would often involve Java 5 or 6.

If you want to build right on top of OSGi – maybe for some reason you can’t use any of those new application frameworks surfacing these days – you should invest some time in building a general, high-level, modernized API that fits your needs. Convenience methods and so on. Just so you can write more code dealing with your application, and less code dealing with putting values into java.util.Dictionary instances, or composing LDAP-ish query strings. Obviously, having everyone mess around this can produce lots of messy code, so that library support can be very useful.

I hope to get some time to outline some handy API’s for this purpose. First up is programmatic support for filter expressions – stay tuned.

I’m the kind of developer that abhors Eclipse and adores IntelliJ. Eclipse gives me the impression – or rather it climbs up a mountain and shouts down at me – that noone is really interested in me getting my job done. Life is hard, so stop whining. Did you think writing code was going to be easy? Gee, wuss much? You realise what people got by with before? Well, do you, punk? Of course, this is subjective.

IntelliJ, by contrast, is near intoxicating at times. It is like a drinking buddy that understands me a little better with each sip. Again, this is my subjective feeling and rest assured the metaphor shall remain unexplored.

Eclipse, on the other hand, runs on OSGi, in the shape and form of its Equinox plug-in engine. I’ve come to value OSGi over the last couple of years, so I’m all for it. I think OSGi contributes something of value. But OSGi, like Eclipse, has an ugly interface – its programmatic API. OSGi is a bit like IBM itself – old and rickety, but solidly built.

The central BundleContext interface has stuff like:

ServiceRegistration registerService(String[] clazzes, Object service, Dictionary properties)

This has been accurately described as old-skool. A Dictionary? Eep. Fortunately, as programmers we can use it and not have to like it. We wrap stuff, like this:

<T> Registration<T> register(T service, ServiceProperties<T> serviceProperties)

Right? Right? Generics. Higher-level abstractions of central concepts. A new layer between OSGi and your logic! Sure, call it vanity, but I like building abstractions and modelling stuff. Some have emphasized the importance of making forward progress over building beautiful models, but in this case I believe these concerns go hand-in-hand.

A forward-progress-maker, not to say a forward-progress-maker fundamentalist (not that I know of any) would ask: What’s the point of wrapping it? After all, invoking the BundleContext works, you wuss.

The model-builder (and his/her fundamentalist counterpart), who incidentally are speaking from experience, then point out that we are dealing with a very low-level API that is very flexible. It also leads to usage of some less-common types, like Dictionary. The higher-level wrapper may serve you well in heading off some undesirable developments:

You might start to see a lot of methods doing Map/Dictionary conversions. You will see casting of Object references with varying degrees of type checking. In general, people will start to establish various usage patterns around the OSGi API. As I said, OSGi is low-level and flexible, and the Dictionary is just a case in point: When the code base starts to grow, your services will be registered with various properties, all with more or less well-defined semantics. This is your low-level architecture emerging, and you will want a handle on it before it turns into a copy-and-paste-fest.

That vanity-item of a method signature is part of that handle. This handle is the layer where you will express certain rules about how your application is using OSGi. You may have an architecture specification saying this, but it is basically worthless unless there is well-tested code enforcing it. It is also important that this layer simplifies and facilitates OSGi usage for the layers above, since developers by nature don’t take to using something that doesn’t make their life easier.

So about that forward progress: I found that once this layer is established and accepted as a thing of sufficient value, forward progress and maintaining existing code becomes easier. If you don’t have it, be prepared to find similar-looking util methods and Dictionary lookups on string literals scattered about your code base.

Hmm. Once I got this down, it strikes me what trivial advice it is. Don’t, like, duplicate code. However, in the face of OSGi, and with no well-known place to put the common code, you’ll be surprised at what can appear. Condensed advice, then: OSGi is super, but it’s a bit unwieldy code-wise, so don’t be a slacker. Sorry if you expected the world to be on fire by this point.

And service filters. Don’t get me started on service filters. That’s for my next post.