When pursuing these three goals, which do you choose? Can you choose? How many? One? Two? Which one(s)? Surely not all three? I think mu.

The whole setup is off – it’s a misleading question. Speaking as a software engineer, Good is the center of gravity in this little triangle, and your only long-term yardstick. For argument’s sake, let’s assume something actually does end up Good, Fast and Cheap. That’s an actual big whoop, a real big deal – obviously, a lot of good work has been done. So you click your stop-watch, record the time it took and count up the money you spent on it. But: How do you assert how Good it really is? You don’t know yet, you have to keep track of that Goodness over time, from that point on. The world changes (it’s probably quite different already), so you will likely have to deal with additions and changes. This will reveal an important aspect of its Goodness: How easy it is to change, in terms of time and money. Also, hidden non-Goodness can pop up. Maybe someone cut a corner back in the Fast days to keep it Cheap? It happens, for more or less legitimate reasons. That will make it less Good later, or incur a little more time and cost to fix it, which makes it less Cheap and Fast overall. Things change, along with your scores for Fast and Cheap, even if they’re not on anyone’s MBO anymore. But are they really the factors you should be measuring in the first place?

Rant, rant. You can probably see where I’m going: I think measuring the Cheap can be misleading, because anything remotely worthwhile is also worth maintaining, and maintenance usually ends up being the biggest cost. The Goodness will deteriorate unless you put in that time doing the maintenance. Does this reduce your Fast score? Sure, but it’s like Oscar Wilde said: The only thing worse than maintaining is not doing maintenance. (Well, he would have, I’m sure.)

With any piece of software, it’s like this: When it’s Done – i.e. when noone is working on it anymore – it’s dead or dying. At that point, you can measure the time and money spent and say how Fast and Cheap it all was – which is useful for accounting and project management of course, but it’s not really relevant to the software, being dead – and well on its way to being replaced by a another project sometime later. Some things will take a long time find their shape, and will only end up as Good on paper, i.e. a measly one out of three. But some of those things really change the world, like the internet, the NASA space programs and the Manhattan project. (Yes, changing the world can be bad as well.) These projects require a whole new way of thinking to gel – slowly – in the heads of many brilliant people. You couldn’t have rushed setting up the internet by throwing more money at it. Most projects are obviously lesser efforts, but quality (here defined as being Good) generally does cost money, and it does take time.

Turning the tables completely on the whole question and flipping the polemics switch, we can phrase it like this: Unless it is actively burning time and money right now, it can’t be all that Good anymore, so forget about the whole thing being fast and cheap. Those are operational parameters that can be more sensibly applied to the day-to-day business of maintaining of the ever-important Goodness. Can we change this? Can we add that? If yes, and fast, you’re good. That’s a bit crudely put, but the polemics switch was on, after all.

If this little (but really simply syndicated) rant seems somehow familiar, it may be due to a high content of well-established, non-controversial truthism, but possibly also because you read my very similar Answer to a Question on LinkedIn.

Advertisements

I was pointed to a lightning presentation on mutation testing the other day. I like the idea of mutation testing, which basically is to mutate code randomly – though the operations are well-defined to keep things well-formed –  creating lots and lots of mutants. A mutant is a variant of your code, with one or more changes applied. Mutation testing tells you how many of these variants are actually caught by your tests. If a test fails on a variant, that mutant has been detected is killed. Obviously you want your tests to kill as many of them as possible, for the highest possible death rate. I piped up on the subject in the forum at smidig, and this is sort of a rehash in English.

In the presentation, a piece of code and its tests are gradually hardened to withstand the assault of the mutants. More mutants are killed, and we prune the mutation space to achieve smaller numbers of mutants. The final move, so to speak, is to declare the method arguments to be, yes, final. This prevents a lot of mutation, since mutants that try to change the parameter simply won’t compile. The mutation space is smaller. This is important, since the number of mutants grows fast, and quickly overwhelms even a small code base.

Now, assigning to parameters is a bad practice in some quarters. (In my own quarter, it’s a howler.) And forbidding the practice of assigning to parameters does seem to help with the mutation rate, but it’s not really tempting to go over the code and harden it piecemeal, littering it with explicit finals all over the place. Also, those modifiers are a double-edged sword. What purpose do they really serve? First, they’re not really needed (unless you do the inner class thing), so one could argue that the only purpose they serve is to prevent mutation. I hate it when some external factor, such as a tool, affects my code so it looks weird. Second, a human programmer could just as easily drop the modifier anyway, if it got in her way. Some programmers remove redundant modifiers on general principle, I do it on morning grumpiness as well.

So, it doesn’t really help. Not really. Worse, it’s quite intrusive code-wise. After all, the mutation testing is just one of many things that the developer needs to know about when writing code, and it gets tiresome after a while to remember that, oh, this weird little thing must be done to keep that automated thing happy, and this one because the so-and-so framework will use reflection to access only, say, protected fields … look out for comments scattered all over saying // make muclipse happy and so on. Stuff like this adds to the time needed to get new people into the project, and generally to the load on programmers’ brains. That’s a scarce resource, so keeping the code as idiomatic and non-quirky as possible is a good thing. Really, try it some time.

Could we lower the mutation rate on a more sweeping scale, and without affecting the code so much? Sure, don’t we have something that we often use to forbid a practice? We do, and it’s called a code standard. A code standard could say “don’t assign to parameters, fool”, and some other tool could enforce that part.

Your IDE is great for it – IDEA points out my code standard misdemeanors with yellow lights.

If the code standard forbids the practice, it seems natural that the mutation tester should know about it. Why should it bother with exploring variants that would be caught by checking against the code standard? In this particular case, it would imply that the mutation tester should behave as if all parameters were final. Presumably, that should lower the number of mutants all over the place, not just in the places we have littered with (all those increasingly mysterious) finals.

This is a lot of philosophizing for a fellow who hasn’t actually tried mutation testing. The idea of marrying mutation testers with code standards could be a silly one, and if so I would love to hear why. If anyone know any reason why these two, and so on and so forth; then comment and talk geeky to me.

Here’s a tip if you need to access HBase from an OSGi environment. I tried packaging them into two separate Hadoop and HBase bundles, but I ran into an unexpected, and quite unseemly, issue. To make a short story sweet (or so): You will need to package them together in a single bundle, and write a manifest that exports the HBase and the Hadoop packages.

The issue, shockingly, is that HBase takes the liberty of declaring a class in a Hadoop package! Specifically, the org.apache.hadoop.ipc.HBaseClient class extends Hadoop’s Client class, defined in that same package. This is the kind of necessary evil that pops up from time to time between two projects, especially in early days – after all, they are in 0.17.1 and 0.2.0, respectively. It does break an important OSGi rule: A package can only be imported from a single bundle, it cannot contain classes contributed from multiple bundles. Which means that other bundles can only ever depend on one of them, never both.

I haven’t thought of any import/export acrobatics that could resolve this, and I’m not sure if anyone should. The HBase bundle needs to see that package exported from the Hadoop bundle, or it can’t extend its Client class. Given that, the HBase bundle then needs to export that same package. I can’t imagine any way that would work, except by accidents in the implementation. Even if it somehow is an edge case within the spec, it certainly isn’t something I would like to spend too much think-time concerning myself about, let alone maintain. The quick fix is to package them together. Do that, and your stuff works.

Feel free to ask for the manifest, if you’re interested. I didn’t have it at hand while writing this post.

Maybe I’m slow, but I recently figured out a handy use for Generics:

    public static S3Service connectS3(String id, String key) {
        AWSCredentials creds = new AWSCredentials(id, key);
        try {
            return new RestS3Service(creds);
        } catch (S3ServiceException e) {
            return failConnect(id, e);
        }
    }

where failConnect is:

    private static <T> T failConnect(String id, Exception e) {
        throw new AWSException("Unable to connect " + id + " @ Amazon!", e);
    }

It feels like being a knowledgeable (runtime) adult, duping an ignorant (compile-time) child. The child is happy, because the connect method has its returns in place, and the failConnect method obviously returns the right type. So I can reuse it here:

    public static SimpleDB connectSDB(String id, String key) {
        try {
            return new SimpleDB(id, key);
        } catch (Exception e) {
            return failConnect(id, e);
        }
    }

failConnect can obviously “return” anything. And it does! Right? Ah, kids are such dupes.

Just to capture your buzz interest, this is DSL-related! Read on …

OSGi filters are under-modelled, because they’re strings. Strings are great per se, they’re portable and very expressive – after all, source code is strings. So, strings – great, love’em. Write’em every day myself, for money, donchaknow.

But OSGi filters represent a shift of responsibility. With specifications, there’s always a specifier, a vendor, and a user. Specifications can come with reference implementations and library support, to help vendors and users. (OSGi is pretty well off in this regard, compared to e.g. Corba in its day.) But the responsibility for filter representation has been passed through the chain and squarely onto the user. The specifier and the vendor have told the user: Just give us back a string, and we’ll look at it and possibly give you an InvalidFilterException back. And yes, it’s checked, because you should be prepared being a gal (or guy) who can’t append strings together correctly.

Fortunately for the users, they’re programmers. They will say: OK then, I’ll have a computer make sure this string is right every time, OK? Just watch me catch that checked exception of yours, and rethrow it as a ThisNeverHappensError.

(I may have forgotten the precise name of the actual exception, InvalidFilterException. That’s because I rarely ever think about it anymore.)

So we should make filter classes, build instances with code, and hang them in trees. Make them print themselves to a well-formed string. Yes, this is the good old shape-draw-yourself thing that we old-hand object modellers thought was a silly example of polymorphism.

I have a small OSGi library brewing, name of vanadis. I will be referring to that from now on. The Filter interface there looks like this, in a condensed version – look up the code for the whole shebang:

interface Filter {
    Filter and(Filter... exprs);
    Filter or(Filter... exprs);
    Filter not();
    String toFilterString();
}

Then there is the Filters factory class, a classic static factory with static methods all over it. But if it had an interface, it would be:

public interface Filters {
    Filter eq(String attribute, Object... values);
    Filter approx(String attribute, Object... values);
    Filter gt(String attribute, Object... values);
    Filter lt(String attribute, Object... values);
    Filter present(String attribute);
    Filter substring(String attribute, String substring);
    Filter objectClasses(Class... objectClasses);
}

So, to create a simple filter that works well for a WerewolfKiller service:

    Filters.objectClasses(SilverBullet.class).and(Filters.isTrue("fullMoon"));

Of course, the Filter interface can host methods that move us closer to DSL land, allowing us to express things with less typing:

   objectClasses(SilverBullet.class).and("fullMoon");

The HelplessFarmer service might want to query for an experienced, yet reasonably stable WerewolfKiller service:

   objectClasses(WerewolfKillerService.class).andGt("killCount", 5).unless("psychotic");

And so on. You might have noticed I pass class objects to the factory methods, not class names. Again, this is an attempt to keep myself DRY – calling getName() on class objects should also be done only once. In this case, the Filter object does it at construction time, so the class object never “hangs” on the tree. However, if making a large, distributed application, you might need some filter creation happening in an infrastructural capacity, which is likely to be in a context where the classes aren’t visible. In that case, using strings to create filters might be the only way to go. But my experience tells me that you will, more often than not, be constructing filters in the context where the classes are known.

Building filter classes and putting them in trees is really quite simple. In my last project, I made a support class that allowed us to build strings correctly and incrementally: Each filter object would hold a legal filter string, and appending stuff like e.g. this.and(that) would simply create a new filter object, with “&(” at the head, then the strings from this and that, ended by “)” at the tail. This might be perfect for your purposes, and very simple.

In vanadis, I have the whole structure represented as a tree. Not only does this save on string garbage, the resulting structure is more pick-apart-able, one can do automated analysis etc. This can be golden if you find that filter matching is becoming a bottleneck.

These are all just ideas, and a work in progress that I hope to make use of in my current job. I’m happy to take comments and feedback if you have any, especially on making the base classes more generally usable.

Look out, unbelievers. I’m on a crusade against the OSGi API’s these days. 

Admittedly, it’s not that the OSGi API’s are bad. They are quite complete and completely workable. The impression you get, though, is that they could do with a brush-up and a make-over, give them that coat of wax. Make them more… modern. They’re old-school. But then, OSGi comes out of the embedded space, and wants to be as portable as possible, so this makes sense. But couldn’t we use, say, the Collections API? 

Though I do like lots of the new-fangled Java features, I would argue against OSGi adopting them recklessly. Why? Because that’s not their job – they want to grow a mature, consistent platform. (Also, they’ve got IBM backing them – a virtual guarantee for glacial-at-best adoption of all the new and cool stuff.) It is a core technology, it can and should get away with being low-level and old-fashioned.

In short, OSGi targets a wide variety of execution environments, and tries to leave no (remotely relevant) configuration behind. It is you, the adopter, that gets to make the decisions that narrow this scope, and building good libraries on your platform of preference. Which, I suppose, would often involve Java 5 or 6.

If you want to build right on top of OSGi – maybe for some reason you can’t use any of those new application frameworks surfacing these days – you should invest some time in building a general, high-level, modernized API that fits your needs. Convenience methods and so on. Just so you can write more code dealing with your application, and less code dealing with putting values into java.util.Dictionary instances, or composing LDAP-ish query strings. Obviously, having everyone mess around this can produce lots of messy code, so that library support can be very useful.

I hope to get some time to outline some handy API’s for this purpose. First up is programmatic support for filter expressions – stay tuned.

Java 6 is finally out on the Mac, officially supported by Apple. 

Apple, whom we would like to keep loving’ – Apple, who really pushed it on this one – Apple, who still makes computers for humans – Apple, who still provides a fine, well-integrated Java on their platform without stringing it up in any proprietary doodads. Well, not like Windows once did, anyway.

I believed it was forthcoming, but I think the wait was excessive this time. It is their platform, so I could either just walk away from it if I don’t believe in their Java commitment anymore, or I could threaten them into taking me on so I can fix it myself. Only, I don’t think it’s good for me to get down with all the native code it takes to get a JDK up, wiring it up with the underlying platform and so on. 

Java on the Mac works great overall – it’s just that the soon-to-be-infamous lag could kill it. That would be a pity, because Macs are popular with developers now, and a lot of them do Java a lot. Java is directly pitted against .NET – which is really the battle between the platforms Windows and Java. Java is the best safeguard against Windows becoming an essential platform for some types of applications. You know, web-based stuff, stuff consultants do, and suchlike. This should be a good enough reason for Apple, who seem out to displace Windows gradually, to embrace Java – not instead of, but along with their own Objective-C-based platform.

And in the meantime, we should support the third-party porting initiatives, along with the OpenJDK, as best we can.