Thursday, June 19, 2008

JRuby on Rails good or bad for Java?

I am going to apologize for the incendiary title. I think the work the JRuby team has been doing at Sun, particularly with enhancing the JVM for dynamic languages, is great and a service to the entire Java community. The real point of this post is a question to Sun and the rest of the Java community. What is the question? Simple. Y'all why all the emphasis on JRuby and JRuby on Rails and not an equivalent amount of love (read: attention/resources) to the Groovy/Grails platform? I am combining consideration of both Ruby and Groovy with their equivalent Web frameworks RoR and Grails since they really are the killer apps (read: primary motivation for developers to try them) for both languages at this time.

So why all the attention at Sun to JRuby? Well I think there is probably a mentality at Sun to think they need to win more people over to the Java platform. That thinking leads them to wanting to support Ruby and Ruby on Rails on the Java platform. This is not the end of the world or necessarily bad thinking. Why? Well you would think the more developers on the Java platform the "mo' better" right? I admit that being a Groovy/Grails proponent, the attention leaves me a bit jealous, but as I said earlier, they have been doing good work and, you know, envy gets you nowhere. Still it begs the question, what about the people who are already on the Java platform doing their work? What is being done to improve their experience and to keep them happy working with the Java platform?

Note: I keep referring to the Java platform since Java really encompasses three components, the JVM runtime, the platform libraries and the Java language. I think of the platform as the libraries and the JVM, in other words, the full Java runtime minus the development time Java language support.

Give Groovy Some More Love Everyone!
Groovy/Grails is native to the Java ecosystem and should be (read: is) easier to support on the Java platform. Groovy/Grails has a far flatter (read: easier) learning curve for existing Java developers. How do I know that it is an easier learning curve? Simple, Groovy is the single reason I have not really tried to do development work in Ruby or another dynamic language. Every time I started to drift off the reservation, for one reason or another, I found that Groovy had it already. So why give up the JVM or platform libraries of which I am very familiar? Grails, the Groovy Web Framework, is still a relatively young technology, but is already showing signs of being one of the, if not the, best Java web frameworks out there. It is amazing how Grails is picking up speed right now. <sarcasm>Its like people want a better way to do Java Web applications and, get this, don't seem to be willing to dump all their previous experience with Java or their previous Java work in the trash. Isn't that just crazy??</sarcasm>

So therefore, I think Groovy/Grails is something that should be better supported, particularly with tooling. A better Groovy/Grails would help keep developers on the Java platform, how? Well remember there does not exist a Groovy/Grails platform outside the JVM and Java platform libraries.

Besides the fact that Groovy/Grails support should be far easier to achieve than Ruby/RoR support, there is the converse argument about JRuby and JRuby on Rails (JRoR?) bringing people to the Java platform. JRuby on Rails could also be viewed as a conduit for allowing development to leave the Java platform as well. Why? Well all new work could feasibly be run on native Ruby platforms if run on JRuby and, in fact, would probably run better on native Ruby platforms than inside the JVM. So unless you have lots of legacy with J2EE, there would be no hook to staying with the JVM or Java platform. Now I hate vendor lock-in, but Java has no vendor to speak of. I am all for allowing existing Ruby/RoR developers to try the Java world, but the question is where should your emphasis lie? I don't think that the Ruby world is anywhere near the size of the Java world, even on the Web.

So, in conclusion, the question is, is it more important to convert others to the Java platform or is it more important to keep and take care of the large Java developer base on the Java platform? My answer is yes. What do I mean? Well you can't just grow a community, you must also take care of what is already there, both aspects are important to a thriving community.

So please Sun, get past the NIH (Not Invented Here) syndrome and go hire a Groovy guy next to Mr. Nutter (no complaints for Mr. Nutter, he has been doing good stuff on the JVM for all dynamic languages). While you are at it, I am going to suggest (even though I am an Eclipse guy) putting better support in Netbeans for Groovy. I am getting a little tired of hearing how IntelliJ is by far the best environment for writing Groovy. I would also suggest improving the Eclipse plugin, but I don't think y'all will go for that(would you?). Any help to Groovy would be in your interests, you wont regret it.

For the rest of the Java community (especially the Groovy/Grails subset), I think I am going to start sounding like Public Broadcasting in the United States during sweeps. If you don't know what I talking about, they are officially called "Pledge Drives", unofficially they are called "Beg For Money Drives." So here goes in my PBS pledge drive voice**:

"Please pledge your support to Groovy Eclipse today. At the 1+ full-time developer level we'll get you an Eclipse mug and sticker. Show your Eclipse pride everywhere you drive or compute, your choice. At the 2+ full-time position level, in addition to the mug and sticker, we'll get a Groovy Eclipse shirt made up for you from somewhere. You know you have been enjoying Groovy/Grails for some time now and been looking for your opportunity to contribute, here is your opportunity. You wont regret it, I promise*."

Seriously, the Java developer world today is on Eclipse (not IntelliJ or Netbeans or whatever), so for Groovy/Grails to grow and be taken seriously by the vast majority of Java developers, Groovy Eclipse has to be first class. Like I overheard once while talking about this subject to a group of people, "People will change languages faster than they will change their IDE(s), so for me to introduce Groovy to my Java developers, I need it on Eclipse."

* If you do regret it anyway, just remember two things. First, I thank you for your support. Two, you may rant and rave at me, but limit it to emails, no 2am crank calls, please. :)

** I think I watched way way too much PBS as a child or something, but where else could I watch Dr. Who before the Sci-Fi network existed??

Monday, June 16, 2008

Versioning: My Happy Place

There has been alot of discussion about the JSR-277 and OSGi versioning standards. I wont revisit the arguments here since people far more intelligent than me have lots to say on that matter. If you want to dive in, go ahead and Google it, and I will talk to you later since you will have forgotten to go back to this blog entry once you are done. For the rest of you, I have to beg your indulgence, I am going to assume that I actually have the qualifications to pontificate on this matter. So here goes.

Defining Versioning
Why do you want to version something? There are alot of reasons for it, particularly when it comes to Source Control Management (SCM) or Version Control. You need to know the version of a particular class that you are working on in that case. Using version control when you are working on software is an idea that goes without saying. There are no debates on versioning or the formats of those versions, you get the one that comes with the SCM you are using, period end of story. No, the issue comes up when you are considering versioning a component, which for OSGi/Eclipse is a bundle and for JSR 277 is called a module. So what does a version stand for in this case?

I like the wikipedia definition of a version in this case. It is pretty generic, but it goes something like this, "Software versioning is the process of assigning either unique version names or unique version numbers to unique states of computer software." The definition tries to cover all cases, so the point to focus in on is the unique identifier of the unique state of the software. So what kind of state are we trying uniquely identify? Well changes to the software you as a developer have made of course, but I think that they fall into two categories at least at the component level. One category are those changes that affect dependency management because the external API on which clients rely has been altered. The second category, I will term bookkeeping for this blog entry. I term it bookkeeping, because it denotes internal changes that should, notice I say should, not affect clients using the external API. "Bookkeeping" comes into play because Software developers are human, and even though a change should not impact clients adversely, they still do. "Bookkeeping" lets a system integrator/admin note that a version of the component X worked, but X+1 sure did not, which is exactly the information someone providing support wants to hear. It is really difficult to say one category matters more than the other. Changes in the first category will prevent a system from coming up correctly, whereas changes in the second category could help explain why the system later goes haywire.

I like to view a version as a shortcut, so that a client does not have to check the full external API at runtime to know if there are changes that it should concern itself with. In fact, ideally the "external API" version should be some sort of "hash", if you will, of the external API that could be automatically maintained. More on the ideal answer later. The "bookkeeping" component is in effect like a branch in version control, it lets the developer go back and retrieve the previous version when problems come in from the users.

I could go on about how many coordinates are needed for this or that or what have you, but there are many many people far more qualified to comment on this and, you know what? They have commented, so if you are interested, go Google for them and check them out. What I am going to do is describe what in my ideal world versioning would do for me.

Versioning: James Ervin's Happy Place
I am not a big Adam Sandler fan, but I am a guy, so I will say that in the movie "Happy Gillmore", I was moved by Happy's idea of his "Happy Place." So what I am going to describe is my "Happy Place" for Software Versioning, though I will say that the "Happy Place" described in the movie is far superior to the one I will go into.

First off, any versioning scheme on the component level has to take into account that all software changes fall into one of the two orthogonal categories that I defined above, they are either external API or, for lack of a better word, internal "bookkeeping" changes. Any tooling support should be able to distinguish.

Secondly, I would like the notion of backward compatibility to go away. You change the external API, you create incompatibilities period. Incompatibilities between versions are a two way street. I mean if you make so called "backward compatible changes", then you shouldn't break existing clients, well that's great right? The problem with this view is that it assumes that your new stuff will be automagically available everywhere, which really does not take a full lifecycle view of your component into account. What you do is allow for a set of clients to be written that will be unable to use your pre-existing components. I don't have a great answer for this, but I would like people to take a greater view into account, all external API changes could force incompatibilities, not just ones that break backward compatibility. Once external API is defined, you need to treat it like set cement. How to potentially better deal with set cement? I try and deal with it in the next point.

Thirdly, I would like better API tooling support. This comes from point two, I want better tooling to help with what is in effect an intractable problem. First I would like to mention I know about the new Eclipse API tooling push and I want to investigate it thoroughly later on. I think it is worth mentioning that I view tooling support here in two categories again, what I would call "static support" and "client feedback support".

Static Support analysis in my view is what the Eclipse API tooling effort is focused on. Give me warnings if I add things to the external API to change the "external API" component of the version and then provide an automatic way to update it. Give me an error/warning to update the "external API" component of the version if I change an existing method or other backward compatible breaking change. Tell me to update the "bookkeeping" component of the version when I make any significant changes to the component that does not impact the external API. In other words, give me some help to make it more automatic. I, as a human being, have and will continue to forget to update versions without warning/errors markers to guide my way.

Before I go into the second category, I want to emphasize again that this is my happy place, my ideal world. "Client Feedback" tools to me are a potential answer to the problem of external API being set concrete after released in the wild. The trouble is that most of the time a whole API is not in use, usually only a subset, but what subset? That information would be invaluable. It would allow a component developer to know what parts of the API could be eliminated or given better documentation so that more clients would use it. "Client Feedback" would provide the targets for refactoring and could prevent the kind of cruft that eventually dooms any software product. How do you best get this feedback? Some sort of tooling support of course and beyond that I dunno. Did I mention this was my "Happy Place"?

Conclusion
So what do I want in a versioning scheme or tooling support? Prevent me from making simple obvious mistakes that I have done again and again. I want support to make me take into account that when a component is released, people write clients to use it and I have to keep them in mind for any changes, period. I think what I really want is the Staples Easy Button, which if you know me, is a metaphor for an impossible to build tool. Still there is alot that can be done before we need an 'Easy Button' to make this problem less problematic.

Notes from Eclipse Democamp in Dallas, Texas

I am late writing this blog since the democamp actually occurred last monday and it is now a week later. I know my memory is a bit faulty, but I wanted to jot a few notes on things that stood out to me.

* Pulse is a cool tool that I really need to take the chance to learn and leverage. I think its ability to generate profiles is a killer app for a Eclipse developer like myself, since I need to switch between versions of Eclipse often. The stuff about being able to provision a workspace should be of great use on my next project. I mean talking to Mr.Williams about the feature convinces me that it is what you want Eclipse Teamsets and exported settings to be.

* You know you should really hold a democamp in a place that has an internet connection. I mean BJ's Brewhouse is a fine enough choice, but no wireless? I wonder how long it will be that any restaurant that has a meeting room will put in wireless internet, it is really starting to become necessary. Todd Williams from Genuitec had a devil of a time demoing what appears to be a really cool product, Pulse. The trouble is that Pulse is based on the Eclipse P2 provisioning platform, which is meant to attend to the Update Manager's legion of flaws. So without an internet connection, there aint alot to demo is there? I think Chris Aniszczyk must have thought of this since the Austin Democamp will be held at Nokia's offices near Stubb's BBQ. Btw, nice choice Chris, Stubb's is an Austin landmark. I would have chosen to do Chuy's or the Hula Hut somehow, but hey, Stubb's is pretty damn cool too.

* I need to come up with a set of cool example scripts so that people get why a monkey scripting tool is so cool. This is a problem of my own making, I see the advantage, particularly coming from the Eclipse plugin development side of the fence, but there aren't that many Eclipse plugin developers. Any help in this arena would be most appreciated. I don't want to see the blank stares again when I goto Eclipse Day at the Googleplex or do the Austin Democamp later on this month.

Wednesday, June 11, 2008

Beware GStrings and their IDE support!

Despite the lascivious implications of the title, this blog entry has a G rating, as in Groovy!! You know, before I continue, I would like to apologize to y'all out there. I realize that punnery is considered, in the eyes of many, to be the lowest form of humor. But you know, why oh why was the name GString chosen and permitted to persist? Even though I am a strong Groovy proponent (some accuse me of being a fanboy), I can't stop the little 12 year old pubescent child in me from snickering every time I read about Groovy 'GString(s)'.

I want to preface my comments by saying that I love Groovy Strings (aka. GString(s)). I love the multiline strings, I love the ability to create regular expressions without slashing yourself all up, I really love the templating or string interpolation that allows arbitrary groovy expressions to be embedded in a string expression and I even love the fact that the GString is lazily evaluated. So why the title? I will get to that next. Before I continue, for you nit-pickers out there, I know that a GString is strictly only a string expression that is double quoted and contains an unescaped dollar sign, so I suppose I should have titled the blog entry as 'Beware Groovy String Support from within IDE(s)!', but really what fun is that?

As I started using Groovy and learned about its support for strings, I grew to love single quoted strings. Single quoted strings are the closest thing to Java string literals, they are immutable and are never evaluated to GString(s). They appeal to my need for simplicity. The problem comes later, as I write code, especially Unit Testing code, when all of a sudden I want to use a GString.

What is the problem? Why don't I just switch the single quotes to a double quote and voila! You ever try it? I don't know if IntelliJ's allegedly miraculous Groovy support does the same thing, but my baby, the Groovy Eclipse plugin sure does annoy me on this one. I suppose I could do a find and replace, but often my first instinct is to use the cursor keys and use that backspace button baby! And when I do that I run smack dab into a little feature that wants to autocomplete the double quotes, you know take " and change it to "". That is fine, but my first instinct is to hit backspace to remove the second and unnecessary double quote, which causes both of the double quotes to vanish, not what I want! So usually I stop, mutter an obscenity under my breath to myself, move my cursor in-between the two double quotes and hit the delete button, which I almost never use except in this one case.

Maybe this is more of a commentary on my coding style and how I interact with the IDE, but it has forced me to reconsider how I use strings in Groovy. From now on, I will try and avoid single quotes like the plague and use them only in cases where they are explicitly called for. In other words, I will no longer use the '' (single quoted) strings in groovy by default and instead use "" (double quoted) strings.

You know there is one other reason for this switch. I have had on occasion the need to convert my Groovy code to Java code and, other than the missing semicolons, fixing single quoted strings to be double quoted is second most laborious/annoying/mind-numbing step I have had to take.

So it is double quoted strings from now on for this boy.

Sunday, June 08, 2008

Using Groovy for Unit Tests inside Eclipse

I have blogged before in the past about using Groovy to write Eclipse Plugins. In fact, I think I am the only person in the world to ever write about the subject ever. This lack of interest in the subject is unfortunate for many reasons, but one reason in particular is the fact that writing in Groovy as opposed to Java can make many plugin related tasks simpler, particularly a lot of the tasks that make writing plugins no fun. For this specific reason, I am considering giving giving Groovy EMF a hard look as part of my ongoing efforts to make sense out of the Eclipse EMF Project. In my opinion, EMF could be like Grails if the generated code were in Groovy instead of Java.

Still I wanted to focus the blog entry on the use of Groovy for Unit Tests inside of Eclipse. This is because, in my opinion, that writing Unit Tests in Groovy is one of the main entry points for a Java developer to get their feet wet with Groovy and to see its benefits (the other two cases, in my opinion once again, are XML document parsing/generation and scripting). So here goes.

Framing the problem
The trouble with using Groovy in your projects is the same for using it for Unit Tests as when using it in production code from within Eclipse. The problem is that you usually have a build process that is run from the command line, often referred to as headless mode that you use for continuous integration or releases, and then you have the incremental build process that runs from within Eclipse. For headless (or batch/command line) builds, there is support from Groovy to assist with this, first there is the groovyc command that is equivalent to javac and second there is the Groovyc Ant Task. To take advantage of Eclipse incremental builds you really need to have an Eclipse plugin supporting that for you. This is where Groovy Eclipse can really be helpful, it provides the needed plugin support for incremental builds for Groovy from within Eclipse. For the next sections we will focus on solutions for utilizing Groovy for Unit tests.

Solution 1: GroovyTestSuite
There is a short snippet on using GroovyTestSuite from within Eclipse. There is one strong advantage to this method, since the groovy code is compiled when the AllTests are run, ostensibly this method could be used in both the headless batch build and from within Eclipse without any changes. So why not just use this method? Well there are downsides:

1. You can't use Eclipse's nice 'Recreate Test Suite' operation to keep the AllTests suite in sync. Over time, this is one of those problems that will bite more than once and usually when you least expect it.
2. If you don't have unit test classes compiled to classfiles, it makes running individual test cases difficult. This is something you feel more when you are trying to get a particularly challenging testcase to pass, in addition to the added compilation time, you will have to run the suite again. Of course you could create a new JUnit Testsuite just for the testcase you are trying to run, but then don't forget to delete it when done and don't check it into your source repository.
3. You have to make an assumption as to the current working directory. If you notice in the example code for GroovyTestSuite you have to assume that the unit test runner is being run from the project source directory. This is fine for Eclipse, just remember to check if that assumption holds for your batch/headless build as well.
4. You have to make assumptions as to the testcase's package. Once again refer to the code example for GroovyTestSuite. In the example see that the source directory, the package path is hardcoded, and the groovy file name is hardcoded. This is fine as long as you never change the source directory or refactor your test code into other groovy files or java packages, (*sarcasm alert*) which will never happen right? Now admittedly if you run your unit tests early and often, you will see this and then you will go and fix them immediately. Still better to not have to make those assumptions right?
5. Be careful about making Groovy code dependent on Java code or vice versa from within the same Eclipse Java project. This is a problem shared with the second approach as well. It is something you must consider (I know that groovyc will now compile Java code as well and it should be evaluated for use from within Groovy Eclipse or from your external build processes). There are a couple of ways to deal with the issue as I currently understand it. First, if you write only Groovy or Java code in a test project, there is no problem. This is usually an ok solution for greenfield test writing, but will not hold long term. Secondly, since there are no generated classfiles from groovy code in your project, you cannot allow your Java code to rely on any groovy code in the same test project period.

Solution 2: Groovy Eclipse Plugin
There is another option from within Eclipse, install the Groovy Eclipse Plugin and then add the Groovy builder and nature to your JUnit test project. Why do this? Well Groovy Eclipse will compile your Groovy code to classfiles and place them in a class folder for Eclipse so that Eclipse sees the compiled code as just another set of compiled Java code. This means that GroovyTestSuite is unnecessary and that running tests will not incur the full costs of compilation at start time, a significant advantage for developers. Another advantage is that the unit tests are compiled, so it is possible to easily run one testcase at a time. If setup correctly, using the Groovy Eclipse plugin makes developing/using Unit Tests in Groovy as seamless and invisible as possible. "So", you may ask, "ok Groovy Eclipse fan boy, what are the tradeoffs?" First off, even though I am a Groovy Eclipse committer and supporter, I am very well aware of its shortcomings, so I am not a fan boy. Secondly, well here are the tradeoffs/issues with this approach.

1. You must include Groovy as part of your batch/headless build process to compile the Groovy Unit Tests first, before the Unit tests are run. This is the same trouble you have if you are using Groovy in your production code. I will refer you to the section above titled 'Framing the Problem', for approaches on dealing with this. Using GroovyTestSuite only requires access to the groovy libraries at runtime.
2. Be careful about making Groovy code dependent on Java code or vice versa from within the same Eclipse Java project. This is not a problem unique to Groovy Eclipse or to writing Unit Tests with Groovy, but it is something you must consider (I know that groovyc will now compile Java code as well and it should be evaluated for use from within Groovy Eclipse). There are a couple of ways to deal with the issue as I currently understand it. First, if you write only Groovy or Java code in a test project, there is no problem. This is usually an ok solution for greenfield test writing, but will not hold long term. Secondly, be careful and avoid the following circumstances, A.groovy depends on B.java which depends on C.groovy or A.java depends on B.groovy which depends on C.java. This is an issue that needs to be addressed in Groovy Eclipse generally, either by leveraging groovyc to compile Java code or do an analysis to provide an Error/Warning marker to the user about the above circumstances. A useful heuristic traditionally is to allow groovy code to depend on java code or java code to depend on groovy code but not vice versa in the same project. If you hold to that heuristic, just consider the issue raised by number 3.
3. The order of the builders in the .project file. First off if you have groovy code depending on java code or vice versa in the same project, make sure that the groovy builder is before or after the java builder in the .project file. This can be easily checked by right clicking the project and then bringing up the properties dialog. Just look for the Builders section, you can easily remove, add, or reorder the builders in your .project file. Secondly, there are other occasions to be careful about the build order. In particular, this one bit me yesterday when I was trying to write a couple of unit tests for Eclipse monkey in groovy. I was pulling my hair out and about to give up on groovy unit tests for monkey (I mentioned I am not a Groovy Eclipse fanboy didn't I?), when I remembered to check the builder order. Sure enough, the groovy builder came after the plugin and manifest builders. Reordering the groovy builder to come before the PDE builders then allowed the PDE JUnit to actually run (Note to self: add better PDE support for Groovy Eclipse).

Conclusion
There are advantages and disadvantages to both approaches to writing Unit Tests for your projects from within Eclipse. The first approach, at first, seems more straightforward and simple, in particular no integration with external builds, compilation of groovy code is reduced to a classpath resolution problem. The second approach, using Groovy Eclipse, leaves the developer with the need to integrate Groovy into their external build processes, but in the longer term provides a more seamless and productive experience. So if you are wanting to write Unit Tests in groovy, give Groovy Eclipse a try!