Notes from Eclipse Day at Googleplex

Before I go into any details, I want to say that the Google t-shirt that I got for attending was almost worth the airfare, hotel stay and car rental. Man am I a cheap date or what? Also I would be remiss if I didn't thank Robert Konigsberg and the rest of the Google Open Source Office for organizing the event.

It was great to meet Ian Skerrett, Bjorn Freeman-Benson
, and others that I have only had opportunity to communicate with over blogs or mailing lists. It was also nice to catch up with Chris Aniszczyk and Wes Isberg there as well. Of course while this is all nice and dandy, lets get to the actual event.

Eclipse @ eBay: Michael Galpin
Mr. Michael Galpin did what amounted to the "keynote" address of the event. He went into a brief history of eBay and the technologies used to develop the site and then went into some of the work that has been done at eBay with Eclipse. Before I continue, I should mention that I worked as a contractor for eBay for a little over a year and that I have a better perspective than most on what is going on there. eBay is HUGE into Eclipse, in fact, I wonder if there are many organizations more committed to using Eclipse and working on creating developer tools using Eclipse. Still I found the presentation on what eBay is doing with V4 in the presentation layer quite eye opening. I was working on an Eclipse plugin to support their internal SOA framework, so I was a bit segregated from V4 work. Now I kinda wish I wasn't so segregated, some of it is quite cool. I mean I know I always wanted to leverage Dervlets, but Spyglass is pretty cool. There is a nice PDF of the presentation notes available at this link. One thing to note is how Java centric eBay really is, just about everything is a Java class. I wish I had asked during my time there why the decision was made to do that, particularly the tradeoffs. One other thing, I wish my plugin work and the SOA framework had garnered a bullet in that presentation somewhere, oh well. I mean Wes Isberg's simple, but cool, Auto-configuration plugin got a mention. Not that I am jealous or anything. ;)

One final note, Mr. Galpin literally said that there are plans to open source all the tools shown during that presentation. People should hold them to that promise.

How Mylyn Changes the Way I Develop: Bjorn Freeman-Benson
After the eBay session, the event bifurcated into two parallel tracks. The net effect was that you needed to choose which event to attend. While I have an abstract interest in CDT at the current moment, I was more intrigued by the session on Mylyn. I mean I have tried to use it and I see the arguments for something like that. What I haven't done is well, get it. I have tried it and never managed to successfully integrate it into my workflows. I have gotten to know and respect Mr. Freeman-Benson while interacting with him over at the Eclipse Dash project, so I was hopeful he might allow me to 'get it' or at least motivate me to try again.

Before I get into the notes I have from the presentation I need to pass off some advice. This goes to Bjorn or anyone else giving a presentation, don't ever give a presentation that someone else prepared, period. I know it was probably due to the fact that Bjorn was stepping in for someone else, but yikes.

In the beginning Bjorn covered the motivation for Mylyn, things that I think everyone will admit that modern software engineers have to deal with even with a nice IDE like Eclipse. The problems are the need to context switch, to mulitask and to manage information overload. He did not go into the traditional ways that developers in Eclipse try to manage this, but it bears mentioning. Eclipse users usually use multiple workspaces, multiple workbench instances and try to make use of things like project/team sets. None of the traditional ways are satisfactory. So what is Mylyn's approach? First off, I love this, Mylyn allows you to keep all your stuff in just one workspace. Mylyn's focus is on cutting down the full workspace to only those elements that you need at a given time.

So how does Mylyn accomplish some of this? Well in Mylyn the notion of a task is made a first class citizen. Before you do anything in Eclipse using Mylyn, you need to do it in the context of a task. Mylyn has stuff to manage tasks and to integrate with common issue tracking packages like Bugzilla and JIRA, among others. Mylyn also allows you to be able to acquire your tasks from multiple sources. So once you have a task, Mylyn stores contextual information about what elements in the workspace you associate with a given task. This is where some of the black magic kicks in, Mylyn has a Degree of Interest Model that it maintains in order to know what to filter out of your views. This has always been the rub for me to start with Mylyn, everything is hidden at first and you must start from a task.

The cool thing though is that once a context is created, it is possible to share that easily with other team members. Now the cool thing for me is that the context is not shared via a team provider, but that Mylyn will create an attachment in the issue tracker that another team member can use automatically in their Eclipse workbench using Mylyn. I love that since it is logically associated with the task and that they did not punt the trouble to the team provider.

Another cool item, in fact, a killer app type item for those who like Agile software development methodologies, is the automatic time tracker. When you are using Eclipse with Mylyn, you are always with in the context of a task, so when you are doing stuff in the IDE then, Mylyn will keep a timer. This is a killer app, since anyone who does agile knows that the most difficult thing to do is to estimate the time to implement a given feature. In fact, it is so difficult that XP for instance tells you to use 'yesterday's weather', in other words, look at the last time you had to implement a given feature, and use that time. Well that only works if you accurately track your time on task. Mylyn could make this seamless and easy. This feature alone makes me want to use Mylyn, even if I leave the filtering off all the time.

I want to make one more comment about the presentation on Mylyn concerning workspace provisioning. Bjorn went into alot of Q&A with the audience and did cover more details of Mylyn. Still, what struck me about the presentation was how close Mylyn is to a full workspace provisioning solution. I need to look at Eclipse Buckminster and check out Genuitech's Pulse product again. Workspace provisioning for me is almost as big a deal as P2 and the Eclipse provisioning effort. I mean how long does it take for new people to be able to install Eclipse and get all they need to be able to check out the code from SCM and start working with it? I mean really, how long, do you even know?

Building Great RIA with ATF: Philippe Ombredanne
First off, I want to say I got the chance to meet Philippe at this event and I really like him. Second, is there any Eclipse project that he hasn't been intimately involved with? I think the list of projects at Eclipse he has been involved with would rival some of my earliest Christmas wish lists to Santa, I mean wow! Most of the presentation (at least the part that I was able to grok) covered the ATF project's live Javascript debugger. I am not an AJAX developer, so I admit much of what he showed was a bit lost on me, but even I admit the fact that you could monitor requests easily. He demonstrated how a click in the ATF embedded browser to Google Maps would show up as request response pairs in a nice viewer inside Eclipse. Makes me want to try and do some Web mashups (me even!). The other thing that struck me was the fact that they took the Mozilla Firefox XUL Runner build and embedded it as a bundle inside of Eclipse. I thought that was cool, since it provides an embedded browser to play with that is not the Eclipse default browser and I wonder what kind of API I could potentially play with. You know I am thinking a Groovy/Eclipse Monkey script to try it out. Awww yeah!

Wiring Hacker Synapses: Collaborative Coding and Team Tooling in Eclipse: Scott Lewis & Mustafa K. Isik
First off I want to say that I love what this demo and plugin could entail. Being able to do shared sessions on the same document is the first step towards better collaboration and hopefully better software development in general. If you haven't seen the video, go Google for Cola and ECF and check it out. The simple demo that Mustafa did there was in the most part better than attending this presentation. I wish he had done some more demo instead of theory. Still this is a good first step. The problem comes later. With shared sessions, how do you integrate properly with Team providers and make sure that the workspaces are provisioned correctly? I think that they should focus on allowing a remote Eclipse user to "shadow" the workspace of a local Eclipse user. This way you have a "master" workspace that the local user is responsible for provisioning and the responsibility is removed from the remote user. Still ECF is very cool, I wish them the best of luck.

Concluding Remarks
At the end of the sessions, there was free food and adult beverages for all. I ran one of the two demo stations there. In hindsight, I think that expecting people to really have any more brainpower to devote to Eclipse after five continuous hours of sessions was a bit much. Still I really want to thank those that came over and gave me the chance to show off Groovy Monkey and discuss my plans for Eclipse Monkey.

Once again thanks to Google and to the folks at the Eclipse foundation for showing up. Thanks to all the participants who attended. The event was, for the most part, packed, not the smaller affairs that Eclipse Democamps that I have attended in the past have been.

Notes from Eclipse Democamp in Dallas, Texas

I am late writing this blog since the democamp actually occurred last monday and it is now a week later. I know my memory is a bit faulty, but I wanted to jot a few notes on things that stood out to me.

* Pulse is a cool tool that I really need to take the chance to learn and leverage. I think its ability to generate profiles is a killer app for a Eclipse developer like myself, since I need to switch between versions of Eclipse often. The stuff about being able to provision a workspace should be of great use on my next project. I mean talking to Mr.Williams about the feature convinces me that it is what you want Eclipse Teamsets and exported settings to be.

* You know you should really hold a democamp in a place that has an internet connection. I mean BJ's Brewhouse is a fine enough choice, but no wireless? I wonder how long it will be that any restaurant that has a meeting room will put in wireless internet, it is really starting to become necessary. Todd Williams from Genuitec had a devil of a time demoing what appears to be a really cool product, Pulse. The trouble is that Pulse is based on the Eclipse P2 provisioning platform, which is meant to attend to the Update Manager's legion of flaws. So without an internet connection, there aint alot to demo is there? I think Chris Aniszczyk must have thought of this since the Austin Democamp will be held at Nokia's offices near Stubb's BBQ. Btw, nice choice Chris, Stubb's is an Austin landmark. I would have chosen to do Chuy's or the Hula Hut somehow, but hey, Stubb's is pretty damn cool too.

* I need to come up with a set of cool example scripts so that people get why a monkey scripting tool is so cool. This is a problem of my own making, I see the advantage, particularly coming from the Eclipse plugin development side of the fence, but there aren't that many Eclipse plugin developers. Any help in this arena would be most appreciated. I don't want to see the blank stares again when I goto Eclipse Day at the Googleplex or do the Austin Democamp later on this month.

Using Groovy for Unit Tests inside Eclipse

I have blogged before in the past about using Groovy to write Eclipse Plugins. In fact, I think I am the only person in the world to ever write about the subject ever. This lack of interest in the subject is unfortunate for many reasons, but one reason in particular is the fact that writing in Groovy as opposed to Java can make many plugin related tasks simpler, particularly a lot of the tasks that make writing plugins no fun. For this specific reason, I am considering giving giving Groovy EMF a hard look as part of my ongoing efforts to make sense out of the Eclipse EMF Project. In my opinion, EMF could be like Grails if the generated code were in Groovy instead of Java.

Still I wanted to focus the blog entry on the use of Groovy for Unit Tests inside of Eclipse. This is because, in my opinion, that writing Unit Tests in Groovy is one of the main entry points for a Java developer to get their feet wet with Groovy and to see its benefits (the other two cases, in my opinion once again, are XML document parsing/generation and scripting). So here goes.

Framing the problem
The trouble with using Groovy in your projects is the same for using it for Unit Tests as when using it in production code from within Eclipse. The problem is that you usually have a build process that is run from the command line, often referred to as headless mode that you use for continuous integration or releases, and then you have the incremental build process that runs from within Eclipse. For headless (or batch/command line) builds, there is support from Groovy to assist with this, first there is the groovyc command that is equivalent to javac and second there is the Groovyc Ant Task. To take advantage of Eclipse incremental builds you really need to have an Eclipse plugin supporting that for you. This is where Groovy Eclipse can really be helpful, it provides the needed plugin support for incremental builds for Groovy from within Eclipse. For the next sections we will focus on solutions for utilizing Groovy for Unit tests.

Solution 1: GroovyTestSuite
There is a short snippet on using GroovyTestSuite from within Eclipse. There is one strong advantage to this method, since the groovy code is compiled when the AllTests are run, ostensibly this method could be used in both the headless batch build and from within Eclipse without any changes. So why not just use this method? Well there are downsides:

1. You can't use Eclipse's nice 'Recreate Test Suite' operation to keep the AllTests suite in sync. Over time, this is one of those problems that will bite more than once and usually when you least expect it.
2. If you don't have unit test classes compiled to classfiles, it makes running individual test cases difficult. This is something you feel more when you are trying to get a particularly challenging testcase to pass, in addition to the added compilation time, you will have to run the suite again. Of course you could create a new JUnit Testsuite just for the testcase you are trying to run, but then don't forget to delete it when done and don't check it into your source repository.
3. You have to make an assumption as to the current working directory. If you notice in the example code for GroovyTestSuite you have to assume that the unit test runner is being run from the project source directory. This is fine for Eclipse, just remember to check if that assumption holds for your batch/headless build as well.
4. You have to make assumptions as to the testcase's package. Once again refer to the code example for GroovyTestSuite. In the example see that the source directory, the package path is hardcoded, and the groovy file name is hardcoded. This is fine as long as you never change the source directory or refactor your test code into other groovy files or java packages, (*sarcasm alert*) which will never happen right? Now admittedly if you run your unit tests early and often, you will see this and then you will go and fix them immediately. Still better to not have to make those assumptions right?
5. Be careful about making Groovy code dependent on Java code or vice versa from within the same Eclipse Java project. This is a problem shared with the second approach as well. It is something you must consider (I know that groovyc will now compile Java code as well and it should be evaluated for use from within Groovy Eclipse or from your external build processes). There are a couple of ways to deal with the issue as I currently understand it. First, if you write only Groovy or Java code in a test project, there is no problem. This is usually an ok solution for greenfield test writing, but will not hold long term. Secondly, since there are no generated classfiles from groovy code in your project, you cannot allow your Java code to rely on any groovy code in the same test project period.

Solution 2: Groovy Eclipse Plugin
There is another option from within Eclipse, install the Groovy Eclipse Plugin and then add the Groovy builder and nature to your JUnit test project. Why do this? Well Groovy Eclipse will compile your Groovy code to classfiles and place them in a class folder for Eclipse so that Eclipse sees the compiled code as just another set of compiled Java code. This means that GroovyTestSuite is unnecessary and that running tests will not incur the full costs of compilation at start time, a significant advantage for developers. Another advantage is that the unit tests are compiled, so it is possible to easily run one testcase at a time. If setup correctly, using the Groovy Eclipse plugin makes developing/using Unit Tests in Groovy as seamless and invisible as possible. "So", you may ask, "ok Groovy Eclipse fan boy, what are the tradeoffs?" First off, even though I am a Groovy Eclipse committer and supporter, I am very well aware of its shortcomings, so I am not a fan boy. Secondly, well here are the tradeoffs/issues with this approach.

1. You must include Groovy as part of your batch/headless build process to compile the Groovy Unit Tests first, before the Unit tests are run. This is the same trouble you have if you are using Groovy in your production code. I will refer you to the section above titled 'Framing the Problem', for approaches on dealing with this. Using GroovyTestSuite only requires access to the groovy libraries at runtime.
2. Be careful about making Groovy code dependent on Java code or vice versa from within the same Eclipse Java project. This is not a problem unique to Groovy Eclipse or to writing Unit Tests with Groovy, but it is something you must consider (I know that groovyc will now compile Java code as well and it should be evaluated for use from within Groovy Eclipse). There are a couple of ways to deal with the issue as I currently understand it. First, if you write only Groovy or Java code in a test project, there is no problem. This is usually an ok solution for greenfield test writing, but will not hold long term. Secondly, be careful and avoid the following circumstances, A.groovy depends on B.java which depends on C.groovy or A.java depends on B.groovy which depends on C.java. This is an issue that needs to be addressed in Groovy Eclipse generally, either by leveraging groovyc to compile Java code or do an analysis to provide an Error/Warning marker to the user about the above circumstances. A useful heuristic traditionally is to allow groovy code to depend on java code or java code to depend on groovy code but not vice versa in the same project. If you hold to that heuristic, just consider the issue raised by number 3.
3. The order of the builders in the .project file. First off if you have groovy code depending on java code or vice versa in the same project, make sure that the groovy builder is before or after the java builder in the .project file. This can be easily checked by right clicking the project and then bringing up the properties dialog. Just look for the Builders section, you can easily remove, add, or reorder the builders in your .project file. Secondly, there are other occasions to be careful about the build order. In particular, this one bit me yesterday when I was trying to write a couple of unit tests for Eclipse monkey in groovy. I was pulling my hair out and about to give up on groovy unit tests for monkey (I mentioned I am not a Groovy Eclipse fanboy didn't I?), when I remembered to check the builder order. Sure enough, the groovy builder came after the plugin and manifest builders. Reordering the groovy builder to come before the PDE builders then allowed the PDE JUnit to actually run (Note to self: add better PDE support for Groovy Eclipse).

Conclusion
There are advantages and disadvantages to both approaches to writing Unit Tests for your projects from within Eclipse. The first approach, at first, seems more straightforward and simple, in particular no integration with external builds, compilation of groovy code is reduced to a classpath resolution problem. The second approach, using Groovy Eclipse, leaves the developer with the need to integrate Groovy into their external build processes, but in the longer term provides a more seamless and productive experience. So if you are wanting to write Unit Tests in groovy, give Groovy Eclipse a try!

Eclipse Monkey - Extension Points it is

For the new Eclipse Monkey I was considering use of Declarative Services in OSGi since I was intending to allow at least the core set of plugins to be able to used within the context of just a OSGi container. I have decided to use extension points based on my research of the past couple of days.

Neil Bartlett has an excellent writeup of extension points versus OSGi services: http://neilbartlett.name/downloads/extensions_vs_services.pdf

My number one concern was allowing people using Equinox, Knopflerfish(really what a name right?), Apache Felix, or other OSGi container could leverage the basic ability to write scripts. This concern addressed by the fact that bundles working in the context of a pure OSGi container only has to add the following two Eclipse Equinox bundles: org.eclipse.equinox.registry and org.eclipse.equinox.common. I don't think this is a bad trade-off because of the following considerations: dynamic bundles and tooling.

The first consideration is dynamism of bundles, whether they can be hot swapped in or out. This is the number one reason to use OSGi Declarative Services, it is made to work with the fact that bundles could come up or down. I don't think this will be a major concern for monkey, extensions will be things like definitions of script metadata elements, DOM objects, etc... These are things where dynamic hot swapping of bundles is not really beneficial. Besides I always prefer to ask the Extension Registry for the list of available extensions and not cache them. Without caching references from extensions, the code will already allow for dynamic hot swapping. If performance becomes a major issue, well that is what the ExtensionTracker is for.

The second consideration is a major one for me. There is some tooling in Eclipse within the PDE for extensions. Actually there is a lot of tooling. Tooling support for OSGi services, much less Declarative Services, is pretty much non-existent.

So given the thought process outlined here, I think Eclipse Monkey is safe sticking with extension points. As long as there is an Eclipse, it will have to support Extension Points and Extensions. If there is no Eclipse, well there is no Monkey either right?

P.S.: During my research on this I have to say I was impressed with the Knopflerfish site and its Eclipse plugin. I was also almost impressed with the Ops4J Pax Cursor. Honestly, is it so hard to integrate with the PDE Knopflerfish developers? I mean do you need to create your own builder and nature? You guys are almost there, just hook into the PDE and provide an Eclipse recognizable Target Platform. That is it, a few days work maybe, and then voila your stuff will work and all the PDE tooling will work for you. Same complaint with the Pax Cursor people, y'all are further along than the Knopflerfish people in that you not only support multiple OSGi containers and integrate better with the Eclipse PDE, but could I just get an Eclipse recognizable Target Platform? I mean y'all are just one little step away. Get Chris Aniszczyk(I take back what I said about Knopflerfish's name earlier) or someone else on the PDE team to point you to how to address the last hurdle. I mean guys, what better way to get some Java developers(in particular Eclipse Plugin developers) to try out your stuff. Also I should not just beat up on the Knopflerfish and Pax Cursor guys, I mean are you listening Eclipse PDE people? Its great you can run your bundles on a launch configuration that says OSGi container, but is it really that cool given that the only container that works out of the box is the Equinox one?? I have worked with some of those containers in the past and I know this aint rocket science. Why don't I do it? As my grandmother used to say, "If wishes were horses, beggars would ride." My interests right now lie elsewhere.

Eclipse Monkey - Core design notes

I am currently working on the new Eclipse monkey, starting this week. You know it is amazing how much time unemployment can free up for you or, er, contract sabbatical. I will make available a temporary public repository when I have something to show for this. I am always open to comments. I am writing this blog entry in order to state what are the design goals for this version of monkey, in particular the core plugin.

In many ways it is very very tempting to just take Groovy Monkey, rename a few packages and classes and then voila, here is the new Eclipse Monkey. There would be nothing wrong with that, I am after all proud of Groovy Monkey, its simple and it works. Still I would be remiss if I did not take advantage of this natural break to consider some refactoring and or redesign.

The main consideration I wish to take into account for the new version of Eclipse Monkey is to make it more modular. What I mean by that, is that I wish it to be possible to use monkey across the full spectrum of the Eclipse platform. Now what does that mean? I think I would be best able to explain it in terms of goals. It is a goal to allow people developing using just an OSGi container to be able to leverage some useful subset of Eclipse Monkey functionality in their applications. Allowing a bundle developer to be able to run scripts embedded in a bundle, specified via a URI/URL or specified directly in a string. Another goal would be to allow that core functionality plus some additional support to be used by a developer of Eclipse RCP applications. At the RCP level, there is now the notion of a workspace, SWT/JFace UI API and Eclipse jobs, so the monkey functionality to support that should be available. The same idea would apply to Eclipse Platform, JDT and finally the PDE (which would constitute the full Eclipse SDK).

There are some consequences of these decisions. There will be more bundles/fragments than would be otherwise, but at least those components should be simpler and smaller in size. I will focus on those that deal with the new org.eclipse.monkey.core component, which will run in the base OSGi container only.

We will have to consider OSGi services (perhaps declarative services), since we are missing Eclipse extension points for one. This will be decidedly different from Groovy Monkey where language extensions and DOM(s) are specified via extension points. Hopefully this will be possible without too much difficulty, also consideration will be made to allow parallel extension points, since the PDE support for extension points is much further along than with OSGi services. Of course allowing for parallel definitions, we will hopefully not have too many headaches with monkey bundle developers trying to define their extensions as both OSGi services and Eclipse defined extension points.

Another consequence of the OSGI only approach is that there will be no IResources API, so there will need to be support for allowing scripts to be embedded in a bundle, located via URI/URL or even to allow script text to be taken in verbatim. This will be a significant difference from Groovy Monkey where everything is assumed to be a resource in the workspace.

Finally, another major impact of this choice, will be the fact that support for threading through the Eclipse Jobs API will not be available. This in and of itself also makes me want to abandon the OSGi only approach. This means that use of IWorkspaceRunnable and Eclipse Jobs will commence only when the org.eclipse.core.runtime bundle is available.

There are two other things that this line of thinking has led me to consider. One, the metadata keywords and definitions should be pluggable themselves. As of right now in Groovy Monkey the script metadata is pretty statically defined in the code for the core plugin. If the layering support is to work, they have to be plugged by different plugins/fragments depending on the framework that monkey is being run in. Second, probably need a way to define which bundles are required or which platform the script is meant to run against. For instance, putting in a Job definition for a script running in an OSGi container is nonsense, however once in the RCP level, it is required. I think that there should be probably hardcoded values: OSGi, RSP, RCP, Platform, JDT and PDE. RSP would be the OSGi container plus the org.eclipse.core.runtime and org.eclipse.core.resources plugins, but no UI support.

Finally, all this work has to be done with the idea of backwards compatibility in mind. Hey I have written lots of monkey scripts I don't wanna throw away either.

New Eclipse Monkey

As you may or may not know, I have been working on Groovy Monkey for some time now. I think it is a useful tool and has a number of benefits over its predecessor Eclipse Monkey. Right now I am hoping that I will have the opportunity to take over Eclipse Monkey and take all of Groovy Monkey's improvements and make them more available to everyone.

I am writing a proposal for this (its a work in progress), you can view it at Eclipse Monkey Proposal. I would love your comments and feedback. I really hope this will happen, I wrote Groovy Monkey because I wanted a tool like that and I think others will too.