Archive
JotBot post mortem: (J)Ruby desktop application development
Today JotBot was released. This is, to my knowledge, the first commercial desktop Ruby application. The web world has been invaded by Ruby, Rails and the crop of post-Rails web frameworks (Merb, Ramaze, Sinatra, etc.), but the desktop world remains largely untouched by Ruby’s rising star. This post mortem explores how Ruby fares for creating desktop applications and our experiences at Happy Camper Studios in developing our first commercial product, JotBot.
Project rational and tool selection
Purpose: JotBot is a desktop-based time tracking app that uses a “pull” model. That means you don’t remember to go log your time, instead JotBot comes to you at a configurable interval and asks you for a time log. (If you want to get a feel for the app, there’s a quick tour video on the JotBot home page that shows off most of the features.)
JotBot prompt
This project was started by me, as a side project because I couldn’t find anything that did quite what I wanted. The “pull” nature of JotBot was something quite important to me as I have a bad habit of getting distracted. I was doing contract work and needed to have accurate time logs for billing. At one point I used a timer that auto-reset and was on a 15 minute interval. When the timer went off I’d go log what I had been doing for those 15 minutes. This is basically what the first version of JotBot replaced. It worked well enough that we decided to pursue a commercial version as a company.
Platform and tool selection: At the time I was doing Rails development so of course I considered doing a web app initially. What I quickly found was that I didn’t want to always have my web browser open to a certain web site. Even when I did have my browser on a given site, that web app getting my attention was severely limited by the browser. If I was on another screen (at the time I was a Windows user but I still heavily used multiple desktops), you could just forget getting anything that was going to grab my attention via the browser. So I decided it was probably best to go the desktop application route.
At the time JRuby was reaching a fairly mature and stable point. I was doing a bit of Swing work for a client and was able to leverage JRuby where possible to make the job easier. This work was proving quite fruitful so I decided to go with Swing + JRuby. This eventually lead to the Monkeybars project which grew up alongside my client work and JotBot’s development. Before I get flooded with people saying I should have gone with framework X, let me state that yes, I am familiar with Tk, GTK, wxWidgets and Fox having evaluated them all at one point and none of them solved my problems as easily as JRuby + Swing promised to. So, with those tools selected my development stack became: JRuby driving the app logic calling into Swing and other Java libraries utilizing the Netbeans GUI editor for my form layout and the Monkeybars and Rawr libraries for the primary infrastructure of the application.
What went right
Over the course of the project several aspects of our chosen development setup stood out as very positive contributors to the success of the project. In no particular order
- Running on the JVM via JRuby meant that our code “just worked” on OSX, Windows and Linux. At various times we had people working on each of these platforms contributing to the code base as well as just running the app on a daily basis. Not having to ever build a platform specific dll or so was very nice and knowing that our interpreter was very stable from platform to platform also meant that time spent chasing down platform specific bugs was very minimal.
- Easy access to very cool app bundling tools. This gave rise to the Rawr project which can produce .exe or .app files for native looking distribution. More importantly you can generate all of these from any platform (Linux, Windows, OSX). Once we had our rake tasks in place, it was a simple matter of “rake installer:win” and bam, I have a versioned installer bundle I can upload to the website. The .dmg bundler was the one exception, that uses Apple provided tools and can only be run on a Mac.
- The GUI editor that comes with Netbeans make layout very easy. Being able to see our forms and tweak them out in a rapid prototyping fashion was extremely handy. Lines of Java code we wrote or maintain by hand in JotBot: zero.
- Ability to integrate with the desktop via JDIC. We decided early on that JotBot really should be an app that lives in the “system tray”. This presented some problems for Swing as it doesn’t have this ability natively (at least pre Java 6 it didn’t). Thanks to an active Swing community this problem had been solved for us via JDIC, the Java Desktop Integration Components. Their system tray support allowed us to look native on OSX, Windows, Gnome and KDE with almost zero extra work from us.
- Monkeybars matured and made Swing fade into the background. Creating a new form and adding some event handlers became so ridiculously easy I have to think hard how I even did it before. There are still rough points here and there but for the most part, Swing is tamed and the common stuff is wickedly fast.
What went wrong
Of course the project wasn’t all sunshine and roses. We ran into several late term problems that caused not insignificant delays.
- Active Record, our original ORM choice, did not play well when the Ruby files were compiled in a jar. I plummed the depths of AR for days but was unable to even find some of the meta-added methods that were failing. I’m not an AR expert and I didn’t feel like becoming one, so we decided to switch to a less “magical” ORM system. We couldn’t use DataMapper because their core Data Objects library is written in C and I was cautioned against using the existing Java conversion. So we settled on Sequel (possibly the worst named library EVAR!). We had been using Derby with Active Record which had worked flawlessly, unfortunately Sequel doesn’t support Derby. I looked into adding support but when I found out Derby doesn’t support the limit key word, the implementation of .first became significantly more complex. Faced with the choice to do non-trivial work on Sequel or pick something that could be supported, I opted for the latter. We settled on H2, a pure-Java SQLite like solution. We did have to create an H2 adapter but it was a matter of overriding 2 or so methods and was done in an afternoon (code posted here). So we went from Active Record using Derby to Sequel using H2. Very few bugs were introduced by this changeover which was incredibly encouraging.
- The Group Layout is not the best at making your forms look native with regards to spacing. Group Layout is what the Netbeans GUI designer uses, for those not intimately familiar with this stuff. We love the GUI designer, it is incredibly valuable, but the layout system leaves a bit to be desired. We designed primarily on Macs and there most things looked fantastic, but on Windows there was often a bit too much extra space. We took pain to make sure almost everything was dynamically sized, so that there was never accidental “left over” space from a component we sized on the Mac. In the end I’m reasonably happy but I often wonder how something like MiGLayout would have fared.
- Packaging up a Ruby application in a jar file is not something most Ruby libs are prepared for. We had some path issues that took time to resolve. Things like file:// getting prepended to the path to a file in a jar file when referencing __FILE__ with or without various expand_path shennanigans. We have bits of code that correct for this, but overall it’s probably one of the most “raw” parts of JRuby in terms of cleanliness. The rest is so fantastically good, paths to files in jars and such jump out all the more. Like with the compilation of Active Record classes in a jar file, these issues came up late in the dev cycle when we were preparing our installer and bundler tasks. Some of these issues have been resovled now so I would expect the next time around for this to be less of a pain point.
Conclusion
So would I do it differently if I was starting again? I firmly believe that JRuby is the way to go for ease of packaging and disribution, not to mention for access to libraries. Swing vs SWT is sort of a contentious topic although I think in the end Swing is available in more places and has more general support and a greater variety of 3rd party components available (we used several SwingX components such as JXLayer and JXTreeTable). Monkeybars was built on Swing but could just as easily support SWT if the demand was there. Speaking of Monkeybars, the choice to build a Ruby helper layer was definately the right choice. The amount of work needed and maintainability of our code before we switched over from the initial raw Swing calls to Monkeybars compared to afterwards was striking. Getting even just a few key pieces of Monkeybars in place dramatically reduced the code we needed to write and its complexity. I highly encourage anyone doing Java integration work to take your top 2 or 3 pain points and spend a bit of time to make them just go away. I did the initial event handling registration and routing code in Monkeybars in an evening. Those 20 lines saved us hours upon hours of work and reduced our code’s complexity significantly. Small things like that can have huge impacts on your project so get them in from the beginning.
I think in the end JotBot demonstrates that doing high quality desktop applications in Ruby is viable today. There’s still lots of apps that can’t / won’t be good web apps, why shouldn’t they be written in Ruby?
EDIT (19 Jan 09) – Added link to H2 adapter code on Rawr mailing list.
JRuby, still the fastest Ruby implementation
A fairly comprehensive benchmark test has been run across all available (and even some pending) Ruby implementations.
JRuby continues to be the fastest 1.8 compatible implementation by quite a long way.
Converting Java developers
I’m the resident “JRuby guy” most everywhere I go. Coupling a lack of Ruby-loving attendees and my predilection to get into “conversations” about the relative merits of various programming environments, the title all to often fits. Guilty as charged your honor. So, tonight I was asked an interesting question:
How do you convince a Java developer to use Ruby?
My answer:
You don’t.
My second answer:
You wait until the person is unhappy with what they have and then you be there as a resource to them when they go looking for a better solution.
Basically, the long version of “it depends”. The particular instance I was being asked about was a fairly large team (30) using Spring MVC. They all knew Java of course and one senior member of the team was trying to convince the team lead (also at the meeting) that JRuby or an equivalent would be a good way forward. This team lead was pretty skeptical, (surprisingly so given my previous interactions with him), and wanted to know what this extra layer of abstraction was buying him. I replied that it vary well may not buy him anything. I think this was an answer he definitely did not expect and I also think it was exactly the “right” answer.
I went on to elaborate on my “it depends” answer, probing to find out where their bottlenecks were. Was it hardware, delivery time, team size? The first answer was performance, “it has to run fast” he said. Some more back and forth and it seemed performance was important but not constraining in the way it would be on a single desktop or an embedded device. No, this was a web app, and he eventually conceded that three new developers is probably a lot more expensive than three more servers. I brought up the fact that they were using Spring instead of just straight Servlets. Why? That’s potentially a ton of layers of logic to go through. Clearly there was a value to the tradeoff and my point was soon made. After about 15 minutes of questions from the team lead he seemed quite satisfied and stated that he felt much better about the prospect of moving to something like JRuby now after talking to me. He even said that he would like me to come speak to his team and field their questions about JRuby.
This was not the first time this has happened to me and I hope it won’t be the last.
So, my takeaway for all Ruby advocates:
- The hard sell of Rails may have worked to get lots of attention for Rails/Ruby and for that I thank everyone who helped make Rails a success, but it won’t pull everyone across.
- After 2+ years of hype, hype, hype, a realistic explanation of what Ruby can and can’t do, what it is good and not so good for is a welcome thing to many skeptics.
- Even the doubtful can be converted if given the proper care and attention. Be passionate, be sincere and be honest and that will come across with far more weight than your expose on how higher order functions are the solution to all of programming’s ills.
- JRuby is the magic bullet that makes a lot of people even capible of conemplating a migration over to Ruby-land. It integrates with Java flawlessley and it’s the fastest Ruby available so that helps a whole lot for the peformance-minded crowd.
RubyConf 2008 proposal accepted
After being shot down for all my proposals at the various regional Ruby conferences (MountainWest, Lone Star, Ruby Hoedown), one of my proposals was accepted for the big tamale, RubyConf! The talk is entitled Monkeybars: easy cross platform GUIs. To my knowledge this will be the first JRuby related project to be featured at RubyConf (or any other Ruby conference for that matter) so we’ll have to see how the J-phobia plays out. I’m actually still pretty surprised my talk was selected, I guess super awesome GUI development is too good to pass up, even if it’s got some Java mixed in there.
What Ruby developers really want
Survey results, 50% of readers to Ruby Inside want more performance
What’s your biggest beef with Ruby today? A need for a better thought out sandbox model, people conflating Ruby and Rails (my personal vote), or some other issue? For 50% of Ruby Inside readers it appears to be performance. This is interesting because it’s one of the easiest things that can be remedied RIGHT NOW. Yup, just switch to JRuby and (barring some native extension problems) you’re up and running and faster. Sometimes this performance boost is quite significant as the JRuby team regularly demonstrates in their blog posts. So what are you waiting for, go grab the fastest Ruby available and start enjoying that performance!
Gemini dev update #2
So much has changed since my last update. First off, we’ve created a system for managers to be plugged into the basic game state. The three core managers are game object, update and render. The default versions of these three get auto-added to a game state unless you specify your own to use.
Why might this be useful or interesting you ask? Well consider the problem of one behavior we were working on. CollidableWhenMoving hooks into a game object’s move method using on_after_move. The behavior then needs to check the object that just moved against anything it might have collided with. There’s a basic behavior (BoundingBoxCollidable) that does the actual coordinate checking, but CollidableWhenMoving still needs to get a list of other game objects to check against. Previously, the behavior was keeping an internal list of objects it had been added to. This sort of worked, except that it used class level variables which really breaks down if you want to be able to pause, swap to another state for a while and then come back. Also, other collision’ish behaviors would have to maintain their own lists of game objects and there was no way to share those lists so CollidableWhenMoving could only collide with other CollidableWhenMoving objects. Oh, and it was also an ugly hack.
So the answer was clear, we needed a way for behaviors to generically get information. This evolved into the idea of a state having some number of managers that could provide that information. So now after the movement of a game object, CollidableWhenMoving can query the GameObjectManager registered on the state and ask it for an appropriate list of objects to check against.
Another big improvement was the idea that behaviors should be able to be used as interfaces. So kind_of? was enhanced to check against behaviors as well. Also a new behavior class method: depends_on_kind_of was added to allow a behavior to express the need for another behavior witout having to specify the type. Unlike depends_on, the depends_on_kind_of method does not require or add any dependant behaviors, it simply sets up a instantiation check that will raise an exception if there is not a sub-class of the declared behavior on the object. This allows you to depend on some kind of algorithm object, even though you don’t care which one.
The keymap concept has been enhanced to give you some fairly nice syntax for setting up input processing. Here’s the keymap file for the example pong-like game.
map KEY_HELD, :source_value => Input::KEY_Q, :destination_type => :p1_paddle_movement, :destination_value => :up map KEY_HELD, :source_value => Input::KEY_A, :destination_type => :p1_paddle_movement, :destination_value => :down map KEY_HELD, :source_value => Input::KEY_O, :destination_type => :p2_paddle_movement, :destination_value => :up map KEY_HELD, :source_value => Input::KEY_L, :destination_type => :p2_paddle_movement, :destination_value => :down
The destination_type and destination_value are totally arbitrary and should be customized to make sense for each particular game. Other supported input map types are key_pressed, key_released, mouse_moved and mouse_button_pressed/released.
Finally, we’ve split out the example game into the examples directory. It’s now an actual game with animating sprites, scores and two player controls.
50% performance boost, for free?
Wouldn’t it be nice to live in a world where you can wake up one day and have some project of yours just magically run 50% faster? Well my friends, JRuby is a world of mystery and amazement. In the past few days the JRuby team has been working feverishly on their Java integration rewrite. Along the way Charles Nutter managed to clean up the number of layers a Ruby call had to go through to get to Java. The net result: my test case for Gemini that I posted about earlier when from a frame rate of 27 fps to 40 fps. None of my actual code changed, I just swapped out the jruby.jar for a newer build. Swapping them back takes me back down to the 27 fps range. So, mana does fall from heaven occasionally, at least in JRuby land.
Java game library + JRuby + awesome DSL = Gemini
I’ve been working with JRuby to build high level systems on top of powerful Java libraries for about 2 years now. My most successful endeavor has been Monkeybars, a libarary for making Swing a lot easier to use. My other project has been in the game development arena. The project name is Gemini, and although it’s not nearly as advanced as Monkeybars, I was able to get to a significant point over the weekend. Yes, here in all its glory, the simple “crapton of sprites bouncing back and forth demo”.
Now you’re probably saying, big deal, I can do that in like 40 lines of code in <insert favorite programming language/framework> and no doubt you would be correct. What’s interesting about the way Gemini does things is in how it gives you an extremely flexible way of compositing together your game objects and your application as a whole.
The code to get this going using Gemini is pretty straightforward. We have a game state that is loaded by Gemini when it starts up. That game state then instantiates 1000 of our game objects and they proceed to load their images and care for their own logic of bouncing around on the screen.
Here’s the code in main.rb that kicks off the app:
require 'gemini'
Gemini::Main.new("Test game", 640, 480)
And here is the state that is loaded by default when Gemini runs: main_state.rb
class MainState < Gemini::BaseState def load @sprites = [] 1000.times {@sprites << Duke.new} end def update(delta) @sprites.each { |sprite| sprite.update(delta) } end def render(graphics) @sprites.each { |sprite| sprite.draw } end end [/sourcecode] Finally we have the Duke class that was instantiated in the MainGame class: [sourcecode lang='ruby'] class Duke < Gemini::GameObject has_behavior :Movable2D has_behavior :Sprite has_behavior :UpdatesAtConsistantRate def load self.image = "duke.png" self.updates_per_second = 30 @direction = :right self.x = rand(640 - image.width) self.y = rand(480 - image.height) end def tick if x > (640 – image.width) || x < 0 @direction = @direction == :right ? :left : :right end self.x = x + (@direction == :right ? 1 : -1) end end [/sourcecode] Apart from some hackish ternary ifs in the Duke class I think the whole thing is pretty darn readable, especially considering all that it is doing. As you can see, everything is extremely declarative including those has_behavior calls in the Duke class. Those nifty bits of metaprogramming bring in all the crunchy goodness of the class and make the x, y, image and updates_per_second methods appear. Movable2D gives us a move method and also pulls in the Positional2D behavior that provides the x and y methods. UpdatesAtConsistantRate gives us an update method that calls tick at the rate specified by updates_per_second. Finally the Sprite behavior gives us the image method and the draw method that gets called in the MainGame class. Most of this is bare bones at the moment. What if you don't have the UpdatesAtConsistantRate? When update is called you asplode. Same for Sprite / render. So clearly a more flexible game state is needed, probably involving the game object behaviors auto-registering themselves with the game state to be called during an update/render cycle. But as a whole I'm quite happy with the way Gemini is starting to shape up. Next on the agenda is getting keyboard mapping hooked into the RecievesKeyboardEvents behavior so you can control duke, then we'll have what could pass for a real game.
My Game Design Podcast
- An error has occurred; the feed is probably down. Try again later.
Get OUT
Does Not Compute 