Using a REPL to debug Capybara/Selenium

The Problem
In the past I’ve always used ruby-debug (or more recently debugger) to set breakpoints and debug my rails integration tests built with Capybara and Selenium. A problem I’ve continually run into is that when the debugger launches it suspends all threads, including both the thread Capybara is running on as well as the rails server thread, so when trying to do something with Capybara that requires a server response it just times out.

Just to verify this behavior of ruby-debug/debugger, you can try: When the debugger launches, if you try something like sleep 2 to pass some time, the separate thread will stay suspended, and it’ll only actually print “done!” once you exit the debugger.

Use a REPL instead
The suspended threads issue can be avoided entirely by using a REPL (read-evaluate-print loop) instead of a breakpoint. Doing this with IRB requires some trickery to get the context right, but with Pry it’s super easy: This allows you to run Capybara methods (visit, fill_in, etc) directly, with other threads running happily so the rails server will respond as needed.

Using multiple Capybara sessions in RSpec request specs
Doing integration tests on apps with real-time multi-user features like chat is incredibly useful since it’s a pain to hop between multiple browsers with different users logged in to test them manually. It wasn’t immediately obvious to me that this capability is built into Capybara, but after spotting this post I adapted it slightly for RSpec request specs since that’s what I use.

A helpful helper
Here’s the helper I use to manage which browser is in use:

Example spec
Here’s how it looks in practice: This assumes the helpers sign_in_as and add_comment exist, just for the sake of keeping the example short. At first it was tempting to use a helper like “as_user(:bob)”, but each time Capybara.session_name is set to a new value it opens a new browser instance, so across a bunch of request specs opening as different user names you could end up with a lot of browsers open which can have a substantial memory footprint.

Routes and Responders in SproutCore: the Routes sample app
Previously we looked at a very simple sample application that uses responders to manage application state. Now that our responder-fu is strong, we can take a further step of using SproutCore routes to manage history via states.

The Underlying Technology
As amazing as SproutCore is, it did not in fact invent the methods it uses to handle history for single-page applications. Applications like Gmail have been doing this for quite a while, and it basically involves setting the browser’s location with JavaScript, then monitoring the location and taking appropriate action if the user has navigated to a previous (or subsequent) location in the browser history. In particular, these single-page applications use the location.hash that’s available in JavaScript, which provides the portion of the current URL that follows the ‘#’ character. For instance, if you’re at, location.hash will return “#inbox”. If there is no ‘#’ in the current URL you’ll just get and empty string. Here’s how SproutCore handles history, as illustrated by the Routes sample app:

Registering a route
In order to make use of routes in SproutCore, you need to register any route changes you want to watch for. This is done by providing a route string that is matched against the browser location, along with a target object and method you want called on that object. Since route changes will often coincide with state changes, it makes sense to use responders to handle them. The Routes sample application has a superstate, Routes.states.root, that all of the other states are substates of, so this is the responder that is registered: Now that this route has been registered, any time the browser location hash changes the route method in Routes.states.root will be called, with a hash that has a number property consisting of everything to the right of the ‘#’ character in the URL. SproutCore will now monitor the location hash for changes that match this route, either in an event-based way for browsers that support it, or by using a timer.

Setting your location
You set a route in SproutCore by setting the location property for SC.routes, for instance SC.routes.set('location', 'here'). In the Routes sample app, when a change in the location hash occurs and the route method in SC.states.root is called, that method simply passes the number property that was captured to the go method: In the go method, the location is set to the route number that is passed in, and as a nice touch the browser title is also updated to reflect the current route number. Finally, the state is set with Routes.makeFirstResponder.

Following route and state changes
Now that they key players have been identified, it may be useful to follow the flow of events when actually interacting with the app. Assume we’re running sproutcore-samples at http://localhost:4020. Immediately upon entering the app by browsing to http://localhost:4020/routes, the route that was set by SC.routes.set above will notice the initial URL as a “change”, call the route method in SC.states.route, and in turn enter the go method. Since the location hash starts off empty, the following code in go will default to the one route, and then set the location hash, browser title and state to one: So the URL that we intially see once the app is done loading is http://localhost:4020/routes#one. Now let’s say we click on another button, three for instance. Looking in resources/main_page.js we can see that all of the buttons have go as their action: Currently the state is, however when we look there we see that this state doesn’t define the go method itself. Instead, all of the substates simply define Routes.states.root as the nextResponder: This is what makes these substates of Routes.states.root, so the go call won’t be found in the first responder, causing it to go up the chain to Routes.states.root. This will in turn change the current URL to http://localhost:4020/routes#three, update the page title and set the state to three. Importantly, each time the current route is changed, those changes are reflected in the browser history. We could continue clicking buttons like this as long as we’d like, accumulating a history of being in the different locations that each button correspond to.

If we next hit the back button in our browser, that will change the browser URL back to http://localhost:4020/routes#one. Because of the route we added, this will trigger calling go in the root responder, and in turn the page title and state being updated to one.

The Routes sample app provides a simple, albeit contrived, example of how routes and states allow us to use history in our apps. In order to apply this to a real app, you would need to consider what states in your app you’d like to make available in the browser history, and how you would accomplish transitioning to those states from essentially any other state in your app. I only know of one real app that uses responders and routes for history management, SproutCore Tests. There are some interesting nuances to how it works with routes, so it’s certainly worth a look if you have the time.

Statecharts and Responders in SproutCore: a simple case
You’ve made it through the Todos tutorial and believe SproutCore is the Next Big Thing, so you start to dig deeper. The term statechart keeps popping up here and there, but it’s not entirely clear how to model and implement one for your app (see discussion). Now there are some excellent writeups on using statecharts in SproutCore, and in fact I highly recommend reading them prior to this post. The only problem with these writeups is that they use the statechart framework that Erich Ocean wrote, whereas SproutCore’s responders are considered a better fit for all but the most complex apps. It’d also be well worth the time to bite the bullet and read David Harel’s original paper on statecharts, which is remarkably readable.

We’ll start by looking at a very simple example of managing state using responders, the sproutcore-samples app Signup (code here and demo here). It has just three states: START, READY and SIGNUP. Although I don’t know how the original author of the Signup sample would have drawn the statechart for this (or if he would have even bothered), here’s one way to draw it:

We’ll trace through the app and see how it goes from one state to another, and what happens in each state.

START state
The START state is *transient*, meaning that the app should exit from it to another state automatically based on an internal event. In order to get into the START state, the main() function makes it the first responder when the app loads: Once in the START state, the didBecomeFirstResponder method is automatically invoked. Inside this method, we see that it simply looks at the hasContent property of the current account controller, and goes to the READY state if it has content or the SIGNUP state if it doesn’t. So we see that the START state doesn’t really do anything, it just serves as a switch between the two other states:

SIGNUP state
Since the currentAccountController won’t have any content when the app loads, it will go into the SIGNUP state. The didBecomeFirstResponder method will be automatically invoked, and do two things: set up a new user to edit and display the signup dialog. One interesting tidbit here is that a nested store is set up by calling This allows the entire user record can be committed at once when the signup dialog is submitted (or discarded if instead the user cancels). There are two ways the application can exit the SIGNUP state, either by the Submit button or Cancel button being hit in the signup dialog. Here are those buttons: Note that these buttons don’t have to set a ‘target’ property, since the actions will automatically go to the first responder, which is currently Signup.SIGNUP. In order for this to work though, the MainPane has to have it’s default responder set to the app’s namespace: If the user hits submit, then the nested store with the user record is committed and assigned to the currentAccountController, and if the user hits cancel then the user record is discarded. Either way, the last step of both the ‘submit’ and ‘cancel’ methods calls ‘Signup.makeFirstResponder(Signup.READY)’. One last thing does happen though as the state is changing. The willLoseResponder method is called in Signup.SIGNUP, which clears the nested store and signupController, and displays the mainPage (effectively closing the dialog).

READY state
The READY state displays Signup.mainPage, which will show the user associated with the accounts controller, or if there’s no user it will show a message indicating that. The READY state can only be exited by the ‘signup’ action, which just transitions the app back to the SIGNUP state: The other possible action is ‘logout’, which clears the current account controller but doesn’t affect the state.

What’s gained by using Responders?
Hopefully it’s now a bit more clear how responders can be used to manage state in this simple example, but an interesting question to ask at this point is what the app would look like without responders, and therefore what we’ve gained. Essentially, you’d have to weave all the state management in the responders into the other parts of your app, presumably the controllers. This would probably look fairly reasonable in the Signup example, but for more complex apps where you’d have multiple states that could be accessed from a single interface, you’d end up with lots of logic in your controllers to make them state-aware. SproutCore makes a good argument for making responders their own layer, beyond the standard MVC layers.

Keeping up with popular web frameworks: HotFrameworks

I spend most of my time using Ruby on Rails, but enjoy exploring other web application frameworks. There’s a wonderful diversity of frameworks out there, with more popping up all the time, but there are two challenges I’ve found with trying to find ones I want to investigate:

  1. Keeping track of them all: This isn’t terribly hard, but I find myself subscribing to a number of news feeds so that I can find out about new frameworks as they come out. If you want to look within a particular language of choice, you can generally search and find good lists that way as well. Looking across languages you can turn to Wikipedia or a few other places, but these don’t always stay up to date and can be biased (Wikipedia lists only 5 Ruby frameworks, and 3 JavaScript ones as I write this).
  2. Knowing which ones are worth looking at: This is far more challenging. When looking at a list of names of frameworks I’ve never heard of, the question that comes to mind is “which ones are people actually using?”. Not that popularity is a perfect proxy for how good a framework is, but I do put some faith in what fellow developers are into, and there are definite benefits to having a good sized community. In order to judge popularity, I look at github stats for projects living on github, and failing that will also look at how much traffic the framework’s site gets and how many people are linking in to it.

The Idea

After numerous occasions of such exploration, it occurred to me that I may not be the only one who does this sort of thing. I decided to formalize my methods for turning up nuggets of web framework goodness and share the results with others. I was also inspired by The Ruby Toolbox, which beautifully does what I’m talking about for Ruby plugins/gems. But unlike Ruby, where essentially everything interesting is happening on github these days, not all (or even most) web frameworks have their source code hosted on github. This meant employing some different tactics for rating popularity.

The Popularity Contest

Since github stats aren’t universally available, I decided to look at other measures of popularity. Traffic statistics seemed a reasonable approach, but these too weren’t universally available since many web frameworks don’t have their own domain (e.g. Tapestry is at a subdomain of, and traffic statistics are typically only available at the domain level. I also considered looking at things like Google search trends or Twitter mentions, but the problem with these is one of specificity. For example, I can be fairly sure that any time I see SproutCore come up on Twitter it’s referring to the framework, but Cappuccino mentions could very well just be talking about the caffeinated beverage. So the third measure I’m employing is the number of inbound links, since these are specific and can be checked against any URL.

So inbound links seem to be the lowest common denominator for measuring popularity, but it would be a shame to neglect the other popularity measures where they’re available. Plus, inbound links are sometimes biased, for instance many Django apps have a “Powered by Django” link at the bottom so Django has an enormous number of inbound links. What I decided to do is take whichever of the three popularity measures (github stats, site traffic, inbound links) are available for a framework, and average them. But since the raw statistics are on very different scales, I first put them on a 0-100 scale, and then simply average the three scores. It’s not incredibly sophisticated, but seems to get the job done.

The Implementation

See the results at:

HotFrameworks (

Currently there are really just three components to the site: rankings charts, individual stats for each framework, and a blog. This is just the first iteration though so there’s plenty more I can envision adding.

Comparing Cappuccino, RestfulX and SproutCore: Server Interaction

When implementing a trivial timer application in Cappuccino, RestfulX and SproutCore, one thing that struck me was how varied their approaches were to handling interaction with a server back end. Cappuccino has the lowest level abstraction here, while RestfulX takes care of more of the process and SproutCore has a high-level and somewhat complex system for server interaction.

The Cappuccino framework provides the CPURLConnection class, which handles XMLHTTPRequests, although the level of abstraction is fairly low. For instance, here’s what I use to submit a request to create a new project in Clocky/Cappuccino: This does allow greater flexibility for any type of back end, however.

RestfulX has a higher level of abstraction, with Service Providers for JSON, XML and others, and models that use the service provides when they’re asked to be created/updated/etc. This allows you to use create a new project (after setting it’s properties) with something like: Of course, the JSONHTTPServiceProvider assumes a standard Rails-style “REST” JSON API, which is great if you’re using Rails, but you’d need to roll your own service provider if you’re not using a back end that’s Rails-style or one of the other existing Service Provider types.

SproutCore clearly has the highest level of abstraction for server interaction, although it’s a bit complex so it may seem intimidating at first. There are actually 3 levels as I understand things: the model, the data store and the data source. Your models (e.g. projects in Clocky) are locally stored on the client in the DataStore, which is back end-agnostic. The data source layer lies between the data store and your server, so it handles putting together the right XMLHTTPRequest. What’s great is that the SproutCore actually keeps track of state on model objects, and you just have to make your application respond to these states appropriately. Unfortunately you have to roll your own data source, and for me getting SproutCore to talk with my standard Rails backend took almost as much time as doing the rest of the app (it would have been easier if I’d been willing to adopt the JSON structures to what SproutCore likes). Of course now I have a fairly generic Rails data source that I could use on future projects.

The winner: it depends
Determining which framework handles back end integration the best will depend on how you want that interaction to work. If you’re doing Rails-style REST on the back end, then RestfulX and SproutCore are going to take care of a lot of the details for you. SproutCore’s ability to track state and do local queries using the data cached on the client could save you a lot of work if you think you’d end up having to implement that yourself with the other frameworks. But if you roll your own back end or don’t plan on doing CRUD over HTTP then you won’t miss what the others have to offer by going with Cappuccino.

Comparing Cappuccino, RestfulX and SproutCore: Debugging

Now that I’ve finished implementing a trivial timer application in Cappuccino, RestfulX and SproutCore, I want to delve into and compare a few aspects of developing in these frameworks. Of course all of my comparisons will be biased by my choice of application to implement, so I’ll certainly miss exciting parts off these frameworks that I didn’t have the opportunity to explore.

One aspect of working with these client-side frameworks that I found challenging was debugging, which is particularly important when first learning them and experimenting with how to properly use their not always well documented APIs. Compared with server-centric frameworks, even ones based on dynamic languages like Ruby, there are far fewer clues when something goes wrong.

When the client lives in the browser, your choice of browser and accompanying tools/add-ons becomes important. The two real choices seem to be either Firefox with the Firebug add-on or Safari with its developer tools. As I’m developing in Linux, I use Firefox/Firebug, so my comments may not be valid for Safari

Compiled or non-compiled
One of the best and worst things about using RestfulX is the compile step. It slows down the development cycle a bit, but does catch a lot of errors. This would be a moot point if SproutCore and Cappuccino were able to report errors, but unfortunately that’s often not the case. I found that both of these frameworks repeatedly left me either with a blank page or a semi-functional interface and no console errors to help me find where things went wrong. Of the two, SproutCore seemed to provide helpful error messages more frequently, and when it did it could show the location of the error in the source, which isn’t currently possible with Objective-J

Logging and Breakpoints
I could be wrong, but from what I understand the only way to use breakpoints with RestfulX (Flex) is to use Flex Builder. Given that it costs a bit of money and I only wanted to briefly explore RestfulX, I stuck with logging, which still took a bit of effort to set up since I needed to have the developer version of Flash. SproutCore had the best free breakpoint setup since all the code is JavaScript and you use the breakpointer in Firebug as you would with any other JavaScript code. It’s also supposed to be possible to hard code breakpoints into your code in Cappuccino and then step through the Objective-J that’s been turned into JavaScript, but I ended up sticking with just plain old logging for Cappuccino as well.

Overall, given that I’m using Firefox and Firebug I found the SproutCore and RestfulX debugging processes to be equal in difficulty, but for different reasons, and Cappuccino slightly harder than either.

Cappuccino (Client side web application frameworks: part 3)

Cappuccino is the third framework I’m including in my project to implement a trivial time tracking application in multiple client-side/rich frameworks. Cappuccino was created by the guys over at 280 North, and originally showcased by their 280 Slides application. Like the SproutCore framework I recently explored, Cappuccino has its roots in Cocoa, but while SproutCore just draws on Cocoa for inspiration, Cappuccino attempts to reproduce a subset of the Cocoa API on the web and uses the Objective-J language that is roughly Objective-C built on JavaScript.

Since Cappuccino and SproutCore are so similar, it’s perhaps worth looking at some notable similarities and differences that I ran across when implementing the trivial timer application in each:

  • Cappuccino strives to let you use a single Objective-J API to implement applications, whereas in SproutCore you need to drop down to the standard HTML/CSS/JavaScript combo when you need to implement a new type of view ,for instance.
  • Both allow you to “compile” and deploy an application as a set of static files that can be served by any starndard web server (although you’ll usually want some sort of backend too), but in development SproutCore requires running a Ruby web server, whereas Cappuccino can run entirely in the browser even in development.
  • SproutCore provides a model abstraction layer that manages much of the interaction with REST-like backends. In Cappuccino there isn’t a model abstraction to handle this so each call to the server is handled manually.

Without any experience using Cocoa/Objective-C I didn’t have the immediate familiarity that is no doubt very enticing to developers coming from that background, however the Objective-J syntax was relatively easy to get used to and the API still seems sensible even without that experience. As with the other frameworks I’ve looked at, Cappuccino is fairly young so the documentation and tutorials are sparse as compared with more mature frameworks.

Now that I have three frameworks (Cappuccino, SproutCore and RestfulX) under my belt, it would be interesting to write more on specific aspects of developing with these frameworks, but I’ll save that for later. For now I’ll simply post a link to the source code, which tells all:

Ubuntu 9.10 on the new Vostro 1220

You can never be completely sure of how well a Linux distribution will work on a laptop until you actually give it a whirl. I just got a new (well, refurbished) Dell Vostro 1220, crossed my fingers as I installed the latest Karmic Koala (9.10) Ubuntu release, and was extremely pleased with the results. So far I’ve verified that all the following work without any tweaks:

  • Hibernate and suspend
  • T-Mobile 3G modem
  • Sound
  • Special keys for volume up/down, brightness up/down, sleep and the audio player play/pause, stop, and forward/reverse

The wifi didn’t immediately work, but after plugging in to a wired network and doing all available Ubuntu updates, two options for wifi drivers became available. Based on what I read in a couple of forums I went with the proprietary driver from the vendor (Broadcom STA) that was reputed to work a bit better.

Performance is great with ext4—I’m seeing about 15 second to boot up. Battery life is roughly 3-4 hours with the heavy flogging I inflict upon it. Looks like this will be a great laptop for the long bus commute.

SproutCore (Client side web application frameworks: part 2)

SproutCore is the second framework I’ve used in my project to implement a trivial time tracking application in multiple client-side/rich frameworks. SproutCore, the framework behind Apple’s MobileMe, is one of the heavyweight contenders in the desktop-like web application arena. Originally released last year in version 0.9.x form, it has undergone major changes over the past year+ to the currently near-release 1.0 version. Thankfully my experience with SproutCore started as it was becoming more stable towards the 1.0 release, although the documentation still hadn’t all caught up to the 1.0 changes (but has since been massively updated and is in good shape now). SproutCore is a Cocoa-inspired framework, but is implemented in JavaScript so it makes use of some of the good aspects of the language. JavaScript as the language choice comes in contrast to RestfulX which uses ActionScript 3, and Cappuccino which uses (and created) Objective-J. Since I’ve already looked at RestfulX, it’s interesting to note some similarities in the platforms. Both have:

  • Ruby tools for code generation and application compiling/building
  • Bindings to wire together the various components of applications
  • Fixtures to let you develop your app prior to switching over to using a real back end server
A big difference comes in SproutCore’s model layer and how it interacts with the back end. SproutCore provides a higher level abstraction that manages when objects are changed and need to be updated, and when they’re busy and shouldn’t be modified, for instance. This is handled via the SC.Store, which your controllers interact with directly. Underneath that is an SC.DataSource, which you have to implement. I can understand the reasoning for not having a one-size-fits-all data source, but I was surprised no one else had crafted one up for Rails yet. Implementing my data source ended up taking longer than the rest of the app, but it was a good exercise in getting to know the framework better. I’ve included a generic data source in this implementation that can interact with a stock Rails server (no tweaks to make the JSON SproutCore-friendly), and may follow up with a post on that. Other miscellaneous notes on my experience with SproutCore: Finally, here is the code: