I’ve been doing Android development for three years now, and I’ve only used Robotium and JUnit for testing my apps. Recently, a friend of mine introduced me to Calabash, an acceptance testing framework for mobile apps. Calabash is based from Cucumber, a ruby BDD framework.

I’ve played around with Ruby before, but I haven’t really used it for a real project because I’m primarily a Java and Python guy. This weekend, I decided to try Calabash to get some Ruby chops. I rolled up my sleeves and read about the differences of ruby from python. I nodded along the way, but I needed something more to help me understand ruby. This talk did the job. Now onto Calabash!

I have some BDD experience with AngularJS, but Cucumber’s way of doing things looks like magic to me. How is it possible to describe tests in a human readable language? Calabash docs assume familiarity with Cucumber, so I started with that first. Thankfully, Cucumber has a nice documentation about the Gherkin syntax, the language being used to define Cucumber tests, or features in Cucumber speak. An example of a feature looks like this:

1
2
3
4
5
6
7

Feature: Ads                              # This is the name of the feature.

  Scenario: As a user I can click on ads  # And this is the scenario we want to test
    Given that I am at the some screen    # A precondition step. Cucumber starts test execution from here
    And I can see an ad at the screen     # Another step which is a continuation of the previous step
    When I press the ad                   # A step which performs the first user interaction
    Then I should be taken to the browser # Verification step. I think about this as Robotium's assert()

A feature file is parsed by Cucumber, which in turn executes each steps. The final step asserts whether the test passes or fails. At this point, it still looks like magic to me, until I learned that I have to define each step from the feature file in a separate Ruby file. While Calabash has a good set of canned steps, you’ll soon find that you’ll have to define some of the steps on your own if you decide to do anything serious with it. I’m not yet good at Ruby, and I don’t have someone with me to help me out. Luckily, there’s the Calabash shell which allows me to try out the Ruby API interactively.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

$ calabash-android console path/to/your/android.apk

irb(main):001:0> start_test_server_in_background
irb(main):002:0> query("webview")
[
    [0] {
        "contentDescription" => nil,
               "description" => "android.webkit.WebView{a7240998 VFEDHVCL ......ID 0,1012-1080,1822 #7f050010 app:id/wv_ads}",
                        "id" => "wv_ads",
                     "class" => "android.webkit.WebView",
                   "enabled" => true,
                      "rect" => {
               "width" => 1080,
            "center_x" => 540.0,
            "center_y" => 1492.0,
              "height" => 810,
                   "x" => 0,
                   "y" => 1087
        }
    }
]

Now, let’s get back to defining the steps for our feature. As an example, let’s define the step “an ad is at the bottom of the screen”:

1
2
3
4
5
6
7

# This tells Calabash to wait for 5 seconds before asserting that an ad exists
Given(/^an ad is at the bottom of the screen$/) do
  ads_container = "webview css:'#ads_container'"
  fail_msg = "No WebView for ads"

  wait_for_elements_exist([ads_container], :timeout => 5) or fail(msg=fail_msg)
end

After fleshing out the rest of the steps, execute calabash-android run path/to/your/android.apk and watch the magic begin!

As a newcomer to Ruby, Cucumber, and Calabash, I’m quite surprised at how easy it is to get started at using the whole suite. Granted, there’s a lot of reading involved, but overall it was a smooth process. It took me less than a day to get stuff running. Another thing that I like about Calabash is its interactive nature. Since it is written in Ruby, I can test the steps I’m planning to write in irb prior to fleshing it out and see immediate results. Compare this to Robotium, where you’ll have to recompile each time you make a change to your tests. Generally, I’m a fan of interpreted languages because of the smoother workflow compared to, say, Java. As an Android developer, Calabash is my gateway drug to the Ruby world.