Ryan's Blog

Ruby Unit Test Framework

I have been writting quite a few unit tests in Ruby using the WATIR
(Web App Testing in Ruby) framework. On the whole I like WATIR and
using the Ruby unit test framework, however I did run into some odd
behavior. The WATIR framework is basically a set of libraries for
Ruby that provide hooks into Internet Exporer. Through these hooks
you can do things like go to a web page, insert text into a field,
click a button etc. WATIR in and of itself doesn't test anything.
WATIR just puts text somewhere on a page, visits a link etc. For
the actual testing part, a unit test framework is used. I just used
the unit testing framework that comes standard with Ruby. This is
very similar to the rest of the unit test frameworks in the xUnit
family, they use similar terminology, concepts etc. So there's a
TestCase class that contains the tests (having method names prefaced
with "test"), a TestSuite class that can contain multiple TestCases
etc.

Refactoring Commonality

In writing a lot of these WATIR and based tests, I found that there
were quite a few common things. Things like insert a certain
string into a certain text field, click a particular link etc.
Knowing this, I created lots of small methods with common actions.
The methods were pretty small an looked something like:


def assert_text_field_blank(field_name)
  assert(text_field(field_name).getContents.length == 0)
end

def set_text_field(id, value_to_set)
  text_field(id).set(value_to_set)
end


I approached this how I would in Java and created a common
superclass. So I had a BaseTest class that extended
Test::Unit::TestCase and then all my test classes extended that
BaseTest class.

Getting in the Ruby Mindset

Originally I set up a TestSuite with the 10 or 15 TestCases that I
wanted to run. While running in the TestSuite, all of my TestCases
worked properly, everything passed. However, as I started to add
more tests, I quickly ran out of patience running all of the tests
all of the time. I just wanted to run one TestCase. What I was
surprised to find was that when I ran my tests individually, I
encountered an error that was not there when I ran the test as a
suite. As a minimal example of this problem, below is the code to
recreate the problem:

require 'test/unit'

class RealTest < BaseTest
  def test_my_test
    common_method
    assert(true)
  end
end


Running the above code will result in the following error:

default_test(BaseTest) [c:/dev/TestClasses.rb:10]:
No tests were specified.

After getting this error, I started to look into how these tests
were being ran. The above code doesn't contain anything that
references a TestRunner, or anything else to run the test.
Normally, if I were creating a TestSuite, I'd have code that
looked like:

Test::Unit::UI::Console::TestRunner.run(MyTestSuite)


Since the above code did not have a test runner specified, Ruby will
create it's own, including in it the current TestCase. When I figured
out that this was happening, I started looking into the Ruby source code to
figure out how it was determining which TestCases to run.

Object Space

In looking into how it determines which TestCase(s) to include in
the auto generated TestSuite, I found that it was referencing an
ObjectSpace class. I hadn't seen this before and upon looking into
it further, I found that ObjectSpace is a reference to all of the
objects in the running Ruby system. So in my case, the unit
testing code was iterating through all of the objects in the
running system, attempting to determine which ones were subclasses
of TestCase. If the class was a subclass of TestCase, it was added
to a TestSuite to be executed later. In most cases this is ideal
an saves code by not having to added the TestRunner code. In my
case it had some undesired behavior. As it was iterating through
all of the objects in the running system, it found an instance of
the BaseTest class and an instance of the RealTest class. Both
were subclasses of TestCase and both were included in the TestSuite
to be executed. Now in looking at BaseTest, it's just used to
house common functionality, but the Ruby code simply looks for all
subclasses of TestCase (which both tests are) and then adds them to
the test suite, not taking into account subclasses of classes that
extend TestCase. Then when it attempts to execute the BaseTest
tests, there are no tests found and so the test fails.

Solution to the Problem

I fixed this problem by changing the BaseTest class. Rather than
having BaseTest subclass the TestCase class, I instead added the
methods to the TestCase class. In Ruby, it's possible to redefine
and add methods to existing classes in the system. In my example,
I just added the common methods I wrote to the TestCase class and
then had RealTest just extend the TestCase class. The code looks
like:

require 'test/unit'

class RealTest < Test::Unit::TestCase
  def test_my_test
    common_method
    assert(true)
  end
end


What is happening here is that our BaseTest class has now changed
to just add a method to the TestCase class in the Test::Unit
library. Now, whenever this Ruby file is loaded, the TestCase
class will incorporate the new methods that I have added and make
them available to the rest of the system. This approach allows me
to use all of the common methods that I created and also allows the
execution of the tests individually without having to add the test
runner code to each of the test cases.

Posted at 09:44AM Aug 14, 2007 by Ryan Senior in Programming  |  Comments[2]