May 09, 2006

Why Aren't My Assertions Counted?

I've finally figured out a solution to a problem that's been nagging me for two years. Sooner or later, when you build a Ruby testing framework of a certain degree of complexity, it seems to stop counting your assertions. So when you run the tests, you are incorrectly told you have no assertions. This is really annoying and makes you look like an idiot. What kind of tester doesn't put any verification in his tests? When things are still working, it looks like this:
  Finished in 10.285 seconds. 
  6 tests, 8 assertions, 0 failures, 0 errors
But then you get 0 assertions, even though you know your assertions are actually being executed. Why? (And if you look closely, you'll see they do get counted when they fail!)

The problem typically comes when you put assertions in your library modules. This isn't hard. All you have to do is put include Test::Unit::Assertions in your module or helper class and then you can include calls to assert, assert_equal and all the other assert methods. It works, except that your assertions aren't counted.

The reason why is that the method add_assertion does the counting. This is automatically called by all the assert methods. When they call the add_assertion method of TestCase, the count shows up in your test results no problem. But the add_assertion method of the Assertions module is just a no op. This is why they don't affect the assertion count. The solution is to add a functional add_assertion method back to your helper module/class that ties back to your test case.

Here's how. The first thing is to keep track of the test case that is running. We'll just put it in a global variable, which is ugly, but it works. The best place to put this is in the setup method of your test case.

  def setup
    $testcase = self
The test case is actually the object the setup method is part of, so self gives us a reference to it when it runs. You may already have other stuff in your setup method as well. That's fine; just add this too.

Now we need to make a change to your helper class/module where you did the include Test::Unit::Assertions. Add this method:

  def add_assertion
The name for what we are doing here is "delegation," except that we are doing it in a totally non-object-oriented way. We are "delegating" the call to add_assertion back to the test case where it belongs. After you've made these changes, tell your developer friends that you fixed the assertion count problem by delegating the call to add_assertion back to the test case. They'll be impressed.

But don't start bragging yet. If you run your code right now, you'll get a complaint that you are calling a private method. I've seen some clever hacks to get around this problem, but actually Ruby makes the solution really easy. Just make the method public! Add this code to your TestCase class definition:

  public :add_assertion 
Try doing that in Java! Don't put in any of the methods; just put it inside the class definition. To recap, your test case should now look something like this:
  class MyTest < Test::Unit::TestCase
    def setup
      $testcase = self
      # other stuff maybe
    public :add_assertion
    def test_something
      # your actual tests...
Posted by bret at 08:22 AM | Comments (1)