October 29, 2007

When is a Bug a Story?

People often recommend that you treat a bug as a story. On agile projects, if you find a bug, the recommendation is that you add it to the product backlog and allow the product owner to prioritize it. I think this approach is incorrect. We've found a better way to handle them.

There are actually two kinds of bugs. Some are just defects. There is a flaw in a story that was completed. The bug is an indication that the acceptance criteria for the story really weren't met. The other type of bugs occur when you have run off the edge of the completed work. The bug represents an incomplete application and the fact that there are still more stories to be done.

In the first case, the bug is a defect. In my view, there is no need to schedule it with the product owner. Simply add the bug to the iteration backlog and fix it at the next opportunity. The team already committed to complete the story and already estimated the work required. Now you've found out that you missed something, so you simply need to fix it. No need to involve the product owner.

In the second case, the bug is part of a story that hasn't been started. In my view, the right thing to do is to attach it to the un-started story. If the story isn't already written up in the backlog, then write up the story. The story can then be schedule by the product owner as usual.

Sometimes it isn't obvious which category a bug falls into. Different people on the team may have different interpretations of the acceptance criteria. It may take some discussion with in the team, and even with the product owner. But usually the team can decide which kind of bug it is. In fact, we review our bugs with the team every day so that we have a group understanding.

For a while, we were treating bugs in the second category as stories, but this lead to poor development work. For example, we had text areas in our app allowed you to embed html. The problem was that we only had a simple implementation of text fields at the moment. We had plans to support rich text in a future story. But when we got the bug report, instead of scheduling to address the problem of embedded html when we added the rich text editor, we did something quick and didn't analyze consequences. As a result our application doesn't allow rich text, but it also doesn't preserve line breaks. Going forward, we now make sure we schedule these kinds of bugs with the stories that ensure a comprehensive treatment of the issue involved.

Some people seem to think that bona fide defects can't be addressed by a development team on stories that have been completed (or so they thought) without product owner sign off. This makes no sense to me. I think the reason for this is simply a matter of accounting. They have already closed off the accounting for the story, and therefore need a new story to track their time against. To me this simply reflects a weakness in the accounting process and the fix is to fix the accounting (and allow time to be logged against closed stories) rather to change the process and waste the product owner's time.

When is a bug a story? Never. It is either a defect that you should fix right away or a sign that you still have more stories to complete. But it’s never a story in and of itself.

Posted by bret at 12:32 PM | Comments (8)

October 10, 2007

Supporting User Options

Normally our Watir-based scenario tests start up the application server to run tests against. But recently, I wanted to run them against a server that I had already started up. This wasn’t hard. I only made a few small changes in a couple of places. But then I realized that my colleagues would also like to be able to run the tests this way. The question for me was what mechanism to use to trigger this option.

There are three basic ways to specify options such as this: using environment variables, command-line options and an option file. I decided to use Brian Marick’s user-choices gem, which supports all three. These makes it easy to configure options and allow them to be set from any of these sources. Since adding support for the start_server option, I’ve been careful to provide options every time I make a temporary reconfiguration of our test suite for one purpose or another. In the past or week or so, I’ve added support for the following options:

browser_visible: true
tests_run_fast: true
turn_autocomplete_off: false
start_server: false
server_port: 3000
reuse_browser: true
teardown_fixtures_after_tests: true

This is actually what my personal options file looks like. I know that many other Watir users are looking for an easy way to configure these kinds of options, so I thought I would walk through how we are using the user-choices gem.

First we define what sources we will use for options.

class WatirOptions < UserChoices::Command
  include UserChoices

 def add_sources(builder)
    builder.add_source(EnvironmentSource, :with_prefix, "watir_")
    builder.add_source(YamlConfigFileSource, :from_complete_path, 
      File.expand_path(File.dirname(FILE) + '/../scenarios/options.yml'))
  end

This sets up the file listed above (options.yml) as a YamlConfigFileSource. Yaml is the name of the data format used; it is structured like XML, but is much easier for humans to read. It also allows these settings to be specified using environment variables. For example the watir_server_port environment variable would set the server_port option. (I did not choose to use command line options.)

Once the sources are specified, you need to specify the options themselves.

def add_choices(builder)
    builder.add_choice(:browser_visible, :type=>:boolean, :default=>true)
    builder.add_choice(:tests_run_fast, :type=>:boolean, :default=>true)
    builder.add_choice(:start_server, :type=>:boolean, :default=>true)
    builder.add_choice(:turn_autocomplete_off, :type=>:boolean, :default=>false)
    builder.add_choice(:server_port, :type=>:integer, :default=>3099)
    builder.add_choice(:teardown_fixtures_after_tests, :type=>:boolean, :default=>true)
end

Now you need to make the options actually do something. This really depends on the specific option. We use two different approaches. One is to trigger the option in the execute method.

def execute
    $HIDE_IE = ! @user_choices[:browser_visible]
    $FAST_SPEED = @user_choices[:tests_run_fast]
    if @user_choices[:turn_autocomplete_off]
        Registry::autocomplete 'no' # when set to 'no', sets forms, passwords and autocomplete off in IE
        Registry::intelliform '0' # when set to '0', turns off autocomplete dialog box
    end
    @user_choices
end

The execute method has access to the @user_choices hash containing the options that are configured. In some cases, this method directly actives the options. This works well when the option affects a global variable (e.g. $HIDE_IE) or a method call (e.g. Registry::autocomplete).

In other cases, the option needs to modify how an existing method works. To support these cases, I made the execute method return the @user_choices hash.

module WatirHelper  
  @@options = WatirOptions.new.execute

And then I saved the hash in the module variable @@options. (You will need to make sure that you explicitly call your execute method somewhere.) This allows me to reference options in other methods defined in the module.

  def start_webserver_if_appropriate
    if @@options[:start_server]
      start_webserver unless $webserver  
    end
  end

For example, the start_server option is checked to determine whether a server should be started.

I really like have a simple options mechanism. One reason is because I am often thinking of ways that I want to change the default behavior of our test harness. Since all of our developers use our test harness, I am concerned that they may not like a change that I want to make. With user choices, I can add the behavior as an option, set up my own option file to set it up as a default for me, and then tell them about it. With time, they may also choose to use it, and if so, I may make it the default. But this can now be a team decision that can be made when the team has had a chance to try it both ways and decide what each of them likes better.

Posted by bret at 05:45 PM | Comments (3)