December 04, 2008

AWTA Workshop will focus on Watir in January

In January, Charley Baker, Pete Dignan and I will be hosting the eighth Austin Workshop on Test Automation. We’ve explored some aspects of Watir at past workshops, but this will be the first dedicated to it. Specifically, we seek to understand the role that Watir is playing in organizations.

This aspect interests me because I’ve observed for many years that test automation efforts struggle with the traditional testing/development relationship. The traditional approach has been for testers to develop and maintain and run the tests on their own (or with a supporting automation person or team). I’ve observed that automation provides more value to the team and is easier to develop when developers are involved. They get value from being able to run the tests themselves or as part of an automated build, and they are motivated to improve testability and reduce maintenance costs. Maximizing team value is key because it helps ensure that the automation gets the time and effort it needs to be successful. But making this work has required challenging the traditional black-box relationship between testers and developers by encouraging closer collaboration.

Before I’d thrown my hat in the open-source arena, I’d been critical of commercial testing tools because they were ill suited for this kind of tester/developer collaboration. Their licensing structure discouraged sharing tests with everyone on the team, and their crap languages were an insultto any self-respecting CS grad. I spent a couple of years sharing this critique, but with limited impact—the languages became slightly less crappy, but the licensing remained anti-team.

Since words were making much impact, it was time for a demonstration. I worked on the precursor to Watir primarily to help people understand, in detail, what a better approach to automation looked like. Even then I assumed that the commercial tool vendors would learn from this — all my work was public — and we’d soon see better tools. Instead, Watir has become one of the leading open-source testing tools and testers all over the world are using it successfully. What started as project for advocacy and education has surprised me by turning into a something that people use everyday to accomplish production testing.

I was a QA Partner/SilkTest expert for most of the 90’s. During that time, I found the need to build my own frameworks with the tool. Experts with other tools came to the same conclusions. We were all building frameworks: data-driven frameworks, keyword frameworks, and even nascent domain specific languages. But the languages that came with the tools were ill-suited for framework development, which made it difficult. Because the tools came pre-wired with their own frameworks that made overly simplistic assumptions about our testing needs. The tool vendors thought limited languages were acceptable because they didn’t think you’d need to write much code.

Knowing that effective frameworks needed to be designed and customized for each testing context, I focussed on developing Watir simply as a browser driver—a library for automating browser actions. Ruby was a great language for developing frameworks, and many of us started off by using Test::Unit as the core of our functional testing frameworks. On the other hand, many of us were also using Watir for concurrency/load testing or for spidering a site and validating links and for this Test::Unit wouldn’t do. So instead we built frameworks with different structures. As such, Watir has been less of a “testing tool” and more of a kit for building a testing tool. And our users have had to assemble their own frameworks.

This has been good, because it has allowed us to develop experience in building testing frameworks, but it also has limited Watir’s reach. New users quickly face a number of momentous decisions. Should they use use Test::Unit or Rspec or something else? How do they initialize the browser and where should this code be put? Should they use global variables? We often see such questions on the Watir General list . The community is always very helpful, and many have shared the frameworks they’ve built, but still I’m sure these daunting issues steer many potential users away.

When I first worked on Watir, I was an independent consultant. Working on an open-source project was good business. It gave me something to talk about and demo, it provided a context to collaborate and learn from others, and it provided the basis for my training classes. Previously, I taught a lecture-based class on test automation. Watir allowed me to teach automation concepts hands-on and this was much more effective. More recently, I’ve worked as a QA manager and test architect for software companies and as such I’d been less active on the Watir project, mostly focussing on the features I needed to test my companies’ products.

My involvement with Watir increased when Pete Dignan and I founded WatirCraft earlier this year. Our goal is to provide commercial support for Watir. When we looked at the commercial opportunities, we realized that two things needed to be done first. First, we needed better support for cross-browser testing. This is why we developed Watir 1.6, with its improved FireWatir integration. And the second thing we needed was a standard framework. The way things are today, it would be difficult to develop a line of add-on products or services because every one is using a different framework.

In other words, the Watir community needs a standard framework to grow, and WatirCraft needs a standard framework to be profitable. So I am developing a standard framework for Watir. I would like to share my work in progress at AWTA and learn from others about the frameworks they've built and the important organization needs that frameworks must satisfy.

We already have a lot of great people planning to join us at AWTA, but we still have a few slots open. We ask people planning to attend to write a letter of introduction. This is mine.

Posted by bret at 02:59 PM | Comments (4)