Both of us will probably continue to be involved with the Watir. Prototest has projects that are using Watir and the WatirCraft framework, and Convio has recently standardized on using Watir internally.
WatirCraft funded my work with Watir last year. There is certainly more work that needs to be done, but it will happen, as it had before, as a result of volunteer labor. I'm not sure how much time I will have for this going forward, but we have a lot of contributors and volunteers, so I have confidence that Watir will continue as an active project regardless of my own involvement.
The watircraft framework is a bit more up in the air. The original idea was to build a pretty good framework as open-source and then offer additional products and services that built on that. A lot of people like what we've developed so far, but it clearly could use more work. I'm not sure what is actually going to happen. I think its a good start and would certainly use it as a basis for any Watir test suites that I develop personally (e.g. at Convio). Convio also has been mostly using the Rasta framework, so I am looking at integrating that with the watircraft framework. Needless to say, the watircraft framework is open-source and hosted on github, so anyone is free to fork it and add features.
We set up Watirbuild.com to host continuous integration of the Watir project. This was originally set up by WatirCraft, but i'm now paying for the hosting of this site personally, as part of my personal commitment to the Watir project.
WatirCraft also developed Watir-based training. Over the past couple of months, I've taught this a couple of times, and have had good feedback from it. Both Pete and I have retained the rights to deliver training based on these materials, but I don't have any immediate plans, nor as far as I'm aware does Pete.
I'm trying to take a break from Watir right now -- from development of the driver and frameworks, from training and consulting. This is hard because there is so much momentum, and it pains me to risk losing that. We're ready to make a new release of Watir shortly. I have a roadmap for the watircraft framework based on all the feedback we've gotten. And I have leads for consulting. But I can only do so much, and I fear that too much of what I have been doing has been done out of inertia or a sense of obligation.
What I really want to do right now is write about Watir: why it has been important to me, the vision I've had for it, and the challenges that lie ahead. Stay tuned. ]]>
Founders at Work, by Jessica Livingston. Fascinating stories of the early days of technology startups. Based on interviews, there are stories both new companies and old. In several accounts, inside information is revealed about startups you've already read about. Learn about how Jobs ripped off Wozniak in Apple's early days, and how Wozniak feels about it now.
I strongly recommend these books for any one involved in a software startup. ]]>
There are three essential elements of an automated testing system: language, driver and harness. Your tests need to be written in a language and you need a language to extend your testing system. I've long believed in scripting languages, such as Perl, Python and Ruby. I find that testers are more productive with them and find them easier to understand and use. I have reasons why I prefer Ruby, but there are other good automated testing systems written using other scripting languages. In fact over the years, I have built testing frameworks in Perl, Python and VB, usually building on languages that were already in use at the client. My emphasis on full-featured programming languages was a contrast to the proprietary languages that had been commonly used in commercial testing tool suites. I had no patience for them.
There are lots of languages to choose from. Finding a suitable driver, however, was often more difficult. A driver is what you need to drive your application. Watir is an example of a browser driver, suitable for web applications. Often the driver determines the language you use. Years ago, I used Expect as a driver for a command line application. Expect used TCL as its language, so the suite was written in TCL. I developed Watir because I wanted a browser driver in Ruby. Watir was our solution for Web Application Testing in Ruby.
The third thing you needed for a testing system was a harness. When we started, we all used Test::Unit, a Ruby test harness designed for unit testing. We found that it could be adapted for functional testing. More recently, Watir users have been using Rspec or Cucumber as their test harness. A test harness is responsible for executing tests and collecting and reporting results. Some people like to build their own test harnesses instead of using one of these.
I developed this vision while working with commercial testing tool suites like SilkTest and WinRunner. These packages are often referred to as tools, but I found that they actually comprised a collection of integrated tools. They had predefined usage expectations that often didn't match actual tester needs. Data-driven testing, for example, required us to break down the suite and then reassemble it in a more suitable configuration. I wanted testers to be able to identify the individual tools that made up a testing system.
Many new Watir users have trouble identifying where Watir (the driver) ends and Ruby (the language) begins. But learning this is key to becoming a proficient user. I'm told that some Java teachers don't like their students to use an IDE (such as Eclipse or Netbeans). Instead they want their students to learn to use the required tools separately -- the editor, the compiler. This way they learn the function of each tool. In the same way, I've also wanted testers to understand the functions of the different tools and components used in a testing system.
We often get questions from new Watir users whose answers are obvious once you understand this. Can Watir read CSV files? Can it do date math? Can it read from a database? The answer to all of these questions is that Watir can't do it, but Ruby can. No browser driver could do these things (it would be like asking whether your compiler supports search and replace), but any full-featured programming language can. It's just a matter of finding the library to do it. These questions are raised in the first place because testers are used to using commercial tool suites that were closed systems. So you only got the ability to do these kinds of things if the vendor added it to the package. Watir is part of an open system, so all the libraries are there already and more are being written all the time, often by people who are not part of the Watir community, per se, but rather part of the larger Ruby community.
Want to learn more? I am teaching classes on Watir, Ruby and the new WatirCraft framework that supports both Rspec and Cucumber.]]>
The "Depot" is part of the framework-examples project. The project also contains lots of examples of how to write Watir tests for it using different frameworks, including Rspec, Cucumber, Rasta and Roo (in a directory called Arc). And the WatirCraft framework. Indeed, we used this project to drive the development of the WatirCraft framework. We created examples using several different frameworks and then pulled the best of each into the WatirCraft framework. We stole the most from the Taza framework. Our WatirCraft example actually started as a Taza example (which is why there is no longer a "Taza" example in the project -- it morphed). Right now, in fact, this example amounts to the best documentation we have for the WatirCraft framework.
But any one looking to learn about any of these testing frameworks will probably find this to be pretty helpful.]]>
If you want to see it, you can look at a small page that I've put together about it for our customers. You can also take a look at the code . It is based on the Taza project , which Charley Baker leads. This framework is extracted from their projects at the Gap. (He works for the division that handles their websites, which are doing better than their stores.) I've also built several application-specific frameworks for previous employers am incorporating ideas from them into WatirCraft.
It seems early because most of my vision for this is still in my head and not yet in the code. And because I'm at a point where it feels easier to develop the features than to describe them. So talking about it, documenting it, writing about all seem like ways to delay creating it. But a little more feedback would be good, especially in a few more weeks and, of course, it will take time for people to try it out before they can provide feedback.
We also are looking for sponsors, people who would be willing to pay us to continue development on it. And I'm teaching classes next month (in Denver and Austin) to people who want to learn first hand about it.
It seems early, because I still don't have a proper homepage for this project, and I haven't really shared my vision. But, today, it's time for me to get back to the code. If you would like to sponsor this work, please contact me or Pete, my business partner in Denver. Or sign up for one of my classes. Thanks. ]]>
Since then, I've seen participants twittering more. There has also been a marked rise in discussion on the #watir IRC channel (although it was quiet during the workshop itself), as well as the watir mailing lists.
I'm really happy with the workshop. A lot of people helped in a lot of different ways. It's helped me frame my thoughts about frameworks: what people are doing, how the different frameworks can share more components, what people really need. I also got a chance to meet with many key contributors to the Watir project. The feedback overall has been very positive.
Next: Watir Frameworks]]>
This aspect interests me because I’ve observed for many years that test automation efforts struggle with the traditional testing/development relationship. The traditional approach has been for testers to develop and maintain and run the tests on their own (or with a supporting automation person or team). I’ve observed that automation provides more value to the team and is easier to develop when developers are involved. They get value from being able to run the tests themselves or as part of an automated build, and they are motivated to improve testability and reduce maintenance costs. Maximizing team value is key because it helps ensure that the automation gets the time and effort it needs to be successful. But making this work has required challenging the traditional black-box relationship between testers and developers by encouraging closer collaboration.
Before I’d thrown my hat in the open-source arena, I’d been critical of commercial testing tools because they were ill suited for this kind of tester/developer collaboration. Their licensing structure discouraged sharing tests with everyone on the team, and their crap languages were an insultto any self-respecting CS grad. I spent a couple of years sharing this critique, but with limited impact—the languages became slightly less crappy, but the licensing remained anti-team.
Since words were making much impact, it was time for a demonstration. I worked on the precursor to Watir primarily to help people understand, in detail, what a better approach to automation looked like. Even then I assumed that the commercial tool vendors would learn from this — all my work was public — and we’d soon see better tools. Instead, Watir has become one of the leading open-source testing tools and testers all over the world are using it successfully. What started as project for advocacy and education has surprised me by turning into a something that people use everyday to accomplish production testing.
I was a QA Partner/SilkTest expert for most of the 90’s. During that time, I found the need to build my own frameworks with the tool. Experts with other tools came to the same conclusions. We were all building frameworks: data-driven frameworks, keyword frameworks, and even nascent domain specific languages. But the languages that came with the tools were ill-suited for framework development, which made it difficult. Because the tools came pre-wired with their own frameworks that made overly simplistic assumptions about our testing needs. The tool vendors thought limited languages were acceptable because they didn’t think you’d need to write much code.
Knowing that effective frameworks needed to be designed and customized for each testing context, I focussed on developing Watir simply as a browser driver—a library for automating browser actions. Ruby was a great language for developing frameworks, and many of us started off by using Test::Unit as the core of our functional testing frameworks. On the other hand, many of us were also using Watir for concurrency/load testing or for spidering a site and validating links and for this Test::Unit wouldn’t do. So instead we built frameworks with different structures. As such, Watir has been less of a “testing tool” and more of a kit for building a testing tool. And our users have had to assemble their own frameworks.
This has been good, because it has allowed us to develop experience in building testing frameworks, but it also has limited Watir’s reach. New users quickly face a number of momentous decisions. Should they use use Test::Unit or Rspec or something else? How do they initialize the browser and where should this code be put? Should they use global variables? We often see such questions on the Watir General list . The community is always very helpful, and many have shared the frameworks they’ve built, but still I’m sure these daunting issues steer many potential users away.
When I first worked on Watir, I was an independent consultant. Working on an open-source project was good business. It gave me something to talk about and demo, it provided a context to collaborate and learn from others, and it provided the basis for my training classes. Previously, I taught a lecture-based class on test automation. Watir allowed me to teach automation concepts hands-on and this was much more effective. More recently, I’ve worked as a QA manager and test architect for software companies and as such I’d been less active on the Watir project, mostly focussing on the features I needed to test my companies’ products.
My involvement with Watir increased when Pete Dignan and I founded WatirCraft earlier this year. Our goal is to provide commercial support for Watir. When we looked at the commercial opportunities, we realized that two things needed to be done first. First, we needed better support for cross-browser testing. This is why we developed Watir 1.6, with its improved FireWatir integration. And the second thing we needed was a standard framework. The way things are today, it would be difficult to develop a line of add-on products or services because every one is using a different framework.
In other words, the Watir community needs a standard framework to grow, and WatirCraft needs a standard framework to be profitable. So I am developing a standard framework for Watir. I would like to share my work in progress at AWTA and learn from others about the frameworks they've built and the important organization needs that frameworks must satisfy.
We already have a lot of great people planning to join us at AWTA, but we still have a few slots open. We ask people planning to attend to write a letter of introduction. This is mine.
]]>We've also provided other features and fixes that will be of interest to users who don't care about Firefox (if such people exist). It is backwards compatabible, so we expect all Watir users will be upgrading to this release. Install or upgrade to this release.
And the end of this note, we mention a few issues that people have encountered when upgrading to this version.
We've fixed many of the reported compatibility issues. FireWatir now supports nearly all off of the elements supported by Watir. Also both implementations now support being able to specify page elements using multiple attributes (e.g. browser.button(:name => 'Purchase", :index => 2).click). Lots of other issues, detailed in the release notes, have also been fixed.
Both Watir and FireWatir users can simply upgrade and continue to run their existing tests. But they can also make use of our new generic browser interface.
require 'watir' browser = Watir::Browser.newOnce you have changed your tests to use this instead of Watir::IE.new, then you can configure which browser will be used at runtime.
We have updated Watir's modal dialog support to work with Ruby 1.8.6. Previously, users who needed to work with the showModalDialog had to use Ruby 1.8.2, but this restriction is no longer in place. We now recommend that all Watir users use the latest version of Ruby 1.8.6.
We made some speed improvements in early version of Watir 1.5 that stopped working with version 1.5.4 - 1.5.6. These speed improvements are now working again. Users who had not upgraded to 1.5.6 because of performance concerns should upgrade to this release of Watir.
We've also added unit tests that will alert us if we happen to break this functionality in the future. Sorry about that.
I mentioned this above, but it bears repeating. Previously, Watir allowed you to access some page elements using multiple attributes, but you could only use a single attribute to access input elements such as buttons or text fields or select lists. Now you can use multiple attributes to access any of these elements. There is no limit to the number of attributes you can use. In other words, multiple attribute support has been improved for both IE and Firefox
For some time, we've been telling people to avoid using "include Watir" in their scripts because it had the potential to cause unforeseeable problems. With this release, it causes conflicts with one of the other libraries that Watir uses. If you have been using "include Watir" in your scripts please read our detailed advice on this topic.
When installing Watir 1.6.2, some users will see this error.
ERROR: While generating documentation for builder-2.1.2 ... MESSAGE: Unhandled special: Special: type=17, text="<!-- HI -->" ... RDOC args: --ri --op e:/ruby/lib/ruby/gems/1.8/doc/builder-2.1.2/ri --title Builder -- Easy XML Building --main README --line-numbers --quiet lib CHANGES Ra kefile README doc/releases/builder-1.2.4.rdoc doc/releases/builder-2.0.0.rdoc do c/releases/builder-2.1.1.rdocIt is harmless and can be ignored. All it means is that the documentation for the Builder gem could not be generated. Everything else will work.
You can no longer run Watir's unit tests from within a gem. This is because we have started integrating our Watir/FireWatir test suites and they have cross dependencies. They still run fine from source. You and run them yourself. We also run them everyday and publicly post results. ]]>
After teaching Watir/Ruby scripting to dozens of testers, I've found that testers -- even those who insist they are not developers -- can learn effective programming techniques and concepts. But they need to be presented in concrete, practical contexts. Most programming manuals, on the other hand, use artificial examples assume an understanding of abstract concepts, such as exception handling or object-oriented programming. That's why they don't work for many testers.
If you are tester who knows some programming -- but not object-oriented -- and strongly prefers concrete, practical examples, then these recommendations are for you.
Testers looking to better organize their code and better understand how to use classes, modules and libraries will benefit from studying Everyday Scripting with Ruby by Brian Marick. It has a tutorial format, walking you through a series of increasingly richer examples. Previously, I said:
In Everyday Scripting with Ruby, Marick teaches the Ruby language by working through a series of practical scripts. These scripts accomplish everyday tasks that every tester can relate to: comparing file lists, monitoring code changes, scraping web pages, and using a notification framework. The book covers not only the Ruby language, including an excellent chapter on regular expressions, but also provides detailed advice on how to develop reusable libraries, something that Watir users are always asking for help with. If you are a serious Watir user, don’t hesitate to get this book.Although this book only touches on Watir explicitly for a few pages, it actually has a deep connection. Back in 2002, Marick and I started teaching a class together called "Scripting for Testers". This class was based on our mutual conviction that testers needed to learn the fundamentals of scripting. Watir was one spin off from this class. The other is this book. Watir grew from a small library we used in the class to show how to use scripting to automate a browser. And Marick poured his commitment to teaching testers scripting into this book. As such, the book doesn't just teach Ruby, but rather is really focussed on teaching good development practices, such as when to use global variables and how to do test-driven development.
Marick's book has a tutorial format suitable for serious study. Another good book for testers is the Ruby Cookbook, By Carlson and Richardson. It consists of over 300 recipes, each only a few pages, that can be read in any order. Here's a quick selection of items from the table of contents:
1.19 Validating an Email Address 3.7 Converting Between Time Zones 4.9 Sorting an Array by Frequency of Appearance 5.11 Choosing Randomly from a Weighted List 9.6 Automatically Loading Libraries as Needed 11.13 Extracting All the URLs from an HTML Document 14.5 Sending Mail]]>
At WatirCraft, we are currently looking for customers, particularly those interested in a Watir-based framework.
One of the reasons that I've been posting here less is that I've been posting on twitter and on something I call my miniblog. I've also been frequenting the #watir IRC channel.
]]>The reason Agile teams can do with less planning is because feedback allows you to make sure that you are on course. If you don’t have meaningful feedback then you’re not agile. You’re just in a new form of chaos.
On my last project, we defined our stories so that they made sense to everyone on the team. Our analysts, testers and developers could all understand and review individual stories. But we found that we had to create a larger grouping, which we called features, to facilitate meaningful review from outside our team. We made sure all the stories in a feature were complete before soliciting feedback from outside the team.
Being able to give and receive meaningful feedback is often a challenge for people. Yet it is crucial to success with Agile.
Agile teams get into terrible binds when executives or clients hand them a list of requirements at the start, tell them to use Agile (because its faster), and then don’t want to participate in the feedback process.
Agile isn’t faster all by itself. Agile is only a benefit in a world that acknowledges the value of adapting. And that adaptability needs to go all the way to whoever is funding the project. It is not enough for the team to be Agile. The sponsors need to be Agile too. Are all of the requirements really required? Do we know exactly what the software needs to look like from the start?
Agile is faster because feedback allows you to find and focus on the most valuable features. If you are certain you know what needs to be built, don’t use Agile. If you don’t have time to gather and act on feedback from customers, then don’t use Agile. If you are sure that everyone understands exactly what needs to be done from the start, then don’t use Agile.
Agile practices build a technical and organizational infrastructure to facilitate getting and acting on feedback. If you aren’t going to adapt to feedback, then this infrastracture is waste that will only slow you down. Previous: Lessons Learned in Agile Development Next: Agile Mostly Means Something Between Scrum and XP ]]>
To install:
We are using many Agile methods in the development of Watir. We are using such methods as continuous integration and emergent architecture in our development. Many Watir users are curious about our development, but unfamiliar with some of our methods. I want them to understand.
Many Watir users and other testers are trying to understand Agile. Many are on teams that have recently begun using Agile methods, and are wondering whether it works. It does, but there are also lots of ways to fail. Testers, always quick to find faults, have a hard time with much of the Agile literature, written as it is with developers or managers in mind. I’m always thinking about the testers.
WatirCraft is also using Agile methods. Explaining Agile gives me a chance to explain to the current and future WatirCraft team some of the rationale behind our methods.
I began studying Agile testing six years ago and for the past four years have worked exclusively with talented, high-performing Agile teams. In the past few years, I’ve also been coaching my collegues on Agile methods in areas beyond testing. In that time, I’ve compiled quite a pile of notes on the topic, including key ideas and common misunderstandings. I’m happy to have this chance to share them.
]]>Invariably, email discussions leave someone important out, and can be difficult to add people to the discussion later. Internal blogs don't have this problem: any one in the company who finds out about the discussion can join in, sometimes even months later. I also find myself reluctant to send out emails because they can be so interrupting. With a blog, it seems easier to write something that is important but not urgent, and you'll get more thoughtful replies because people know that they replies will be archived.
I remember pushing the idea that QA people should be called “Testers” (and not “QA”) on my ThoughtWorks blog. We had pretty much come to agree on this point in the Context-Driven community before I joined ThoughtWorks, but I came to see that ThoughtWorks, like the industry at large, still labelled the role “QA”. Because I was then in a position of some standing (Director of Testing Practice), making an issue of this on my public blog would have been a positional move, likely polarizing both me and those who disagreed. By publishing my position internally, I essentially acknowledged that I wanted feedback from my collegues on this topic. I remember Rebecca Parsons suggested “QA/Tester,” which, though awkward, didn't run into the various objections I had laid out against QA, and since then I've been more comfortable with the “QA” term, even though I still usually prefer “Testers”.
At both ThoughtWorks and Dovetail, we made our blogs private simply by publishing regular blogs on our internal networks. You either had to be in our offices or connect over the VPN to see them. Web-based feedreaders (like Google Reader) wouldn't work, but you could use any client-based feedreader (Outlook has one) when you were connected to the company network.
I looked closely at Blogger and WordPress and also more generally across the field and was not able to find any ready-to-use blog engine that supported private blogs and provided a feed. It is possible for feeds to be password protected. For example, we use Highrise and it provides a password-protected feed containing recent changes. But i could not find such a thing for any blogs.
Apparently several blog engines added support for private blogging a couple of years ago, but left public feeds on them. These were crawled by search engines and people were finding their private posts on Google! The quick fix was to remove the feeds, and apparently the issue hasn't been addressed since. One option that might work for some would be to have a blog that isn't strictly private, but is protected from search engines, so you'd have to know the URL to read it. This option is offered by WordPress.
At WatirCraft, we have not yet set up a VPN, so that is not an option for us. For now, it seems the best way to run an internal blog is to make one private and then to set up an accompanying mailing list that you post to when there are updates on the blog.
]]>The general term for this process is continuous integration—a common practice with Agile teams. I’ll write more about why this practice is important for the Watir project another day. Today’s post is about what I did to set this up.
First, I got a server, a virtual private server (VPS) running Windows from VPSLAND. I would have preferred a machine running XP, since most Watir users use that, but the closest I could get was 32-bit Windows 2003 Server. Cost: $19 a month, courtesy of WatirCraft. This VPS is named watirbuild.com.
Next I installed the Google Toolbar, because I can’t find anything without it. I also had to loosen up security on IE so that I could use it to download the software I needed. (The server VPSland provided came with security really locked down.)
I then installed Ruby 1.8.6-26. The next step was to do a “gem update --system” to update the gem installer to the latest version (which Watir needs), but this command failed with this error.
c:/ruby/1.8/yaml.rb:133: [BUG] Segmentation fault
A little research showed that this was a common problem on RAM-constrained VPS’s. Apparently the pre-update gem installer is a memory hog and the VPS’s don’t have any swap space. The common workaround has been manually download the gems before installing them. (Only the remote gem installer runs into memory problems.) But I used a different approach.
I wanted our test environment to match, as much as possible, the environments that other people are using for testing, so I really wanted to have the latest gem installer. I also knew that it had better memory handling. So what I did was install Ruby on one of my own systems with more RAM, and then did a “gem update --system” on it. Then I zipped up the resulting “ruby” directory (using 7zip), uploaded it to watirbuild.com and then unzipped it there over the original installation. I was then able to do remote gem installs without any further trouble.
Because Watir now depends on several other gems. I did a “gem install watir” and then uninstalled the Watir gem itself. This left all the dependent gems installed.
Watir uses Subversion (SVN) as its development repository, so we needed a client to access that code. I installed the command line client.
The next step was to install the continuous integration tool itself. I chose CruiseControl.rb because it is Ruby-based and easy to configure. Pre-deployment testing had shown, however, that the latest release (1.3.0) did not work on Windows (After starting the server, it would get repeated "Access is denied." errors from SVN.) Instead I needed to use the latest from its development repository. They recently migrated that project from Subversion to Git, so that meant I needed to install a Git client. Once it was installed, installing Cruise was simply a matter of doing
git clone git://rubyforge.org/cruisecontrolrb.git
With Cruise installed, I added the Watir project:
cruise add watir https://svn.openqa.org/svn/watir/trunk/watir
I also needed to add a “cruise” target to Watir’s rakefile. Which I did with the following code:
desc 'Run unit tests' task :test do load 'unittests/core_tests.rb' end task :cruise => :test
(This code is now part of the Watir project, so you wouldn't need to provide this code if you were running Watir. But you will need something similar if you are setting up your own project with CruiseControl.rb.)
Then I started Cruise using “cruise start -p 80”, and this immediately kicked off Watir’s unit tests. Some failed on first run, and I had to modify a couple of IE’s security settings to fix this.
To test everything, I checked in a minor documentation change to Watir and verified that the unit tests were automatically kicked off.
]]>