cookies and code

Over the years I’ve been using Selenium as my tool of choice for UI testing on the web. Recently, we’ve had the need to regression test a vendor web application due to some poor quality builds being produced. Based on past experience, it takes us about 40 hours of work to regression test the product UI by hand, including double checking for annoying copy changes that always seem to slip through. Today we finished the first chunk of a baseline set of tests that accounts for roughly 30% of our test plans. It took us about 150% of the time needed to do one regression test to get this first chunk done. Since we regression test regularly, we’re going to incrementally add more tests each time, getting into a habit of updating our Selenium test suites with the latest changes per build. With that said, let me just tell you this is probably one of the most detailed applications I’ve had to test with Selenium. Guess how long it takes to run our tests that don’t require some type of manual setup … less than 5 minutes. So we’ve taken a third of our tests that took about 10 hours to execute and reduced them to 5 minutes of watching the browser go nuts with all the typing and clicking. There is still work to be done manually that we will never escape (global system setup), but talk about saving time. No matter how much I use Selenium I never get tired of the benefit it provides in the long run.

So all though these stats are encouraging for management to hear, but what I really like about Selenium is how simple it is to create test suites. Don’t get me wrong though, there a few things that can cause hang-ups. Hacking out XPath statements is one that I face quite frequently. Consider a command in Selenium called verifySelectLabel which takes a locator (typically an XPath expression) to find the select box in the DOM and a pattern (typically a simplified regex) to match the text of the option currently selected in the select box. The Selenium IDE has a “Find…” button to help validate the XPath statement and some context menu options in the browser to help, but in most cases I always find myself messing around with the XPath statements. Recently, while playing around with Firebug (my favorite JavaScript swiss army knife) I noticed that under the HTML tab when right-clicking on a tag you get the option Copy XPath. Selenium’s XPath support is really robust and the Firebug’s syntax ports over flawlessly. I did notice that while running in IE I had to simplify the statements sometimes to get them to pass, but otherwise, it’s cut my recording time down considerably.

Another thing that always sucks up my time is when I have to hack together JavaScript to get data ready for a test (especially when it comes to dates). The storeEval command will execute a JavaScript snippet and take the result of the last line and put it into JS variable for later use. In the past I’d fire up a kind-of scribble pad HTML doc with some inline script and test my code, but this time around Firebug is all I needed. The Console tab gave me my own scribble pad with the added help of the Firebug console object. Another cool JavaScript helper I hadn’t used before was the user-extensions.js file, which is imported into the test runner giving you access to whatever JS you want to re-use. I found a pretty descent date API that I copy/pasted into this file making working with dates in my tests stupid simple. Finally, this is the first time I’ve tested a product that I didn’t help write and that was written using ASP.NET Web Forms. With the help of debug(), monitor(), and the Inspect tool in Firebug, I was able to find my way around the mess of scripts on the page very quickly. Even though the generated code wasn’t my favorite, I have to give credit to ASP.NET’s management of inputs; each input was generate with a unique ID which made testing incredibly easy.

Overall, it was great to get back into using Selenium and I’m really excited at the possibilities of starting to use Selenium Remote Control and Grid. The prospect of getting my unit testing framework (NUnit) into the picture will definitely help to complete and simplify the build process. What’s even cooler is that once we’ve put all the work into recording our tests in HTML, the Selenium IDE will translate them into C# so we don’t have to repeat ourselves. With new extensions for working with Flex, this tool’s potential continues to grow. If you’ve never tried out Selenium, definitely give it a shot. The barrier to entry is low and you can choose the level of complexity you want to bring to the table.

Tags: flex, javascript, regression, selenium, testing, user-interface, web

Related posts

• ActionScript: Just different enough to piss you off
• Which assertions do you use in unit tests?
• Using Stubs for HTTPService and RemoteObject in Flex
• Uses for non-visual Flex components
• Testing Tools for Flex