Which assertions do you use in unit tests?

Brian LeGros | February 6th, 2009 | programming  

Over the last few days, I’ve decided to start putting some time into adding some new assertion functions into fluint. Here are a few that I was thinking of adding:

assertRegExp(expected : RegExp, actual : String) : void
assertArraysEqual(expected : Array, actual : Array, compare : Function) : void
assertDictionariesEqual(expected : Dictionary, actual  : Dictionary) : void

Along with each of these, their boolean counterpart would also be available (i.e. – assertArraysNotEqual). My question to other developers is, which assertions would you like to see available in your unit testing framework of choice? The way I figure, the more options available in the assertion stockpile, no matter how simplistic they may seem, the more explicit we can make our tests. Would anyone be interested in seeing some type of assertion using callbacks (i.e. assertReturnsEquals(actualCallback : Function, args ..rest, expectedResult)? Feels like a bit of a stretch, but it’s a datatype, so maybe it should be considered. Anything for events that may be useful?

What do you think?

ActionScript: Just different enough to piss you off

Brian LeGros | October 19th, 2008 | programming  

So I’ve started my new job and I’m working with Flex and ActionScript 3. I was tasked with throwing together a couple classes and there respective unit tests this week. The classes don’t integrate with any external resources and mostly do pass through except for some cross cutting concerns (e.g. – logging, security, etc). I figured writing my tests would be pretty straight forward: throw together the appropriate test cases and in each mock out any dependencies.

Now when I think about a language I try to put it into the context of other languages I’ve had experience with, identify differences in language elements, and work from there. With respect to ActionScript, I tended to problem solve in the context of JavaScript since they are both based on ECMAScript, but I’ve run into some frustrations along the way.

To mock out an object in a test using JavaScript I’d typically write something like:

1
2
3
4
5
var dependency = new MyDependency();
dependency.methodToMock = function () { return true; };
 
var obj = new MyClass(dependency);
assertTrue(obj.passThroughMethodUsingMockThatICreated());

or if I wanted to mock out objects being instantiated by a dependency that I couldn’t inject:

1
2
3
4
MyDependency.prototype.methodToMock = function () { return true; };
 
var obj = new MyClass();
assertTrue(obj.passThroughMethodUsingMockThatIDidntCreate());

In ActionScript, assuming that MyDependecy is declared in an ActionScript class, the first and second approach I used in the above will not work. I did some reading on the Flex 3 livedocs and found out that the culprit was the traits object that was introduced in AS3. To abridge, the traits object is a copy of all applicable properties from parent classes and defined properties in the current class that need to be available at run-time to help reduce the performance hit of walking the prototype chain previously used for inheritance. The traits object is an internal mechanism which is not accessible by the developer. Prototype-base inheritance is still available, but falls secondary to traits. Awesome.

So unless I declare the methods of my class as members of the prototype for the class, I can’t manipulate them at run-time. So instead of the following:

1
2
3
4
5
public class MyDependency {
   public function methodToMock() : Boolean {
      ...
   }
}

I’d have to write:

1
2
3
4
5
public class MyDependency {
   prototype.methodToMock = function() : Boolean {
      ...
   }
}

So now I’ve got methodToMock available in a data structure I can mess with at run-time. I still, however, can’t accomplish what I did in JavaScript unless I add the keyword dynamic to the class declaration (as below) or remove the “strict” flag from the compiler:

1
2
3
4
5
public dynamic class MyDependency {
   prototype.methodToMock = function() : Boolean {
      ...
   }
}

Now I can manipulate the object and its class’ prototype the way I would in JavaScript. It irks me a bit that performance was the driving force for all of this hoopla, but I’m just an application developer, so I won’t pretend to understand all of the motivations for these decisions.

As another alternative, I thought about just using Object for all of my mocks, but I quickly dismissed the option since we’re using the explicit typing with ActionScript, as I suspect most are since its promoted for its performance boost. I also looked into using the Proxy class as a run-time solution, but alas it can’t be instantiated, only extended; this would mean writing class declarations for every mock I wanted to create, which seems kinda silly, and I still would have to deal with the explicit typing issue. On a side note, it would have been nice if Proxy had the option to be instantiated with the object it’s proxying as a constructor argument, mimic the type passed as the argument, and allow run-time manipulation. No idea if this is feasible though, so I’ll count it as a pipe dream for now. In any case, from my experience mocks are intended to be used within a context and disposable for the purpose of testing; why would I write anything more than inline code to utilize mock objects?

In the end, I’m probably going to be making end-to-end tests using stubs for the classes I want to test. Once we have some time to evaluate mock object libraries for ActionScript, I’ll probably start digging again for more answers. We’ll eventually need something to help us with expectation-based testing anyway, so a library will be the way to go. Hopefully I’ll end up finding something more like Mocha or Moq than JMock, although I think the opposite may be true.

There are language elements in ActionScript that I’m extremely grateful to have at my disposal (e.g. – events, properties, functions, etc). A few quirks exist though (e.g. – object/class manipulation at runtime and reflection) that still get me when I try to think about ActionScript in the context of JavaScript. Going forward I think I’m going to switch gears and solve problems more as I would a Java/C# developer than a JavaScript developer, favoring the use of libraries over the language features to get every day tasks done. I like ActionScript, but what good would a blog be if I couldn’t complain.

:)

NOTE: I can’t take credit for the tagline, some Flex developers I know have begun to adopt it and I borrowed it from Dan.

Diving back into Selenium and saving me some time

Brian LeGros | September 24th, 2008 | programming  

Over the years I’ve been using Selenium as my tool of choice for UI testing on the web. Recently, we’ve had the need to regression test a vendor web application due to some poor quality builds being produced. Based on past experience, it takes us about 40 hours of work to regression test the product UI by hand, including double checking for annoying copy changes that always seem to slip through. Today we finished the first chunk of a baseline set of tests that accounts for roughly 30% of our test plans. It took us about 150% of the time needed to do one regression test to get this first chunk done. Since we regression test regularly, we’re going to incrementally add more tests each time, getting into a habit of updating our Selenium test suites with the latest changes per build. With that said, let me just tell you this is probably one of the most detailed applications I’ve had to test with Selenium. Guess how long it takes to run our tests that don’t require some type of manual setup … less than 5 minutes. So we’ve taken a third of our tests that took about 10 hours to execute and reduced them to 5 minutes of watching the browser go nuts with all the typing and clicking. There is still work to be done manually that we will never escape (global system setup), but talk about saving time. No matter how much I use Selenium I never get tired of the benefit it provides in the long run.

So all though these stats are encouraging for management to hear, but what I really like about Selenium is how simple it is to create test suites. Don’t get me wrong though, there a few things that can cause hang-ups. Hacking out XPath statements is one that I face quite frequently. Consider a command in Selenium called verifySelectLabel which takes a locator (typically an XPath expression) to find the select box in the DOM and a pattern (typically a simplified regex) to match the text of the option currently selected in the select box. The Selenium IDE has a “Find…” button to help validate the XPath statement and some context menu options in the browser to help, but in most cases I always find myself messing around with the XPath statements. Recently, while playing around with Firebug (my favorite JavaScript swiss army knife) I noticed that under the HTML tab when right-clicking on a tag you get the option Copy XPath. Selenium’s XPath support is really robust and the Firebug’s syntax ports over flawlessly. I did notice that while running in IE I had to simplify the statements sometimes to get them to pass, but otherwise, it’s cut my recording time down considerably.

Another thing that always sucks up my time is when I have to hack together JavaScript to get data ready for a test (especially when it comes to dates). The storeEval command will execute a JavaScript snippet and take the result of the last line and put it into JS variable for later use. In the past I’d fire up a kind-of scribble pad HTML doc with some inline script and test my code, but this time around Firebug is all I needed. The Console tab gave me my own scribble pad with the added help of the Firebug console object. Another cool JavaScript helper I hadn’t used before was the user-extensions.js file, which is imported into the test runner giving you access to whatever JS you want to re-use. I found a pretty descent date API that I copy/pasted into this file making working with dates in my tests stupid simple. Finally, this is the first time I’ve tested a product that I didn’t help write and that was written using ASP.NET Web Forms. With the help of debug(), monitor(), and the Inspect tool in Firebug, I was able to find my way around the mess of scripts on the page very quickly. Even though the generated code wasn’t my favorite, I have to give credit to ASP.NET’s management of inputs; each input was generate with a unique ID which made testing incredibly easy.

Overall, it was great to get back into using Selenium and I’m really excited at the possibilities of starting to use Selenium Remote Control and Grid. The prospect of getting my unit testing framework (NUnit) into the picture will definitely help to complete and simplify the build process. What’s even cooler is that once we’ve put all the work into recording our tests in HTML, the Selenium IDE will translate them into C# so we don’t have to repeat ourselves. With new extensions for working with Flex, this tool’s potential continues to grow. If you’ve never tried out Selenium, definitely give it a shot. The barrier to entry is low and you can choose the level of complexity you want to bring to the table.

acts_as_conference 2008 : BDD With RSpec

Brian LeGros | February 10th, 2008 | conferences  

After lunch we came back to Bryan Liles presentation on BDD which he retitled from “BDD with RSpec” to “BDD for Normal People”. Bryan is definitely a kindred spirit of mine, if only in his sense of humor. He was a great presenter and put a great face on the his own approach to TDD and BDD. What I liked about the presentation was his ability to bring practicality into the mix and give his audience direction and a place to start on their test driven journey. In any case, on to the topic at hand.

Brian started out describing his own journey into testing and some of the struggles involved. He pointed out that as most of us begin to integrate testing, many of us are adopting different approaches but still writing poor tests to cover poor quality code. Brian identified that tools can help us in our journey by introducing more clear test names and concepts such as context, but we still have to apply these tools more effectively. He gave the following steps as a basic guide to BDD:

  • Get comfortable with the concepts in TDD.
  • Make it your mission to practice TDD all of the time (even when it hurts).
  • Think of your application as objects expressing behaviors.
  • Create examples of those behaviors [and use them as the basis for your tests].

He expressed that TDD is painful and there is an upfront cost, but that upfront cost is migitated by the quality of the tests you produce and the less trouble you have as the software grows.

On the practical side of things, Brian, although praising what RSpec has done for testing in Ruby, suggested looking into alternatives that work a little better under the hood. He suggested sticking with Test::Unit, shoulda, and RSpec StoryRunner as he’s had the most success with this combination. Additionally, in the questions segment, Brian said he doesn’t have much faith in code coverage tools and only uses them to show a relative change in coverage to confirm that coverage has gone up or down.

Overall, I’ve got a lot of new topics to read on, and I’m sure Dan and Tyler will be happy to see me giving TDD/BDD another shot. Great work Brian!

NOTE: I forgot to mention the most important note from the entire presentation … TDD == BDD, it’s just a matter of perspective. Thanks to Dan for reminding me of this.

Frameworks Conference 2007 : Testing Frameworks

admin | May 17th, 2007 | conferences  

John Paul Ashenfelter gave this session. This session turned out the same way it has in the past 2 years, a good overview of what’s available in terms of free testing tools. He advocated the use of cfcUnit for unit testing and Selenium for integration testing. John always does a great job speaking and the session was informative.