cookies and code

This past weekend, I finished my port of the Hudson Xvnc plugin to the FlexUnit Ant task. I’m hoping this will help those who don’t have access to the plugin, or dont’ use Hudson, but want to execute headless builds using FlexUnit and Xvnc. In addition to this work, I’ve added a few wiki pages to the newly launched FlexUnit.org’s wiki instance. A break down of the all the features available in the FlexUnit Ant task can be found @ http://docs.flexunit.org/index.php?title=Ant_Task. Additionally, to see the Ant task in action in a few variations, check out the FlexUnit4SampleCIProject on Github; I also show a basic POM using Flexmojos and FlexUnit4. For a general culmination of FlexUnit and continuous integration experience for running headless builds, check out http://docs.flexunit.org/index.php?title=Continuous_Integration_Support. Hopefully this information can help to prevent some people from experiencing the painstaking lessons I had to go through.

Next I’m off to update the FlexUnit 4 CI builds to work with versioning as well as integrate the new FlexPMD (CPD, Metrics) to get some more sexy Hudson reporting going on. If you’re interested in seeing the latest rendition of our CI process, check out http://flexunit.digitalprimates.net.

Over the last year or so, I’ve worked to integrate xvfb into the projects I work with at the office and in the community to allow the execution of truly headless builds for Flex-based applications. Since the Flash Player and AIR require a windowing environment and typically unit and integration tests require this environment to be executed, integrating Flex into a CI process on Linux gets pretty hairy without having X-Windows support. Recently I’ve found myself switching from xvfb to Xvnc to more easily support headless builds in Flex. VNC server packages are pretty ubiquitous for most Linux distros, so getting something installed is pretty simple. What took me a couple of steps was getting setup based on the configuration of my CI server. In an effort to make searching a little simpler, here are the steps for using the Xvnc-plugin with Hudson on Ubuntu Desktop 8.10 with the default Tomcat 6 installation:

1. Install the vnc4server package (i.e. – “apt-get install vnc4server”).
2. Assuming the Ubuntu installation came with Tomcat 6 and is setup to run as the tomcat6 system user, do the following:
1. Execute vncpassword as any user, following the prompts to set a password to be used by the “vncserver” command. This will create the “~/.vnc/password” file needed to run the vncserver command.
2. Copy the file at ~/.vnc/password to /usr/shared/tomcat6/.vnc/password which is the default home directory for the tomcat6 user on the standard install of Ubuntu Desktop.
3. Change the ownership rights on the “password” file to be owned and created by the “tomcat6″ user (i.e. – chown tomcat6:tomcat6 /usr/shared/.vnc/password)
4. Edit /etc/init.d/tomcat6, adding a line to set the HOME environment variable to “/usr/shared/tomcat6″ (i.e. – export HOME=/usr/shared/tomcat6). Since “tomcat6″ is setup as a system user, from what I can tell, this environment variable is not set by default when a process is running as this user. The “vncserver” command will need this variable to determine where the “password” file is located to launch its process.
3. If Tomcat is running as a standard user, execute the “vncserver” command using any display number (e.g. – vncserver :99) as the user. The command will force a prompt to enter a password to use for the server which will create the “~/.vnc/password” file with the appropriate ownership rights in the appropriate home directory.
4. In Hudson, install the Xvnc-plugin for Hudson (http://wiki.hudson-ci.org/display/HUDSON/Xvnc+Plugin)
5. OPTIONAL: In Hudson, under “Manage Hudson” -> “Configure system” -> “Xvnc”, set the “Base display number” field to 99. This is the base display number xvfb-run uses and I’m partial to reusing this setting.
6. Under the configuration for you CI build, check the box labeled Run Xvnc during build.
7. If all goes as expected, then your build should kick off the “vncserver :99″ command, set the DISPLAY environment variable to 99, execute your build to completion, and then execute “vncserver -kill :99″ to kill the Xvnc process.

Based on these steps, I’m am currently building Xvnc support into the FlexUnit4 Ant task for those who don’t use Hudson and can’t take advantage of the Xvnc-plugin. The Xvnc support is a basic mirror of what the plugin for Hudson aims to provide. Hopefully someone will find this footwork useful and save some time. I’m by no means a Linux guru, so if anyone has more insight, please feel free to share so others can benefit from your knowledge as well.

Happy building!

UPDATE: I’ve now tested and deployed the Xvnc support for the FlexUnit Ant task. Hopefully this will help anyone who’s not using Hudson but wants to take advantage of headless FlexUnit test executions using Xvnc.

If you’re using FlexCover as a code coverage tool within your Flex build process, the resulting report will be an XML file with a .cvr file extension. Currently there are no FlexCover plugins for the popular CI servers (i.e. – Hudson, Cruise Control, Bamboo, etc), so to take advantage of the cool baked-in reporting for code coverage, an XML report for a supported report format (e.g. – Emma, Cobertura, Clover) is needed. Currently Paul over at eyefodder has a solution using a custom build of the CoverageViewer and CoverageAgent which will produce an Emma formatted report.

Instead of creating my own custom build of FlexCover, I decided to work on an XSL transformation to the Cobertura XML report format to see if I could have any success. After a pain-staking journey back into the world of XSL, I was able to throw together an XSLT that is working with Hudson’s Cobertura plugin. You can find the XSLT under FlexUnit4 source control @ http://opensource.adobe.com/svn/opensource/flexunit/branches/4.x/FlexUnit4Test/fc-to-cobertura.xsl. If you’d like to see the reports generated for Hudson in action, check out http://flexunit.digitalprimates.net/view/Flex%203.2/job/FlexUnit4-Flex3.2/cobertura/. We currently only have the builds using Flex SDK 3.2 running with FlexCover since the latest version of FlexCover (0.81) only supports versions 3.0 and 3.2 of the SDK. Looks like some work has been done to integrate with Flex 4.0, so as support is added to FlexCover, I’ll add that work into the FlexUnit4 CI builds.

To use this XSLT to transform the CVR file, make sure you are using an XSLT 2.0 compatible engine (e.g. – Saxon) with your build tool of choice. Additionally, you will need to provide three paramaters to the stylesheet for it to work correctly:

• sourcePath – A comma delimited list of absolute paths to the source directories, each excluding a trailing slash, for which the instrumented SWC/SWF was created.
• version – The version number of FlexCover you used to generate the CVR file.
• timestamp – The date/time in which the transformation occurred in ms from epoch. I’m using a date format of “MM/DD/yyyy HH:mm:ss.SSS’ and I’m not having any issues, but it may be ignored by the Hudson plugin.

If anyone has any suggestions on rewriting the XSLT using XSL 1.0 instead of 2.0, please let me know, I’m definitely open to refactoring it. Thus far the results look fairly consistent with a Cobertura XML report and integration with Hudson’s Cobertura plugin is working out great. Hope this helps those who are trying to tackle the FlexCover CI integration problem.

Today the 2nd beta release of FlexUnit4 was published. This release comes with a large list of bug fixes and improvements. This past weekend I burned the midnight oil with Mike and we buttoned up the FlexUnit4AntTasks project and I threw together a new sample project for FlexUnit4 showing how to integrate with Ant and Maven (via the Flex-Mojos plugin). Currently xvfb support isn’t available but we hope to add it in the next beta along with a few bug fixes and refactorings that we have time to finish. If you’re running your CI server on Windows using the Local System account, you should be able to take advantage of the new FlexUnit Ant task; the MacOS X and Linux crowd will have to work with source for now.

If your interested in more details, check out Mike’s latest blog post on the Beta 2 release.

At the August Adogo meeting, I presented on the new features coming in FlexUnit4 and mock-as3. During the presentation, I showed the following sample (which has been updated) of how a FlexUnit 4 test using a customer runner for mock-as3 would work. Please keep in mind the example is a bit contrive but exemplifies the configuration options for the runner.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

package us.adogo.mock
{
   import com.anywebcam.mock.Mockery;
 
   import org.flexunit.Assert;
   import org.flexunit.assertThat;
   import org.hamcrest.core.not;
   import org.hamcrest.object.equalTo;
   import org.hamcrest.object.hasProperty;
   import org.hamcrest.object.nullValue;
 
   [RunWith("com.anywebcam.mock.runner.MockAs3TestRunner")]
   public class ExampleUsingMockAs3Runner {
      public var mockery : Mockery;
 
      [Mock]
      public var user : User;
 
      [Mock(inject="false")]
      public var account : Account;
 
      private var controller : UserController; 
 
      [Before]
      public function setUp() : void {
         account = mockery.nice(Account, ["1234567890"]) as Account;
         this.controller = new UserController(); 
      }
 
      [Test]
      public function testAddUserWithUsernameAndPasswordOnly() : void {
         //setup mock
         mockery.mock(user).method("save").calls(function () : void {
            user.id = 1;
         }).once;
 
         //populate object properties as you usually would
         user.username = "bobdobbs";
         user.password = "mysecret";
 
         //execute my controller method being tested
         controller.addUser(user);
 
         //test that the currentUsers went up by one and that its id is 1
         assertThat(controller.currentUsers.length, equalTo(1));
         assertThat(controller.currentUsers.getItemAt(0).id, equalTo(1));
         assertThat(controller.currentUsers.getItemAt(0).username, equalTo("bobdobbs"));
         assertThat(controller.currentUsers.getItemAt(0).password, equalTo("mysecret"));
      }
 
      [Test]
      public function testValidateUserAccount() : void {
         //setup mock
         mockery.mock(account).method("isValid").withArgs(User).returns(true).once;
 
         //execute my controller method being tested
         var expected : Boolean = controller.validateUser(new User(), account);
 
         //test that the account is valid
         Assert.assertTrue(expected);
      }
 
      [Test(verify="false")]
      public function testDefaultUserSize() : void {
         assertThat(controller, hasProperty("currentUsers"));
         assertThat(controller.currentUsers, not(nullValue()));
         assertThat(controller.currentUsers.length, equalTo(0));
      }
   }
}

This evening I updated the Adogo SVN with a working version of this runner and thought I’d detail its initial implementation.

• To use the runner place the metadata [RunWith("com.anywebcam.mock.runner.MockAs3TestRunner")] on the test class’ declaration.
• Once annotated with metadata, the test class will not run unless a public variable, or setter, is available of type com.anywebcam.mock.Mockery. This will allow the runner to inject a Mockery object and prepare() the mockery to produce mock objects. Using and preparing a mockery is a new requirement for mock-as3 if you’re utilizing type-safe mocks due to the asmock integration.
• Any public variable, or setter, annotated with the metadata [Mock] will have a type safe version of it injected automatically prior to the execution of each test method. By default a nice mock will be used, but if you specify the “type” attribute (e.g. – [Mock(type="strict")]) as “strict” then a strict mock will then be used. The difference between nice and strict mocks, is the equivalent of the “ignoreMissing” constructor argument on com.anywebcam.mock.Mock being set to true, or false, respectively.
• If the [Mock] metadata is used on a class with a constructor requiring arguments, whether they are optional or …rest, the runner cannot automatically inject the mock due to a limitation in its underlying dependency on the asmock library. If you need a mock object instance using constructor arguments, set the attribute “inject” as false (e.g. – [Mock(inject="false")]. Doing so will tell the runner that you’re going to use the mock-as3 framework directly to create the mock. The ideal place to do this in your test is within the [Before] methods defined for the test class. By going through all this hoopla, you get automatic mock prepration and verification, just as with the automatically injected mock objects.
• After the execution of each test method, the runner will automatically call verify() on each object variable/property marked with the [Mock] metadata. If you’d like to disable verify() from being called on all mocks for a test method, simply add the attribute “verify”, with a value of false, to your [Test] metadata (e.g. – [Test(verify="false")]). This can be helpful if you want the runner to inject the mockery and all of your mocks, but you would like to specify which mocks are verified for the test.
• Currently this runner requires FlexUnit 4 Beta 2 and a custom build of mock-as3 to work as stated. You can find SWCs for each in the Adogo August project linked above.

What’s important to note, is that this runner is more restrictive than using the mock-as3 framework directly, so it may not suite your needs. I based this runner off of concepts I saw in the JMock runner for JUnit 4, the @Mock annotation found in Mockito for Java, and my own testing practices. If anyone finds time to play with it and you find any gremlins, let me know. In speaking with Drew, it looks like this may make it into the next release of mock-as3 along with a couple of other cool features, so I hope people find it helpful.

UPDATE: Drew has been kind enough to deploy this FU4 test runner to the mock-as3 SVN repository under the class name com.anywebcam.mock.runner.MockRunner. Please use this copy for future reference. Please also note this version will only work with FlexUnit4 Beta2 and earlier. FlexUnit4 RC1 has changed the implementation for test runners and this code will need to be updated.

Well I gave my hastely assembled and zero practiced version of a FlexUnit4 and mock-as3 presentation last night at the Adogo and I only caught a few people snoozing, so that is a +1 in my book. Unit testing can be extremely dry for most people, but building tools to make me more productive always holds my interest so that’s how I’ll explain my enthusiasm. Thanks to Drew Bourne and Michael Labriola for the help with the presentation. I blubbered through most of the points on their APIs and definitely mispoke on a few instances, but as long as the recording doesn’t get too much traffic we should be fine. Thanks as well to Russ, Vincent, Brian, and Greg for keeping me company after the meeting over a few beers while I waited for my wife’s flight to get in.

I’ve published the source and recording for anyone who is interested.

I’m going to be presenting at this Monday’s (08/03/2009) August Adogo meeting after a few month hiatus due to the new baby. I’ll be running through the new features in FlexUnit4 and some work I’m doing with Drew on mock-as3. Afterwards, we are going to have a group discussion on “Uses for RIAs” where we’ll try to talk about where RIAs are applicable and how they can be abused. If you’re interested in testing in Flex or developer discussion, come by and see the latest and greatest coming out of the community. We have a sponsor for the meeting, so there will be food.

I hope you can make it out.

Just a quick note. Last night the Mike did a blog post on the new features coming to FlexUnit and the working title of FlexUnit 4 for the project. You can find the blog post @ http://blogs.digitalprimates.net/codeSlinger/index.cfm/2009/5/3/FlexUnit-4-in-360-seconds. There is a link to the alpha in the post and as soon as it’s available on Adobe’s servers, we’ll publish that URL as well. Keep in mind that it does support legacy FlexUnit and Fluint tests, so dig in and let us know what you think!

Mike Labriola today announced on the Fluint Discussion group the big news a few in the Fluint team have been sitting for months now. The Fluint team is leading an OSS initiative to create the next version of the FlexUnit product. What’s great about this initiative (as Mike outlines on the mailing list) is that the new version of Fluint has been designed to mimic a lot of the conventions found in JUnit 4 while still addressing all of the problems Fluint solves for us in the Flex world. I know, for me, this means being more productive when writing tests and finally having better tooling when working with Flex. Here are some of the features listed from Mike’s email:

• Metadata based Test and Suite identification (no more need to extend testcase or testsuite)
• Hamcrest matchers courtesy of the hamcrest-as3 project.
• Theories, DataPoints and Assumptions
• Ignorable tests
• Enhanced Sorting and Filtering
• Custom runner integration

Probably one of the coolest part of this entire framework thus far is that last item. The Fluint team has already been able to successfully run legacy unit tests from the current versions of FlexUnit and Fluint. This means that you can have hybrid suites of tests in the new FlexUnit (e.g. -FlexUnit, Fluint, FUnit, asunit, etc) making migration between frameworks relatively simple.

We’re going to be launching a public alpha release in the near future, but we are still continuing work on Fluint at least as far as release 1.2 (1.1.1 was release yesterday). Please keep in mind however, the goal is for FlexUnit to become what would have been Fluint 2.0. Look for more details in the coming weeks.

Here’s to better testing!

Over the last few weeks, while preparing my presentation for FlexCamp Miami a bit more, I have run into quite the plethora of tools to help developers test their Flex applications. I thought I’d take some time to break down what types of tools are available to help us out.

Unit Testing Frameworks

• fluint – http://code.google.com/p/fluint/
• flexunit – http://opensource.adobe.com/wiki/display/flexunit/FlexUnit
• funit – http://blog.funit.org/
• asunit – http://www.asunit.org/

Mock Object Frameworks

• mock-as3 – http://code.google.com/p/mock-as3/
• asmock – http://asmock.sourceforge.net/
• mock4as – http://code.google.com/p/mock4as/

Other useful libraries

• hamcrest-as3 – Assertion library – http://github.com/drewbourne/hamcrest-as3/tree/master
• asx – Extension methods for AS3 base types – http://github.com/drewbourne/asx/tree/master
• loom – Byte-code weaving framework for AS3 – http://code.google.com/p/loom-as3/

My favorite combination right now is fluint, mock-as3, hamcrest-as3, and asx. Although asmock supports type-safe mocking, I’m not a big fan of the record/replay model; for now I’m gonna stick with mock-as3 and wait until it supports type-safe mocks (<cough>loom integration<cough>. I’ve also found a major need for stubs when classes in my integration tests depend on HTTPService and RemoteObject. Sometimes its tough to mock out these classes because of complex interfaces these objects maintain, so I’ve come up with a pretty cool way of handling this. Hopefully I can publish the sample code here in the next few days for those who are interested.

I’d also like to give props to a couple people I’ve been working with in the community: Michael Labriola, Max Porges and Drew Bourne. Mike is working tirelessly to bring a more modern unit testing framework, a la JUnit 4 + more, to Flex via fluint along with tons of other initiatives. mock-as3, hamcrest-as3, and asx are all APIs that Drew has written; hamcrest-as3 alone is going to be providing some really powerful integration for assertions in unit testing frameworks. Max has been chipping away at the ABC spec with loom for months now, and once finished it’s going to provide a lot of great potential for more mature tooling/libraries in the Flex world … think AOP. Each of these code-bases are intuitive, easy to use, and were some of the biggest pieces missing from the Flex testing world until now. Keep up the great work guys! With any luck, we may soon have the tools we need to be as productive as developers in almost every other language out there.