Converting a FlexCover CVR to Cobertura XML report

Brian LeGros | October 19th, 2009 | programming  

If you’re using FlexCover as a code coverage tool within your Flex build process, the resulting report will be an XML file with a .cvr file extension. Currently there are no FlexCover plugins for the popular CI servers (i.e. – Hudson, Cruise Control, Bamboo, etc), so to take advantage of the cool baked-in reporting for code coverage, an XML report for a supported report format (e.g. – Emma, Cobertura, Clover) is needed. Currently Paul over at eyefodder has a solution using a custom build of the CoverageViewer and CoverageAgent which will produce an Emma formatted report.

Instead of creating my own custom build of FlexCover, I decided to work on an XSL transformation to the Cobertura XML report format to see if I could have any success. After a pain-staking journey back into the world of XSL, I was able to throw together an XSLT that is working with Hudson’s Cobertura plugin. You can find the XSLT under FlexUnit4 source control @ http://opensource.adobe.com/svn/opensource/flexunit/branches/4.x/FlexUnit4Test/fc-to-cobertura.xsl. If you’d like to see the reports generated for Hudson in action, check out http://flexunit.digitalprimates.net/view/Flex%203.2/job/FlexUnit4-Flex3.2/cobertura/. We currently only have the builds using Flex SDK 3.2 running with FlexCover since the latest version of FlexCover (0.81) only supports versions 3.0 and 3.2 of the SDK. Looks like some work has been done to integrate with Flex 4.0, so as support is added to FlexCover, I’ll add that work into the FlexUnit4 CI builds.

To use this XSLT to transform the CVR file, make sure you are using an XSLT 2.0 compatible engine (e.g. – Saxon) with your build tool of choice. Additionally, you will need to provide three paramaters to the stylesheet for it to work correctly:

  • sourcePath – A comma delimited list of absolute paths to the source directories, each excluding a trailing slash, for which the instrumented SWC/SWF was created.
  • version – The version number of FlexCover you used to generate the CVR file.
  • timestamp – The date/time in which the transformation occurred in ms from epoch. I’m using a date format of “MM/DD/yyyy HH:mm:ss.SSS’ and I’m not having any issues, but it may be ignored by the Hudson plugin.

If anyone has any suggestions on rewriting the XSLT using XSL 1.0 instead of 2.0, please let me know, I’m definitely open to refactoring it. Thus far the results look fairly consistent with a Cobertura XML report and integration with Hudson’s Cobertura plugin is working out great. Hope this helps those who are trying to tackle the FlexCover CI integration problem.

FlexUnit 4 public alpha now available

Brian LeGros | May 4th, 2009 | news  

Just a quick note. Last night the Mike did a blog post on the new features coming to FlexUnit and the working title of FlexUnit 4 for the project. You can find the blog post @ http://blogs.digitalprimates.net/codeSlinger/index.cfm/2009/5/3/FlexUnit-4-in-360-seconds. There is a link to the alpha in the post and as soon as it’s available on Adobe’s servers, we’ll publish that URL as well. Keep in mind that it does support legacy FlexUnit and Fluint tests, so dig in and let us know what you think!

Fluint 1.1.0 Released!

Brian LeGros | February 12th, 2009 | programming  

Over the last few months, I’ve been working with Michael Labriola on the next minor version release of Fluint. Well, this evening Michael and I put the finishing touches on release 1.1.0. This release brought all of the changes I made to get Fluint working at the office for our CI process to the community. There are a couple of cool little gems that I think will be helpful for some:

  • Separation of failures and errors in test reporting (visual and XML).
  • XML compliant output with most CI servers as well as the JUnitReport task in Ant and the Surefire Reporting plugin in Maven.
  • Support in the Ant task for truly headless executions of the AIR Test Runner via xvfb-run.
  • Support for relative paths in the Ant task and AIR Test Runner.
  • Better error handling for non-compliant modules loaded into the AIR Test Runner.
  • LogBook integration for debugging
  • A big folder re-organization and Ant scripts to support developers when they need specialty builds.
  • Agreed upon a branching and tagging strategy so the community can have stable snapshots of source to build.

We probably should put together a ChangeLog wiki page, so that may be available soon. We did a few updates to the wiki, so stop by the Google Code site and check out the changes. Also, provide feedback on the discussion list, if there are features you’d like to see implemented, bugs that you find, or just general questions you need help with. People are great about helping on the list, so don’t feel discouraged.

This is my first major release of an OSS project that I’ve had code go into, so thanks to everyone who helped with the release, especially my wife for putting up with a lot of late nights. There is a lot more to come with Fluint over the next year that is going to be really exciting. Look for announcements during my presentation at FlexCamp Miami and on the discussion mailing list. Hope you enjoy the new release!

Continuous Integration with Maven, Flex, Fliunt, and Hudson

Brian LeGros | December 17th, 2008 | programming  

Recently I was tasked with streamlining our build process at work so we could get a continuous integration (CI) server up and running. We use the common stack of technologies found in most Flex shops (basic SDK, some libraries, and Flex Builder) as well as Maven. I ran into some challenges getting our CI process to work as we wanted, so I figured I’d go through some of the gotchas I encountered.

On the build side of things, when I came on, Maven was already in place using flex-mojos. Now I’m a big fan of the simplicity that Ant brings to the mix, but the issue of dependency management being baked into Maven makes it extremely appealing; I do like Ivy as an alternative when using Ant, but I wasn’t going to re-write the company’s build process. So we had flex-mojos building our source and producing artifacts for deployment to our team’s Maven repository, but we needed to integrate our unit tests into our build. We were using dpuint and were excited to see that fluint had been released with Ant support. Currently flex-mojos doesn’t support Fluint, although my colleagues tell me they’re working on it, so I knew I was going to have to use Ant. To start I had to get flex-mojos building my test SWF so I could use the Fluint Ant task. The Ant support in Fluint requires that you produce a module SWF that will work with their test runner written in AIR. After an hour of messing with flex-mojos, I was unable to get the compile, or test-compile, goal to do what I wanted, so I decided to use the maven-ant-run plugin to compile our tests as well.

Below is the snippet I was able to get working to compile our tests and execute the Fluint test runner:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
<properties>
   <flex.home>PATH_TO_FLEX_SDK_HOME</flex.home>
   <fluint.testrunner>PATH_TO_FLUINT_AIR_RUNNER_EXECUTABLE</fluint.testrunner>
</properties>
...
<build>
   <plugins>
      <plugin>
         <groupId>info.flex-mojos</groupId>
         <artifactId>flex-compiler-mojo</artifactId>
         <version>2.0M9</version>
         <extensions>true</extensions>
         <configuration>
            <skipTests>true</skipTests>
         </configuration>
      </plugin>
      <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-antrun-plugin</artifactId>
         <executions>
            <execution>
               <id>fluint-test-compile</id>
               <phase>test-compile</phase>
               <configuration>
                  <tasks>
                     <!-- Pull in Flex Ant Tasks -->
                     <taskdef resource="flexTasks.tasks" />
 
                     <property name="FLEX_HOME" location="${flex.home}" />
 
                     <!-- Create test-classes directory -->
                     <mkdir dir="${project.build.testOutputDirectory}" />
 
                     <mxmlc file="${project.build.testSourceDirectory}/AirRunner.mxml" 
                        output="${project.build.testOutputDirectory}/AirRunner.swf" 
                        keep-generated-actionscript="false">
 
                        <load-config filename="${FLEX_HOME}/frameworks/flex-config.xml" />
                        <source-path path-element="${FLEX_HOME}/frameworks"  />
                        <compiler.library-path dir="${FLEX_HOME}/frameworks" append="true">
                           <include name="libs" />
                        </compiler.library-path>
                        <compiler.library-path dir="${project.build.directory}/.." append="true">
                           <include name="libs" />
                        </compiler.library-path>
                        <compiler.library-path dir="${project.build.directory}" append="true">
                           <include name="*.swc" />
                        </compiler.library-path>
                     </mxmlc>      
                  </tasks>
               </configuration>
               <goals>
                  <goal>run</goal>
               </goals>
            </execution>
            <execution>
               <id>fluint-test-run</id>
               <phase>test</phase>
               <configuration>
                  <tasks>
                     <!-- Pull in Fluint Ant Task -->
                     <taskdef name="fluint" classname="net.digitalprimates.ant.tasks.fluint.Fluint" />     
 
                     <property name="test.report.loc" location="${project.build.directory}/surefire-reports" />
 
                     <!-- Create reporting directory -->
                     <mkdir dir="${test.report.loc}" />
 
                     <fluint debug="true" 
                        headless="true"
                        testRunner="${fluint.testrunner}" 
                        outputDir="${test.report.loc}" 
                        workingDir="${project.build.testOutputDirectory}">
 
                        <fileset dir="${project.build.testOutputDirectory}">
                           <include name="**/AirRunner.swf"/>
                        </fileset>
                     </fluint>      
                  </tasks>
               </configuration>
               <goals>
                  <goal>run</goal>
               </goals>
            </execution>
         </executions>
         <dependencies>
            <dependency>
               <groupId>org.apache.ant</groupId>
               <artifactId>ant</artifactId>
               <version>1.7.0</version>
            </dependency>
            <dependency>
               <groupId>flex.ant</groupId>
               <artifactId>flexTasks</artifactId>
               <version>1.0.0</version>
            </dependency>
            <dependency>
               <groupId>net.digitalprimates</groupId>
               <artifactId>FluintAnt</artifactId>
               <version>1.0.1-SNAPSHOT</version>
            </dependency>
         </dependencies>
      </plugin>
   </plugins>
</build>

Couple of things to point out about the above snippet:

  • There are two dependencies on resources being available on the disk, the Flex SDK (flex.home property) and the Fluint Air Runner (fluint.testrunner property).
  • Using Ant I had to create the “test-classes” and “surefire-report” directory to stick with Maven conventions.
  • We adhered to the convention of naming our test runners for the Fluint Ant task “AirRunner.mxml” so we could use this snippet in a parent POM.
  • I had to change the dependency for the maven-ant-run plugin from Ant 1.6.5, which is the default, to Ant 1.7.0, which is required by the Fluint Ant task.

You may also notice that I’m using snapshot versions of the Fluint library and the Fluint Ant task. I ended up having to change the source of the Fluint library, Ant task, and AIR runner to get Fluint to work as I wanted it to with my build. Fluint is an awesome unit testing library, it just needed some tweaks. I made changes to fix the following:

  • The XML output from the Fluint AIR runner wasn’t compliant with what the Surefire Report plugin was expecting.
  • The name of output file from the Fluint AIR runner was in the convention “TEST-*.xml” which the SureFire reporting plugin expects.
  • Fluint had the notion of an error and failure being separate but it wasn’t implemented for the Flash or AIR test runners.
  • The Ant task didn’t allow the user to specify a working directory so that the AIR runner could be launched from the appropriate directory.

I later found out that AIR and relative paths don’t play nicely together (= at all, unless there is helper code), so we also had to re-factor our test suites to NOT rely on any assets unless they are embedded or referenced with absolute URIs. This made the change to the Fluint Ant task kinda worthless, but I kept it in anyway for when AIR works in the future. Additionally, it’s important to note, that the “headless” mode in the Fluint AIR runner is really just a minimized window that closes after the XML report is written; if you plan on running your CI build on an OS without a windowing solution, then FLuint will not work since AIR does not support running in a true headless mode. On a side note, my changes should address issues #5 and #22 on the Fluint Google Code site; issue #21 should be solved by the Ant dependency fix I spoke about above and issue #20 is just a matter of the fluintAnt15.jar being compiled with Java 1.6 instead of 1.5, I believe. I’ve submitted these fixes along with my code to the Fluint guys in an email, just haven’t heard anything back yet.

So at this point I had the build process working as I wanted such that I could run “mvn clean deploy site” and find a snapshot in our team’s maven repository and site documentation generated. On to CI. I have used CruiseControl many times in the past, but the idea of being entrenched in XML, especially with all the Maven and Ant fun, was discouraging so I decided to give Hudson a try this time around. Wow … Hudson is amazing improvement over CruiseControl. Completely UI driven, I have yet to find myself digging through XML and best of all. The post-build support feel a little lighter than CruiseControl’s, but I think that’s just because I haven’t come across an X10 plugin so we can get a stop light or glowing orb setup. Hudson provides trend reporting on builds and unit tests as well as embedded reporting for unit tests and xdoclet-like documentation; it also has tons of Groovy integration which I really like (not that we’re using it … yet). Initially I chose to go with the pure Maven build for our projects, but I then decided to switch back to the free-style build; I couldn’t get trend reporting for unit tests to work with the pure Maven build, so I think my conventions are off for the Fluint reporting. In the free-style build I set the “site” directory as the Javadoc location and the “surefire-report” directory as the test report directory. Even though there is more configuration in a free-style build, it was simpler in the short run to get what I wanted in Hudson running. If unit test trend reporting isn’t as important to your CI needs, then the pure maven build may be more along the lines of what you’d like to use so that you can get the additional build trigger “Build whenever a SNAPSHOT dependency is built”. On a side note, I’m working towards using the FlexCover support in flex-mojos to make our site reports complete, but haven’t had a chance to dive in yet.

I hope some of the hurdles I encountered can help if you’re trying to get your CI process working with Flex. The flex-mojos, Fluint, and Hudson guys have some good walkthroughs/tutorials to cover the details I left out. I’m always up for suggestions, so definitely feel free to rip into my solutions. :)

cf.objective() 2007 : Test Driven Development

admin | May 6th, 2007 | conferences  

This session was with Paul Kenney who is now working with Adobe in their Hosted Services Group. He is also the creator of cfcUnit, so I was eager to see what the framework could do compared to our experience with cfUnit.

Paul basically described test driven development (TDD) as the incremental process of designing your unit test cases prior to writing your code. He was explicit in saying however that you should develop test cases and their corresponding code one at a time. The goal was not to spend the entire time developing a test suite that didn’t have the ability to run. He said TDD was the process of implementing the most basic functionality needed (the provided specs in most cases) to satisfy the Test (I usually call this a test method). Once you’ve completed your code base and test cases, TDD requires that you participate heavily in refactoring to seek out the following issues:

  • Duplication of responsibility
  • Unclear Intent (classes with low cohesion)
  • Code Smell (=Intuition about your code)
    • Adding relevant comments
    • Indentifying if something should be a data class (which I think is what he’s calling a VO)
    • Code Duplication
    • Inappropriate Intimacy
    • Large Classes (possibly low in terms of cohesion in which size is a potential indicator)
    • Lazy Class (not sure about this one)
    • Long Method (again another low cohesion issue)
    • Switch Statements (possible indicator for seperation of responsibilities)
    • Shotgun Surgery (code changes propagated across multiple classes, possible identification of tight coupling)

I like some of the terms he used because they’re quirky, but I didn’t see a lot of formalization in the presentation.

Paul then showed us the basics of how cfcUnit works. He has a nice test runner compared to cfUnit, but with the CFEclipse integration now, I don’t see that as a huge advantage. I also noticed that on a failure, only the failure message, default or custom, is delivered in his test runner, but no line number is available. This kinda sucks and I’m not sure cfUnit has that functionality, but it’d be nice to see. Overall, I didn’t really see a reason to switch over to cfcUnit from cfUnit at work. Pauls says now that he’s getting a lot of community support, he is going to start putting together a bunch of new resources for the API, improving support, and begin looking into what upgrades he can give the framework. I’d definitely like to see more in terms of the hacked form of annotations I’m seeing used now at the conference (idea of added extra attributes to a tag, so getMetaData() can be used during introspection on the tags).