Testing Tools for Flex

Brian LeGros | February 20th, 2009 | programming  

Over the last few weeks, while preparing my presentation for FlexCamp Miami a bit more, I have run into quite the plethora of tools to help developers test their Flex applications. I thought I’d take some time to break down what types of tools are available to help us out.

Unit Testing Frameworks

Mock Object Frameworks

Other useful libraries

My favorite combination right now is fluint, mock-as3, hamcrest-as3, and asx. Although asmock supports type-safe mocking, I’m not a big fan of the record/replay model; for now I’m gonna stick with mock-as3 and wait until it supports type-safe mocks (<cough>loom integration<cough>. I’ve also found a major need for stubs when classes in my integration tests depend on HTTPService and RemoteObject. Sometimes its tough to mock out these classes because of complex interfaces these objects maintain, so I’ve come up with a pretty cool way of handling this. Hopefully I can publish the sample code here in the next few days for those who are interested.

I’d also like to give props to a couple people I’ve been working with in the community: Michael Labriola, Max Porges and Drew Bourne. Mike is working tirelessly to bring a more modern unit testing framework, a la JUnit 4 + more, to Flex via fluint along with tons of other initiatives. mock-as3, hamcrest-as3, and asx are all APIs that Drew has written; hamcrest-as3 alone is going to be providing some really powerful integration for assertions in unit testing frameworks. Max has been chipping away at the ABC spec with loom for months now, and once finished it’s going to provide a lot of great potential for more mature tooling/libraries in the Flex world … think AOP. Each of these code-bases are intuitive, easy to use, and were some of the biggest pieces missing from the Flex testing world until now. Keep up the great work guys! With any luck, we may soon have the tools we need to be as productive as developers in almost every other language out there.



Tags: , , , , , , , , , , , ,

Related posts

Discussion

  1. Maxim Porges Says:
    March 3rd, 2009 at 11:53 pm

    Thanks for the props, Brian… I look forward to thoroughly disappointing as soon as Loom is released. :)

    - max

  2. aetuo Says:
    June 10th, 2009 at 11:28 am

    Why do you prefer mock-as3 to mock4as ? I don’t see the difference…

  3. Brian LeGros Says:
    June 10th, 2009 at 9:59 pm

    @aetuo – Thanks for stopping by. Right now my preference leans more towards mock-as3 mainly due to documentation and flexibility. In the post I said I preferred it over asmock, but didn’t really comment on mock4as. Here are a couple things I noticed:

    - mock4as docs are pretty verbose and don’t give good examples, whereas mock-as3 does a good job of simply communicating this, you can see what I mean by the developer’s guide for mock4as (http://code.google.com/p/mock4as/wiki/DeveloperGuide) vs mock-as3(http://code.google.com/p/mock-as3/wiki/SettingExpectations).

    - The source for mock4as is not available on the Google Code site. I’m a big proponent of contributing back to a project when I find a bug; I’m also a bigger proponent of fixing the bug to work for my code first and not having to wait on a OSS developer :P

    - From a technical standpoint, at least from what I can tell based on the google code sites, in mock-as3, more work seems to be places on supporting variations with the different statments for expectations. For example, withArgs in mock4as is only every shown, from what I saw, passing explicit argument values, where mock-as3 can except regex, types, and values. I also like the variations for multiplicity that mock-as3 offers (e.g. – never, always, twice, etc.); I feel this contributes to more readable tests.

    - I’ve spoken with Drew a few times and know he’s a pretty smart dude and has done some good work with hamcrest-as3 which is going into the new FlexUnit 4 as a basis for assertions, so I like his approach to building APIs. I don’t know the author(s) of mock4as or asmock.

    At the end of the day it’s developer preference. If mock4as brings mock objects to your application with a simplicity you prefer, then go for it. Just my 2 cents.

  4. Pete Hodgson Says:
    June 28th, 2009 at 1:10 pm

    Hey Brian,

    I’ve been contributing a little bit recently to the mock4as project. It was really interesting to read your post, and your comments re: the choice between mock4as and mock-as. I agree with you on the documentation issue. I’ve written up a couple of the newer mock4as features as blog posts (would love to hear any feedback you might have), but in general the documentation is a little disorganized.

    Oh, and the mock4as source is freely available on the code.google site. You can access it via svn – http://code.google.com/p/mock4as/source/checkout – or just browse it on the site itself – http://code.google.com/p/mock4as/source/browse/.

    I also agree that Drew is contributing great things to the Flex ecosystem. I’ve been playing with hamcrest-as3 recently. I’m thinking about trying to integrate it into mock4as in order to get more advanced argument matchers available.

  5. Brian LeGros Says:
    June 28th, 2009 at 10:25 pm

    @Pete – Thanks for stopping by. I went looking for the source for mock4as at the URL you gave you me as well as via the “Source” tab on the Google project page and all I find is a trunk with samples and tests in it, but no source code. Could the source only be available to project members? I’m unable to see it.

    Great to see mock4as offering more features, especially when normalizing on libraries like hamcrest-as3. The willExecute extension you’re working on is useful as I find myself using calls() in mock-as3 a lot to accomplish something similar. Keep up the great work and best of luck on mock4as.

  6. Robert Penner Says:
    July 24th, 2009 at 6:14 pm

    Mockito is out for AS3! Check it out:

    http://bitbucket.org/loomis/mockito-flex/wiki/Home

    I much prefer Mockito’s syntax to the other mocking frameworks.

  7. Brian LeGros Says:
    July 25th, 2009 at 7:40 am

    @Robert – Thanks for stopping by. Mockito for Flex wasn’t out when I wrote this post, but it’s really encouraging to see the work from the asmock team coming over into a great mocking framework like Mockito. I’m hoping that the Mockito team will work on integrating with FlexUnit4 and providing a Runner class similar to their JUnit runner in Mockito for Java. I’d also like to see the [Mock] metadata provided as the @Mock annotation is for JUnit. With all the new features in FU4, it’s totally feasible.

  8. Kris Says:
    August 5th, 2009 at 9:32 am

    Brian,

    Use of metadata is a great idea. However there is a limitation that has huge impact on usability of this feature. Due to the nature of flex, generated mock classes have to call the constructor from the base class. If you take a look at the mock() function it allows you specifying constructor arguments. If I just go through properties marked with the metadata I may not be able to create mock objects for them beacuse of that. It would be possible to come up with some primitive way to pass on the arguments but that’s much more work than it seems from the beginning.

    I might start with at least gathering classes to mock that way and let you guys create mocks by hand.

    Please come to our mailing list to discuss feature requests.

    http://groups.google.com/group/mockito-flex

    Regards,
    Kris

  9. Brian LeGros Says:
    August 5th, 2009 at 10:00 am

    @Kris – You are correct, there is no easy way to inject Mocks which do not have a no-arg constructor using the current metadata constructs. Very rarely, however, do I find myself not including a no-arg constructor on my AS3 classes. The classes I want to mock are either mirrors of a server-side domain, for which the AMF gateway mandates a no-arg constructor, classes in the Flex SDK which typically have a no-arg constructor, or classes which will be used in MXML, which also mandate a no-arg constructor. I look at the runners as an optional way of working with FlexUnit 4 such that if you follow a particular style you can take advantage of the boiler-plate the runner reduces. Yes constructor injection is something to be addressed, but I think it’s an edge case that a lot of AS3 devs won’t run into.

    If you haven’t joined the FlexUnit 4 forums yet, try to get on; we’ve been having discussions re: runners and asmock, mock-as3, etc that has talked about similar topics. I’m currently writing a runner for mock-as3 that injects a mockery, injects mocks declared as public properties and annotated with [Mock], and verifies that each mock was satisfied. It’s restrictive, but it’s the style myself and my team have found ourselves using when writing tests. I just signed up for the group Flex-Mockito, so hopefully we can dialog more about this.

  10. Phil Leggetter Says:
    August 21st, 2009 at 11:44 am

    I was wondering if you could possibly save me some time and point me in the right direction? I’m not looking for an in-depth detailed response, just some links would be very much appreciated. The reason I’m posting this here is that you’ve listed your preferred stack of Flex unit testing components (fluint, mock-as3, hamcrest-as3, and asx) but I wanted to know if your opinion had changed since February 2009, when you wrote this post. The reason this is of insterest to me is that I’m about to embark on my first Flex project and today I’ve been tasked with the following:

    1) Investigate the best Flex IDE – the defacto for this appears to be FlexBuilder.
    2) Investigate a Flex unit testing framework.
    3) Investigate a Flex acceptance testing framework
    4) Investigate how 2) and 3) can be linked into a continuous integration environment.

    So, my questions for you are:

    1) Is FlexBuilder/FlashBuilder (probably the Eclipse Plugin for me) definitely the best IDE? Would you stick with FlexBuilder 3 or go with Flash Builder 4 beta?
    2) What unit testing framework do you recommend? You’ve posted about Fluint in the past. Is Fluint now Flex Unit 4?
    3) Can you recommend an acceptance testing framework? Using this framework I’d also like to measure performance of the library I’m developing so a framework that has little to no impact on the runtime environment (probably memory and CPU used by the browser) would be ideal.
    4) Do you have any recommendations in terms of integration of Flex unit tests and acceptance tests into a continuous integration environment?

    You opinion on this would be greatly appreciated.

  11. Brian LeGros Says:
    August 21st, 2009 at 9:43 pm

    @Phil – I’ll try to keep the response short and then if you’d like more detail, feel free to email me @ me at brianlegros dot com so we can discuss further.

    1) FlashBuilder 4 beta is a much better IDE than FlexBuilder 3 IMO. If you can cope with the trial period until it’s released then you should be ok (no idea when that is). Keep in mind project files from FB4 are not backwards compatible with FB3.

    2) I’d suggest going with the latest FlexUnit4 beta; it is the latest rendition of the FlexUnit framework published by Adobe. It’s extremely stable and the most feature rich framework out there. Additionally, it’s compatible with running fluint and FlexUnit1 tests if you decide to go back to one of these frameworks. Documentation is a bit thin, but the forums have lots of answers out there. In its final release FlashBuilder4 should have contextual support to run tests for it as well.

    3) I’m a little confused by the term acceptance testing framework, but there are functional testing frameworks and profilers that work with Flex. FB Professional 3&4 have a Flex Profiler that will let you analyze the memory footprint and CPU utilization when running the final Flash movie. Ideally you should use any OS level monitoring tools you have in addition to the profiler to get a better idea of the system impact. In terms of functional testing, check out FlexMonkey 1.0. It’s matured quite a bit and can help in automating quality assurance testing for Flex applications.

    4) I would suggest going with Ant and the Flex and FlexUnit Ant tasks to create a simple build for your Flex project before diving into anything more complicated like Flex-Mojos and Maven. The build files produced by FB cannot be re-used by a CI server unfortunately so you’ll need an external build tool to integrate with a CI process. In terms of a CI server, I would suggest giving Hudson a go instead of something like Cruise Control. IMO, Hudson is a vastly superior tool.

    Good luck with your search. Let me know if I can help.

  12. Phil Leggetter Says:
    August 22nd, 2009 at 6:04 am

    @Brian – Thanks for the response. You are spot on with your reply to 3). By Acceptance Testing I did mean a integration or functional end-to-end testing framework although since it’s just a library that is being developed, not an application, there will not be a user interface so normal UI testing frameworks probably won’t be of any help. It’s probably more likely that the integration/functional tests are executed using a unit testing library such as FlexUnit.

    In terms of our IDE choice; the library needs to be delivered in a reasonably short space of time so I wouldn’t think developing using FB4 would be acceptable since it’s still in beta.

    I’ll email you if I have any further in-depth queries.

    Thanks again for your help.

  13. Rich K Says:
    January 2nd, 2010 at 2:50 am

    @Brian Pete is right about the mock4as source, e.g, you can find the Mock class here: http://code.google.com/p/mock4as/source/browse/trunk/src/org/mock4as/Mock.as

  14. Brian LeGros Says:
    January 2nd, 2010 at 9:04 am

    @Rick K – You’re right. I guess I had a caching issue with my browser or I’m just a moron and I missed it :) I’m gonna go for the latter since it’s more likely. Thanks for the heads up.

Add A Comment