FlexAbility

There have been a couple of questions on the Google FlexUnit list recently about how to structure a project for unit testing. I’m putting my response here so that I can point others at this post in the future.

If you’re interested in doing testing as part of an automated build script, one of the best resources is Antennae. Antennae is an Allurent open source project and does much more than automated testing (compiling, etc.) but it provides a good example of one way to structure a project.

Also, I use a “modified antennae” approach for the Visual FlexUnit (VFU) project. Wheres Antennae tends to use Allurent-standard folder naming (“flex2″, etc.), I decided to use more standard folder names (“src”, etc.) for VFU, so that non-Allurent users would find it a bit easier to incorporate into their projects.

BTW, you might also want to take a look at my Testing, Debugging & Agile Methods page – you’ll find a lot of related links there, and I update it regularly.

HTH!

I was recently asked to write up a summary pointing out some of the more interesting bits of Visual FlexUnit‘s code for a code review by a potential employer. As I wrote this I realized that I might as well dual-purpose these notes as a blog post. If you’re curious about how VFU works, read on.

Unless otherwise specified, all code is in VFU’s dev/vfu/visualflexunit/src/flex/ folder.

1. com.allurent.flexunit2.vfu.flexui.VfuTestRunner – This file includes comments that talk about VFU’s ‘tacked-onto-FlexUnit’ nature, resulting kludgishness, and potential next-step refactoring.

2. Take a quick look at VFU’s documention. I don’t suggest that you read the whole thing but, in particular, Step 2.B contains a sample test method that shows the sort of thing users would be writing.

3. com.allurent.flexunit2.vfu.framework.testsequence.
TestSequenceManager – One of the primary challenges of writing VFU was the combined result of these factors:

• We wanted test writers to be able to write clear, concise test methods like the sample method I just mentioned.
• Such methods inevitably result in multiple asynchronous processes, e.g. component rendering & file reading.
• FlexUnit has a mechanism for handling asynchronous processes, but can only handle one such process per test method.

A poor solution to this set of factors would be to ask test writers to write a chain of methods for each test, linked together with FlexUnit’s addAsync() mechanism. Ack!

Instead I created this manager class that creates and manages a queue of command objects, as requested by the user’s test code.

4. You might also want to take a look at some of the command classes in com.allurent.flexunit2.vfu.framework.testsequence.commands, especially AssertComponentMatchBaseline.

5. AssertComponentMatchBaseline includes a _judgmentStrategy:IBitmapMatchJudge private variable. The strategy object stored here is responsible for deciding whether the rendered component’s bitmap is close enough to a stored baseline bitmap to be visually indistinguishable. By default this variable holds an instance of com.allurent.flexunit2.vfu.framework.strategies.DefaultBitmapMatchJudge. Test writers have the option to write their own alternative strategies and pass them in. A simple example of using composition to allow flexibility.

6. /dev/vfu/vfuexampleTest/build.xml – Shows how we simply override a single Ant property in order to run automated visual tests. The build process is identical to the process used when running standard FlexUnit tests, except that we tell it to use a different MXML app template. Marty Frenzel showed me this elegantly simple approach.

7. /dev/tools/build-common-targets.xml – For the most part this is slightly-modified Antennae code. I added the build-visualflexunit and test-visualflexunit targets.

8. /dev/vfu/visualflexunit/template/testprojecttemplate/build.xml – VFU includes a bootstrap mechanism which makes it easy for users to create the framework for visual test projects. This file’s bootstrap-visual-test-project target copies files and uses Ant’s token replacement mechanism to insert appropriate project, file and folder names.

Visual FlexUnit (VFU) is a FlexUnit extension that allows you to do automated testing of components’ visual appearance using “visual assertions”. In a nutshell, a visual assertion asserts that a component’s appearance is identical to a stored baseline image file. These visual tests can be run in either a GUI or an Ant-based build process.

I created VFU during my recent internship at Allurent. We’re releasing it as open source on Google Code under the MIT license. Its home page is here.

Why should you be interested in VFU? Here’s an extract from its main documentation page:

Why Is This Useful?

Regression Testing

By adding visual tests to your build process you can be assured that any changes to your codebase, or to the Flex/Flash framework, that effect the appearance of your components will be noticed immediately.

Testing For Cross-Platform Rendering Differences

While Adobe has done a great job of making Flash & Flex a write-once / deploy-anywhere platform, there may be differences in how images render:

• Between the Flash Player and the AIR runtime
• Between Windows, Mac, and Linux
• And, of course, between the many combinations of these variables

To date our limited testing has only detected a) differences that aren’t visually noticeable, and b) differences in very unusual cases that you would never use in the real world. Still, it doesn’t hurt to check. By running regression tests on multiple platforms you can be assured that any unseemly differences in component rendering will be brought to your attention quickly.

(end of extract)

We invite you to play with VFU and would love to get your feedback. You can either enter issues into Google Code or add a comment to this post. While it looks as though this could be a very useful tool, at this point we haven’t worked with it enough to assert that visual assertions belong in your toolkit. Let us know what you think!

The following people contributed greatly to this project, and made my work on VFU both easier and extremely educational. Thank you!:

• Joe Berkovitz
• Marty Frenzel
• Daniel Rinehart
• Tim Walling

Finally, I should mention that Allurent is an extremely smart and competent company, at the forefront of Flex development, and a really nice group of people. I learned an incredible amount working there, including a great deal about using Ant to implement automated build processes. (I’ll be blogging shortly about the Ant build process that I’ve included with VFU.) If you get an opportunity to work with Allurent, I suggest that you grab it.

It’s been a goal of mine for some time to learn how to implement unit testing, test driven development and continuous integration. I’ve understood these concepts in the abstract but, as a freelance programmer working on my own, the barriers to entry have been a bit too high for me to take the plunge.

With Allurent‘s release of their Antennae Ant-based build framework all this has changed. Antennae provides a framework that automates the entire build/test/deploy process. Its use can speed you on your way to one of the holy grails of software development – the one-click build – for your Flex projects.

Here are a few of its features:

• You can build multiple projects with, of course, one click
• Projects can contain sub-projects
• Some of your projects can be code libraries, and you can specify which libraries each project uses.
• Antennae compiles projects using mxmlc, and libraries using compc…
• And runs unit tests using FlexUnit

Above I said that Antennae will ‘speed you on your way’. Well, speed is a relative thing. If you’re fairly new to Ant (as I was) you’ll still have some studying to do. I invested a good deal of time studying the Ant manual and Antennae’s code. What Antennae did for me was to move my goal from the realm of “I’d like to do this, but where do I start?” to “I can do this!”

As part of my process I decided that I should get a good overview of Ant. I learn best by taking notes so I ended up developing – not precisely an ‘Ant cheat sheet’ – but more of an ‘Ant reminder sheet’. Simply stated, it’s a list of much of what Ant can do in ‘somewhat organized’ format. I don’t know how helpful it would be to others, but you’re welcome to take a look. Here’s a download link (right click & “save link as”):

AntRef.doc

It’s a Word doc consisting of three pages of fairly fine print, which says something about Ant’s extensive capabilities. Also, be aware that it’s oriented towards Flex developers and omits some of Ant’s extensive Java-related capabilities.

I’m still far from an Ant expert but at least I’ve got an overview. And I’m happy to report that I’ve now implemented the Antennae framework for my own projects and am happily doing one-click builds.

Paul will be presenting at the Bay Area ColdFusion User Group and the meeting will be broadcast live via Acrobat Connect. Meeting time is 7:00 PM PDT, Wednesday May 16. The URL for the broadcast should be available on the group’s site on the day of the meeting. Here’s Jennifer Larkin’s post on the meeting.