Experiences from using Grails to migrate legacy web services
Brian LeGros | August 15th, 2009 | programmingRecently I was tasked with migrating a version of PHP web services to our new and growing Java stack. I had to maintain the current XML interface over HTTP but use our Java services behind the scenes to satisfy the requests. In an effort to make things easier on myself, I decided to use Grails because we were using a fairly typical Java stack of technologies (Spring, Hibernate, JUnit, etc.) that integrate extremely well with Grails’ underlying architecture. Basically, I had everything I needed in terms of persistence, domain, validation, etc in our existing stack, I just had to use Grails to expose it with a specific XML interface. Now that the proof of concept phase is complete for the project, I figured I’d share some of the niceties and challenges I encountered while approaching this problem with Grails.
First off, the expressiveness of Groovy as a language (I used 1.6.3) made the biggest task of mapping a very rigid, existing XML interface to our internal domain much easier and shorter than if I would have written it in Java. The messages were basically a flattening/expanding of our internal object graph with a couple of really quirky translations of data. Being able to treat literals as objects and the use of collection operators (*.) and methods (collect, each, find) saved me quite few lines of code. On the testing side of things, strictly from the perspective of Groovy, the as operator just kicks ass. Since mapping these interfaces was my focus, I didn’t find the need to mock too many expectations from our services, so stubbing out dependencies using maps and closures made testing SO much easier. I did find a down side to stubbing directly in Groovy however. If you are attempting to stub out a class without a no-arg constructor, you’re going to have to use a library like GMock which will allow you to create a stub passing constructor arguments. I’m not too partial to play, rewind type mocking frameworks, so I was a little bummed to have to pull in another resource to test, but it got the job done.
On the Grails side of things, integration with our existing services couldn’t have been easier. We used Grails 1.1.1 for our implementation. With a few tweaks to resources.xml, I was able to reuse our entire set of bean declarations and have them injected by name into our controllers. Although, we didn’t place our Hibernate mapping files in the /hibernate folder, I was able to create my own session factory with relative ease to get Hibernate up and running. Once I had this working, I knew I could eventually take advantage of the real reason I wanted to use Grails, RESTful interfaces over HTTP. The former PHP interface implemented a Front-Page Controller for all of its services, so using URLMappings I was able to emulate index.php to the rest of the world pretty easily. On a side note, the former interface was expecting a request variable name action which Grails reserves, but using a simple closure in UrlMappings allowed me to map it as a new request variable name and go on my way. In any case, since this is a specific endpoint, I am left open to create new controllers using Grails’ REST support as we migrate away from the older interfaces. This also means, from what I understand (thanks implicit GORM support from setting up Hibernate), I can start to use xml property binding to create my domain objects and the encodeAsXml() and encodeAsJson methods to serialize those object across the wire. This is pretty huge for us and has definitely improved my opinion of working with Java as a langauge on the web. It also solidified for me that Grails has matured quite a bit since I last gave it a test run. Lastly, I have to say that I’m really digging the simplicity of functional-testing plugin. Initially I had looked at WebTest, but this plugin is much less intimidating and was dirt simple to get started with.
Let me just stop hear and say that there were quite a few challenges to bringing Grails into our stack as well. The challenges weren’t particularly due to the implementation code itself but more on the supplemental side of things in terms of testing, continuous integration, and IDE support. Coming from an existing Java code base, we already had a set of testing tools we liked based on JUnit 4, JMock, and Spring testing. This included some really productive tools we built around dbunit supporting yaml for integration testing. Currently, Grails only supports JUnit 3, so ideally test cases should extend GroovyTestCase; there is also GrailsUnitTestCase which helps out with the mocking story quite a bit on unit tests for domain classes. Having to use JUnit 3 meant making a decision to go back and refactor testing support to work with both JUnit 4 and JUnit 3 or just rely more heavily on the functional testing support in Grails. Since I was just building controllers and a couple of domain classes, I chose to rely more on functional testing. For what its worth, there is a ticket out there for JUnit4 support. JUnit4 support aside, probably one of the most detracting things from testing with Grails is the tedium involved when testing controllers. Grails will inject a MockHttpServletRequest, MockHttpServletResponse, and MockHttpSession object for use by your controllers in the test case so you check redirected/forwarded urls, what responses will look like, easily create request edge cases, etc. By executing grails test-app -integration you can quickly see what I’m talking about. The injection process is painfully slow, at least on my last gen Macbook Pro, and I feel like I am always sitting, waiting around for these tests. I could have probably thinned down my controllers a bit, but most of the cost of executing these tests was on startup of the overall run. The only other thing I would have like to seen supported is the ability to have multiple test classes for the same domain, service, controller class. The permutations of response messages from each controller action were quite numerous, so I would have liked to create a test case per controller action rather than just one for the entire controller for readability. I only had luck getting a single test case to execute following the Grails naming conventions per object.
On the CI side of things, we build our Java projects using Maven. For large Java applications, we’ve been unable to find a simpler approach to build our software than using multi-module projects in Maven. Grails’ Maven support unfortunately did not work well with our multi-module project although it looks like they’ve made some updates for a quick fix that may help. Until then, I have our project POM hacked together a bit marking the executions of the plugin myself and using the dependency plugin to create a lib directory so I can work with grails outside of Maven w/o issues. Here are some of the relevant changes I made for my POM:
... <build> <plugins> <plugin> <groupId>org.grails</groupId> <artifactId>grails-maven-plugin</artifactId> <version>1.1-SNAPSHOT</version> <extensions>true</extensions> <executions> <execution> <id>grails-clean</id> <goals> <goal>clean</goal> </goals> <phase>clean</phase> </execution> <execution> <id>grails-test-app</id> <goals> <goal>test-app</goal> </goals> <phase>test</phase> </execution> <execution> <id>grails-war</id> <goals> <goal>war</goal> </goals> <phase>package</phase> </execution> </executions> <dependencies> <dependency> <groupId>org.codehaus.groovy</groupId> <artifactId>groovy</artifactId> <version>1.6.3</version> </dependency> </dependencies> </plugin> <plugin> <artifactId>maven-antrun-plugin</artifactId> <executions> <execution> <id>copy-grails-test-results-for-ci</id> <phase>test</phase> <configuration> <tasks> <property name="report.loc" location="${project.build.directory}/surefire-reports" /> <mkdir dir="${report.loc}" /> <copy todir="${report.loc}"> <fileset dir="${project.build.directory}/../test/reports"> <include name="**/TEST-*.xml" /> <exclude name="TEST-TestSuites.xml" /> </fileset> </copy> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution> <execution> <id>copy-grails-war-to-target-for-lifecycle</id> <phase>package</phase> <configuration> <tasks> <copy todir="${project.build.directory}"> <fileset dir="${project.build.directory}/.."> <include name="*.war" /> </fileset> </copy> </tasks> </configuration> <goals> <goal>run</goal> </goals> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-clean-plugin</artifactId> <version>2.3</version> <executions> <execution> <id>clean-dependencies-in-lib-for-clean</id> <phase>pre-clean</phase> <goals> <goal>clean</goal> </goals> <configuration> <filesets> <fileset> <directory>${basedir}/lib</directory> <includes> <include>**/*.jar</include> </includes> </fileset> </filesets> </configuration> </execution> </executions> </plugin> <plugin> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>copy-dependencies-into-lib</id> <phase>generate-sources</phase> <goals> <goal>copy-dependencies</goal> </goals> <configuration> <outputDirectory>${basedir}/lib</outputDirectory> <excludeGroupIds>org.grails,org.codehaus.groovy</excludeGroupIds> </configuration> </execution> </executions> </plugin> ... </plugins> </build> ...
In terms of CI server integration, we use Hudson and the only hiccup I ran into was to make sure the user the app server is running as has rights to create a /.grails folder in its home directory. It would be nice to be able to run the unit and integration tests with lifecycle rather than all at once (GRAILS-4569) as well as have a goal to just run functional tests, but I know they are working on it. It looks like a lot of work has been done with this plugin since the 1.1-SNAPSHOT we were using, so I’m eager to give it another try as we add more web services to the mix.
The last hurdle I had to overcome in using Groovy and Grails was IDE support. Working with Java and Flex, I’ve come to find that Eclipse is my staple development tool. Unfortunately, working with the combination of Maven, Groovy, and Grails, I found myself continually frustrated with the Groovy Eclipse plugin. m2eclipse really didn’t help either since a Grails application directory structure doesn’t match the traditional Maven project layout. In the end I found myself just using the code highlighting and limited refactoring support in Groovy Eclipse and then command-line for everything Grails related. I’d heard a lot about Netbeans 6.7 support for Grails, so I was optimistic, but the experience was more of a honeymoon than anything else. Netbeans has great Maven support, but unfortunately Grails projects in Netbeans can’t take advantage of that support; Netbeans doesn’t seem to have project nature support like Eclipse. I found myself using the built-in context menus for the Grails commands to keep me from going out to the command line, but this was pretty much the only feature I could justify using in Netbeans. The code completion support was far from responsive. I found myself waiting almost 30s in some cases for the “Please wait…” pop-up to only find that the Groovy editor couldn’t tell me anything about my typed/untyped variable other than the defaults. The Subversion client is terrible. When committing, each file in the changeset has a drop-down box for what you’d like to do with the file (exclude, add, commit) and the default is add or commit. If you have a working copy with lots of files you want to exclude (e.g. – artifacts from a build process) you have to one-by-one exclude each file. Also, I’m not sure why but the patch command is not available from the SVN context-menu on the project or files view. I had to hunt through the Team menu under Subversion to finally find the option. Subversive and Subclipse for Eclipse are a reason enough to avoid Netbeans if you’re an SVN user, IMO. Maybe I couldn’t find the hotkey or context menu option, but there also didn’t seem to be the ability to refresh a folder on the project or files view? I noticed that Netbeans was constantly “scanning folders” to find changes on the file system but I had no manual way of just saying refresh. Consequently I had a lot of folder sync’ing issues between the file system and Netbeans in the rare case I needed to use the command line. In the end, a lot of the day-to-day niceties I had become accustom to just weren’t in Netbeans and for the way I was using it, it just wasn’t for me. I had heard really good things about the IntelliJ support for Grails, but due to a license fee, my poor experience with Netbeans, and the fact that Eclipse does everything else I need for development, I ended up just sticking with Eclipse. On a more positive note, SpringSource has released an alpha build of a new Groovy plugin for Eclipse which has some great promise. There is also talk of STS for Eclipse adding Grails support, so I’m hopeful the Groovy/Grails IDE experience can only improve from here.
Overall, despite the challenges I encountered, Grails is a dynamite web framework if you’re already working with Spring and Hibernate in your stack. I’ll acknowledge I may be a complete n00b in terms of the learning curve still, so if I’ve totally missed the boat on some of my points, please let me know so I can update this post. I’m excited to continue using Grails, especially with the newer releases coming up which will including Spring 3.0 and Spring-Flex.