Using Groovy Extension Modules with Gradle Shadow

December 7, 2014

John Engelman's Gradle Shadow plugin is a useful tool for building Groovy applications into single executable JAR files. At CommerceHub, we use it to build many of our Dropwizard service deployment artifacts. Groovy extension modules are a handy way to add methods to existing classes in a way that is compatible with type checking and static compilation, and we make use of them as well. Back in August, we ran into a bit of a snag, though: we discovered that the extension module descriptor file in one of our services built with the Shadow plugin was malformed.

The issue had to do with the fact that the groovy-all distribution itself contains extension modules. When Shadow builds a JAR, it extracts the contents of all of the project's JAR dependencies and places them together in the final JAR. Since multiple JARs may contain a given service file, Shadow includes various transformers that can be used to merge incoming service files together. The transformer used when you call mergeServiceFiles() in the shadowJar task's configuration merges service files by concatenation. Extension module descriptor files are Properties files, so that isn't what we need. The behavior of loading a Properties file containing duplicate property keys is undefined.

For example, with this shadowJar task configuration:

shadowJar {
    mergeServiceFiles()
}

Merging the application's extension module descriptor:

moduleName=my-module
moduleVersion=1.0
extensionClasses=foo.groovy.FooExtension,foo.groovy.BarExtension
staticExtensionClasses=foo.groovy.FooStaticExtension,foo.groovy.BarStaticExtension

With the groovy-all descriptor:

# This is a generated file, do not edit
moduleName=groovy-all
moduleVersion=2.3.6
extensionClasses=org.codehaus.groovy.jsr223.ScriptExtensions,org.codehaus.groovy.runtime.NioGroovyMethods,org.codehaus.groovy.runtime.SqlGroovyMethods,org.codehaus.groovy.runtime.SwingGroovyMethods,org.codehaus.groovy.runtime.XmlGroovyMethods
staticExtensionClasses=org.codehaus.groovy.jsr223.ScriptStaticExtensions

Resulted in the following in the shadow JAR:

moduleName=my-module
moduleVersion=1.0
extensionClasses=foo.groovy.FooExtension,foo.groovy.BarExtension
staticExtensionClasses=foo.groovy.FooStaticExtension,foo.groovy.BarStaticExtension
# This is a generated file, do not edit
moduleName=groovy-all
moduleVersion=2.3.6
extensionClasses=org.codehaus.groovy.jsr223.ScriptExtensions,org.codehaus.groovy.runtime.NioGroovyMethods,org.codehaus.groovy.runtime.SqlGroovyMethods,org.codehaus.groovy.runtime.SwingGroovyMethods,org.codehaus.groovy.runtime.XmlGroovyMethods
staticExtensionClasses=org.codehaus.groovy.jsr223.ScriptStaticExtensions

To remedy the situation, two changes needed to be made to the Shadow plugin. First, we needed to implement a new transformer specifically for handling extension module descriptors. After discussing the issue with John, we realized we also needed a way to prevent the mergeServiceFiles() transformer from processing Groovy extension module descriptors. I was able to implement the new transformer by adapting the FatJar plugin's solution. Adding support for include/exclude patterns to the mergeServiceFiles() transformer was a bit trickier, primarily due to my lack of previous experience working on a Gradle plugin. In the end, I was able to work it out.

As of version 1.1.0, Shadow supports specification of include/exclude patterns when using mergeServiceFiles(). Groovy extension module descriptors are excluded by default. In addition, extension module descriptors can be merged with mergeGroovyExtensionModules().

So, with this shadowJar task configuration:

shadowJar {
    mergeServiceFiles()
    mergeGroovyExtensionModules()
}

The resulting file in the shadow JAR for the previous example becomes:

#Sun Dec 07 20:08:39 EST 2014
staticExtensionClasses=foo.groovy.FileStaticExtension,foo.groovy.ObjectIdStaticExtension,org.codehaus.groovy.jsr223.ScriptStaticExtensions
extensionClasses=foo.groovy.BlobExtension,foo.groovy.StringExtension,org.codehaus.groovy.jsr223.ScriptExtensions,org.codehaus.groovy.runtime.NioGroovyMethods,org.codehaus.groovy.runtime.SqlGroovyMethods,org.codehaus.groovy.runtime.SwingGroovyMethods,org.codehaus.groovy.runtime.XmlGroovyMethods
moduleName=MergedByShadowJar
moduleVersion=1.0.0


Older posts are available in the archive.