Spring RestTemplate and Gzip compression (continued)

Last month I blogged about the lacking gzip support in the Spring Android RestTemplate. Some collegues at my current client were struggeling invoking gzip encoded RESTful services using Spring 3.0 Web MVCs RestTemplate. While searching the web they found my blog post about Spring RestTemplate and the gzip troubles and decided to ask me for some help ;-) In this article I will explain how to unmarshall a gzip encoded response from RESTful service that uses XML messages.

gzip encoded content

When invoking RESTful services from a client you typically specify in the client request, using HTTP request headers, what the Accept type is (JSON or XML) and content encoding you support as a client. The RESTful service inspects the HTTP request headers and returns the requested content (if supported). This proces is called content negotiation. Have a look at a sample request below:

Not all services support different types of content encoding and/or return types. Sometimes you find yourself in a situation that you have to invoke services that return specific content types or encoding formats, for example compressed content using gzip compression.

In the HTTP Accept-Encoding header you specify the content encoding you are supporting. When invoking services that use gzip encoding you need to specify the Accept-Encoding: gzip,deflate in the request headers. Spring RestTemplate using commons-httpclient does not add this by default and disallows you to invoke services that requires a client supporting gzip encoding, this typically results in a HTTP 400 Bad Request. In this blog I described an option to solve this problem in the RestTemplate, but thats only part of the solution.

Guess what happens when you invoke a RESTful service using Spring Web MVCs RestTemplate and the content body contains gzip encoded content? You are out of luck… or not? Read on… ;-)

Gzip Enabled MarshallingHttpMessageConverter

The RESTful service my collegues were invoking returns XML data in the response body. So the return value of the RESTful service is an XML message, this message will be unmarshalled using an XML unmarshaller like JAXB. The way to do this using Spring Web MVCs RestTemplate is by adding a MessageConvertor called MarshallingHttpMessageConverter to your RestTemplate and you are good to go.

The MarshallingHttpMessageConverter takes care of marshalling and unmarshalling messages passing between the client and RESTful service. But if your service returns gzip encoded content this is not enough. Because the text/xml content of the response body is gzip encoded we first need to deflate the content before JAXB2 is able to unmarshall the content. The response below contains response content that is gzip encoded. The tool I used, SoapUI, already deflated the content for me, but when invoking with the RestTemplate you have to do this yourself:

I created a GzipEnabledMarshallingHttpMessageConverter that supports both gzip and non-gzip encoded content. This message convert only works for XML messages that need to be (un)marshalled. For JSON services you need to implement a different solution. In the MessageConverter I perform a check to see if the Content-Encoding actually is gzip encoded, if so, I use a GZipStreamSource to read the Gzip compressed content from the InputStream of the response body.

The implementation below is a combination of the Spring MarshallingHttpMessageConverter and the AbstractXmlMessageConverter. I had to extend the AbstractHttpMessageConverter because the StreamSource is created within this class.


package com.oudmaijer.rest.spring.xml.message.converter;

import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.AbstractHttpMessageConverter;
import org.springframework.http.converter.HttpMessageNotReadableException;
import org.springframework.http.converter.HttpMessageNotWritableException;
import org.springframework.oxm.Marshaller;
import org.springframework.oxm.MarshallingFailureException;
import org.springframework.oxm.Unmarshaller;
import org.springframework.oxm.UnmarshallingFailureException;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;
import org.springframework.util.Assert;

import javax.xml.transform.Result;
import javax.xml.transform.Source;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import java.io.IOException;
import java.util.List;
import java.util.zip.GZIPInputStream;

/**
 * Add gzip decompression support for HttpResponse body's that are encoded with gzip.
 * <p/>
 * Implements the AbstractHttpMessageConverter in the same way as
 * the Spring AbstractXmlMessageConverter and adds the MarshallingHttpMessageConverter
 * functionality to it.
 *
 * @param <T>
 */
public class GzipEnabledMarshallingHttpMessageConverter<T> extends AbstractHttpMessageConverter<T> {

    private final TransformerFactory transformerFactory = TransformerFactory.newInstance();
    private Marshaller marshaller;
    private Unmarshaller unmarshaller;

    /**
     * Protected constructor that sets the {@link #setSupportedMediaTypes(java.util.List) supportedMediaTypes}
     * to {@code text/xml} and {@code application/xml}, and {@code application/*-xml}.
     */
    public GzipEnabledMarshallingHttpMessageConverter(Jaxb2Marshaller marshaller, Jaxb2Marshaller unmarshaller) {
        super(new MediaType("application", "xml"), new MediaType("text", "xml"), new MediaType("application", "*+xml"));
        Assert.notNull(marshaller, "marshaller must not be null");
        Assert.notNull(unmarshaller, "unmarshaller must not be null");
        this.marshaller = marshaller;
        this.unmarshaller = unmarshaller;
    }


    @Override
    public final T readInternal(Class<? extends T> clazz, HttpInputMessage inputMessage) throws IOException {

        StreamSource streamSource = null;
        boolean gzip = false;
        HttpHeaders headers = inputMessage.getHeaders();
        List<String> contentEncoding = headers.get("Content-Encoding");

        if( headers != null && contentEncoding != null ) {
            for(String contentEncodingString : contentEncoding) {
                if( contentEncodingString != null && "gzip".equalsIgnoreCase(contentEncodingString.trim()) ) {
                    gzip = true;
                }
            }
        }

        if( gzip ) {
            streamSource = new GZipStreamSource(inputMessage.getBody());
        } else {
            streamSource = new StreamSource(inputMessage.getBody());
        }

        return (T) readFromSource(clazz, inputMessage.getHeaders(), streamSource);
    }

    @Override
    protected final void writeInternal(T t, HttpOutputMessage outputMessage) throws IOException {
        writeToResult(t, outputMessage.getHeaders(), new StreamResult(outputMessage.getBody()));
    }

    /**
     * Set the {@link Marshaller} to be used by this message converter.
     */
    public void setMarshaller(Marshaller marshaller) {
        this.marshaller = marshaller;
    }

    /**
     * Set the {@link Unmarshaller} to be used by this message converter.
     */
    public void setUnmarshaller(Unmarshaller unmarshaller) {
        this.unmarshaller = unmarshaller;
    }


    @Override
    public boolean supports(Class<?> clazz) {
        return this.unmarshaller.supports(clazz);
    }

    /**
     *
     * @param clazz
     * @param headers
     * @param source
     * @return
     * @throws IOException
     */
    protected Object readFromSource(Class<?> clazz, HttpHeaders headers, Source source) throws IOException {
        Assert.notNull(this.unmarshaller, "Property 'unmarshaller' is required");
        try {
            return this.unmarshaller.unmarshal(source);
        } catch (UnmarshallingFailureException ex) {
            throw new HttpMessageNotReadableException("Could not read [" + clazz + "]", ex);
        }
    }

    /**
     *
     * @param o
     * @param headers
     * @param result
     * @throws IOException
     */
    protected void writeToResult(Object o, HttpHeaders headers, Result result) throws IOException {
        Assert.notNull(this.marshaller, "Property 'marshaller' is required");
        try {
            this.marshaller.marshal(o, result);
        } catch (MarshallingFailureException ex) {
            throw new HttpMessageNotWritableException("Could not write [" + o + "]", ex);
        }
    }
}

The GZipStreamSource comes from an XML library created by Kohsuke Kawaguchi. I made some minor modifications to the code but the credits for this code goes to Kohsuke.

package com.oudmaijer.rest.spring.xml.message.converter;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.zip.GZIPInputStream;

import javax.xml.transform.stream.StreamSource;

/**
 * {@link StreamSource} that reads from gzip-compressed XML stream.
 * <p/>
 * THIS SOFTWARE IS IN PUBLIC DOMAIN. NO WARRANTY.
 *
 * @author Kohsuke Kawaguchi (kk@kohsuke.org)
 */
public class GZipStreamSource extends StreamSource {

    /**
     * Creates a {@link StreamSource} from a gzip-compressed XML file.
     */
    public GZipStreamSource(File file) throws IOException {
        this(new FileInputStream(file), file.toURI().toURL().toExternalForm());
    }

    /**
     * Creates a {@link StreamSource} from a gzip-compressed stream.
     */
    public GZipStreamSource(InputStream stream) throws IOException {
        super(new GZIPInputStream(stream));
    }

    /**
     * Creates a {@link StreamSource} from a gzip-compressed stream.
     * <p/>
     * <p>This constructor allows the systemID to be set in addition
     * to the input stream, which allows relative URIs
     * to be processed.</p>
     */
    public GZipStreamSource(InputStream stream, String systemId) throws IOException {
        super(new GZIPInputStream(stream), systemId);
    }

    /**
     * Creates a {@link StreamSource} from a gzip-compressed stream.
     *
     * @param url
     * @throws IOException
     */
    public GZipStreamSource(String url) throws IOException {
        this(new URL(url).openStream(), url);
    }
}

Spring configuration

And finally you have to configure your RestTemplate to use the new MessageConverter.

<bean id="httpClientFactory" class="org.springframework.http.client.CommonsClientHttpRequestFactory">
  <constructor-arg ref="httpClient"/>
</bean>
<bean id="restTemplate" class="org.springframework.web.client.RestTemplate">
  <constructor-arg ref="httpClientFactory"/>
  <property name="messageConverters">
    <list>
    <bean class="com.oudmaijer...GzipEnabledMarshallingHttpMessageConverter">
      <constructor-arg ref="jaxbMarshaller"/>
      <constructor-arg ref="jaxbMarshaller"/>
    </bean>
    ...
    </list>
   </property>
</bean>

Conclusion

Using this MessageConverter you can invoke RESTful services that return gzip encoded content. Lets hope Spring adds it to the default message converters in the future because gzip compressed content is an elegant solution when you need to return large messages from a RESTful service.

One Response to “Spring RestTemplate and Gzip compression (continued)”

  1. Me says:

    Great info, thanks. Perhaps this approach is now baked into RestTemplate (2013).

Leave a Reply