GZIP Compressed Web Requests in WP7 - Take 2

I earlier wrote about how .NET API in Windows Phone 7 doesn’t zip-compress it’s web requests, and how using Mango’s socket support allows you to circumvent that to get more efficient use of the limited data connection phones often have. The approach had some problems and limitations, and wouldn’t work in all scenarios. Even so, compressing your webrequests is very important on a phone since you can often save 50-80% of the data transmitted, and many people pay by the byte they transmit.

Since my blogpost Windows Phone 7 Mango has gone through a few iterations, and using a socket is no longer necessary, since the “accept-encoding” header needed to request GZIP compressed requests is no longer blocked (thank you to those who voted for getting this unblocked). However, GZIP compression is still not a built in feature (Why not Microsoft?!?). If you set this header, you are still in charge of uncompressing the content. I’ve updated my GZipWebClient class to take advantage of the new allowed header. It simplifies the class quite a lot, as well as made it more stable.

Also I “forked” the DotNetZip library and included in this project, so you don’t need to go find 3rd party dependencies, and in the process removed all parts of the library that is not needed to keep the assembly as compass as possible. So here’s what the client now basically looks like:

public class GZipWebClient : WebClient
{
    [SecuritySafeCritical]
    public GZipWebClient()
    {
    }
    protected override WebRequest GetWebRequest(Uri address)
    {
        var req = base.GetWebRequest(address);
        req.Headers[HttpRequestHeader.AcceptEncoding] = "gzip"; //Set GZIP header
        return req;
    }
    protected override WebResponse GetWebResponse(WebRequest request, IAsyncResult result)
    {
        WebResponse response = null;
        try
        {
            response = base.GetWebResponse(request, result);
            if (response.Headers[HttpRequestHeader.ContentEncoding] == "gzip")
                return new GZipWebResponse(response); //If gzipped response, uncompress
            else
                return response;
        }
        catch
        {
            return null;
        }
    }
}

The key here is adding the header that tells the server “Hey I understand GZipped responses as well”. When the response comes back and the header says “Hey here’s some gzipped content”, I use a custom GzipWebResponse class that uses the DotNetZip library to uncompress the stream. Apart from some trivial code, this is the meat of that class:

internal class GZipWebResponse : WebResponse
{
    WebResponse response;
    internal GZipWebResponse(WebResponse resp)
    {
        response = resp;
    }
    public override System.IO.Stream GetResponseStream()
    {
        return new SharpGIS.ZLib.GZipStream(response.GetResponseStream()); //Uncompress
    }
}

You use it exactly like the normal “WebClient” class, except you instantiate the GZipWebClient instead. The rest of your existing code doesn’t change at all. Example:

WebClient client = new SharpGIS.GZipWebClient();
client.DownloadStringCompleted += client_DownloadStringCompleted;
client.DownloadStringAsync(uri);

You can download the source code below, and you can also get it on NuGet/SymbolSourceusing command “Install-Package SharpGIS.GZipWebClient”

Download source

I also rely heavily on this in my WinPhone Twitter client ‘Peregrine’ and have seen quite a performance improvement when updating your twitter timeline. You can download Peregrine for free here: http://www.windowsphone.com/en-us/apps/75067abc-c9d1-47b7-8ace-76aede3911b2?wa=wsignin1.0

UPDATE!!!

With the latest v1.1 update on nuget (source also updated here), you don’t have to do the above. All you need to write is the following two lines of code and put it in your app.xaml.cs startup file. After this ALL existing code (including 3rd party libraries you don’t have control over) will start using GZIP compression!

WebRequest.RegisterPrefix("http://", SharpGIS.WebRequestCreator.GZip);
WebRequest.RegisterPrefix("https://", SharpGIS.WebRequestCreator.GZip);

If you just want to do this for some domains, only register that prefix part. So no need to go replace all your WebClient as mentioned above. More on how this work in this blogpost.

UPDATE2!!!

Note that using RegisterPrefix causes this to use an internal custom HttpWebRequest class and that can have some serious effects on your app. Unfortunately Microsoft ‘forgot’ to unlock the Get/Set UserAgent property when they ported from Silverlight (there the user agent was always the browser), and many 3rd party libraries  tries to set this property, which will throw a NotImplementedException. Also note that cookies are not supported either (some libraries use that). They should of course be checking the “SupportsCookieContainer” property before using it, but not all do (I’m looking at you RestSharp), and will therefore throw an exception. However for almost all the simpler scenarios, the above register works like a charm.

UPDATE3!!!

Now also on GitHub: https://github.com/dotMorten/SharpGIS.GZipWebClient

UPDATE4!!!

Update 2 no longer applies :)

Comments (8) -

  • Hi Morten,

    I really like your GZip package. Specially the WebRequestCreator. Though in combination with some AdControls I'm having trouble. I've seen that they are at least trying to set the UserAgent which fails when using the WebRequestCreator.GZip.

    System.NotImplementedException: This property is not implemented by this class.
       at System.Net.HttpWebRequest.SetUserAgent(String value)
       at System.Net.HttpWebRequest.set_UserAgent(String value)
       at Microsoft.Advertising.Mobile.Shared.HttpRequest.set_UserAgent(String value)
       at Microsoft.Advertising.Mobile.Shared.WebRequestWrapper.StartNextWebRequest()
       at Microsoft.Advertising.Mobile.Shared.WebRequestWrapper.SendAsyncRequest(WebRequestCompleteCallback webRequestCompleteCallback)
       at Microsoft.Advertising.Mobile.Shared.AdPlacement.GetAdvertisementInternal(Boolean notifyOnRefresh, Boolean forceDownload)

    Not sure if there are more things that fail, but this is something I found almost immediately after applying the WebRequestCreator.

    Best,

    Mark Monster
  • Mark: Unfortunately it's impossible to implement "UserAgent" in WP7. For some reason Microsoft forgot to unblock that when they ported from browser-Silverilght. I suggest you limit the domains you register the prefix for.
  • Is there anyway to use this code with a HttpWebRequest and cookies? I read above that it will not work with cookies for some libs, but I was not sure if you meant all libs would not work. Thanks Smile
  • I am finding that the AvailableBytesIn (ZLibStream) is not always 8 and therefore I occasionally get exceptions similar to what is listed in this bug in the DotNetZip Library Codeplex project. It appears that the header length could variable but the library is expecting exactly 8 bytes.

    http://dotnetzip.codeplex.com/workitem/8679

    I looked at the ZLibStream file and the code is the same as the version of the library at the time the bug above was reported.

    It would also be helpful if the ZlibException was not an internal class so that it may be handled in a try catch block.
  • Possible to use your classes with Silverlight?
  • Mickaël: Yes and no. First of all, the browser already does the GZip compression for you, and when you rely on the browser for the requests, you are not allowed to set the content-encoding header (since the browser handles this already and that header is the browser's responsibility). You can get Silverlight to take over the requests from the browser to be able to set that header, but why do that when the browser already does it all for you?
  • Are you sure Silverlight is using gzip? With Fiddler, I can't see any of the request with "Accept-encoding : gzip"...
  • Mickaël : Are you running in-browser or out-of-browser? Again if you are running in-browser, the browser handles all this, and it will be gzip compressed IF the browser supports it.

Pingbacks and trackbacks (4)+

Add comment