Http request-response with compressed body in Python
I just want to show how to ask to a server for a compressed representation of a resource.
In this specific example I want to download the gzipped form of Google's homepage.
The first thing that I have to do is to set my request's header and state that I want a compressed representation of the resource that i'm asking for (thinking in a REST way). So I have to add the gzip representation in my 'Accept-encoding' header field.
Then when I'll receive my response I have to deflate the body of my response. If you look at my code I'm checking whether the response body is actually compressed or not. That's because the server might not be able to deliver the compressed representation of the resource that I'm asking for. In that case I'll deliver straightforward the response body.
import urllib2
from StringIO import StringIO
import gzip
request= urllib2.Request('http://www.goolge.com')
request.add_header('Accept-encoding', 'gzip,deflate')
response= opener.open(request)
if response.info().get('Content-Encoding') == 'gzip':
buffer = StringIO( response.read())
deflatedContent = gzip.GzipFile(fileobj=buffer)
return deflatedContent.read()
else:
return response.read()
Written by Michele Capra
Related protips
1 Response
I just got a tip from @sammyrulez that you could also use Requests (http://www.python-requests.org/en/latest/community/faq/) that support automatic GZip decompression.