Deployment of static files in Django

Static files of web applications are all stylesheets, javascripts, images, fonts, etc. Since these files are static, they can be served from cache or CDN (Content delivery network) and speed up the load time of a webpage. However, caching of static files either at server side or client side has one major problem: Cache invalidation.

Long live to files in cache

Each browser caches static assets of webpage. Using max-age header we can instruct browsers for how long they should keep our files in their cache. The best performance is achieved when the file is cached indefinitely, requiring only one request to webserver for each such file.

This is great, because user loads all static assets of our webpage at the first visit and each subsequent request is blazingly fast. However, when we change our CSS or JS, we are screwed, because browser will not download the newer file. We can't even instruct browsers to invalidate their caches, only users can.

The solution is simple: rename files every time they change. So when we append MD5 or SHA1 hash to the original filename, it will be unique after we change it. Now we only need to update references in HTML, references in stylesheets (like url properties) and maybe somewhere else in our codebase… Let's machines do their work!

The Django way

Fellow developers of Django already invented this wheel. Since Django 1.7, there is ManifestStaticFilesStorage, which renames files and it's references every time we run collectstatic. The name of the latest static file is stored in staticfiles.json manifest, so when we call {% static 'my/file/latest.css'%} we get something like: http://static.cdn/my/file/latest.a1b2c3.css (which is combination of STATIC_URL and hashed filename).

This solution is perfect because of its simplicity. Most projects already use static tag for finding static file names, which is more flexible than prefixing STATIC_URL. ManifestStaticFilesStorage handles renames automatically when we run collectstatic, so actually this approach solves our problem with minimum effort. There's only a minor caveat though: what happens during deployment of new release?

Chicken or the egg

The staticfiles.json manifest is overwritten when we call collectstatic. If we collect static files before the server is restarted, our old webpage will load new static files and some things might break. If we reverse the order and collect static files after the server is reloaded, the new webpage will load old static files and again, some things might break.

Since we have versioning for static files, we can add versioning for staticfiles.json manifest as well. We can simply append the app version which is being bumped during every release. Then we can safely collect static files during release process and new static files will be used right after the process is reloaded.

If we add __version_static__ to __init__.py of our project, we can use following static files storage:

from django.contrib.staticfiles.storage import ManifestFilesMixin
from storages.backends.s3boto import S3BotoStorage

import lingui

class StaticRootS3BotoStorage(ManifestFilesMixin, S3BotoStorage):
    """
    Appends version of static files (git commit sha)
    to static files manifest before uploading to S3.
    """
    location = 'static'

    @property
    def manifest_name(self):
        filename = 'staticfiles-{version}.json'
        return filename.format(version=lingui.__version_static__)

The __version_static__ is updated automatically every time the static files are released with short git commit hash. It differs from project version, because not all code releases requires also releasing of statics. Since running collectstatic could take some time, this is a little speedup of release process.