HTTP Caching in Symfony2 - max-age, ETag & gzip

#php

#apache

#gzip

#symfony

#etag

#http cache

#max-age

Symfony is cool, HTTP cache is cool, so HTTP cache in Symfony is even cooler. Still estimates went south when we implemented max-age and ETag. By we I mean myself, Václav Novotný and Jenkins.

max-age

Always set max-age directive. Don't expect the max-age is set. Following example from Symfony's documentation looks reasonable, but Jenkins (Ubuntu, Apache 2.4, Symfony 2.5.6) returned Cache-Control: public which caused that browser loaded page from cache without asking the app! The code from documentation didn't work for us:

// create a Response with an ETag and/or a Last-Modified header
$response = new Response();
$response->setETag($article->computeETag());
$response->setLastModified($article->getPublishedAt());
$response->setPublic();
// Check that the Response is not modified for the given Request
if ($response->isNotModified($request)) {
    return $response;
}

ETag

Make sure that ETags works as you expect, especially if you use gzip. Tom Panier found the -gzip problem before us. For example you set ETag ABCDE. But when you look at network sniffer you see header ABCDE-gzip. Guess what happens when browser sends this header:

If-None-Match: "ABCDE-gzip"

Of course response will be generated once again (isNotModified returns false). Tom Painer's solution didn't work for us (maybe because setEtag internally use similar code?). The problem is matching ETags from Request. We decided to fix the Request:

function fixETag(Request $r)
{
    $oldETag = $r->headers->get('if_none_match');
    $etagWithoutGzip = str_replace('-gzip"', '"', $oldETag);
    $r->headers->set('if_none_match', $etagWithoutGzip);
}

Replacing substring is ugly hack, but it's the simplest solution that solved our problem:

/** @dataProvider provideIfNoneMatchHeader */
public function testShouldFixETagHeader($header)
{
    $r = new Request();
    $r->headers->set('if_none_match', $header);
    fixETag($r);
    assertThat($r->getETags(), hasItemInArray('"094b34ae543043a951185b2c7c0f145b"'));
}

public function provideIfNoneMatchHeader()
{
    return array(
        'pure' => array('"094b34ae543043a951185b2c7c0f145b"'),
        'gzipped' => array('"094b34ae543043a951185b2c7c0f145b-gzip"'),
    );
}

Who caused the troubles? Symfony, Apache or we?

No mention about appending ETag in RFC2616, so it's probably not Symfony's fault. We tell Apache that we will care about generating ETag (FileETag None). Still Apache modifies our ETag, but it does not modify If-None-Match header. If you know why, please share explanation/solution in comments. We are looking forward to delete str_replace hack!

Why and how we generate ETags?

ETags are powerful, because calculating Last-Modified header can get ugly when you try to involve all possible events that should cause cache invalidation. Except the obvious reasons like changing content, we also want to invalidate cache when:

new version is deployed, so user should use the latest CSS style
user logged in, so he can see Edit link in menu
last modified date is changed (just to be sure, what if Apache unsets Last-Modified header :)

Generate and test such “complex” ETag is very transparent:

use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Security\Core\Role\Role;
use Symfony\Component\Security\Core\Authentication\Token\TokenInterface;

class ResponseFactory
{
    private $repository;
    private $appVersion;

    public function __construct(ModificationDatesRepository $r, $appVersion)
    {
        $this->repository = $r;
        $this->appVersion = $appVersion;
    }

    // token is retrived from "@security.context".getToken()
    public function cached(Article $a, TokenInterface $user)
    {
        $lastModified = $this->repository->findLastModifiedDate($a);
        $r = new Response();
        $r->setPrivate();
        $r->setMaxAge(0);
        $r->headers->addCacheControlDirective('must-revalidate', true);
        $r->setLastModified($lastModified);
        $r->setETag($this->getEtag($a, $lastModified, $user));
        return $r;
    }

    private function getEtag(Article $a, \DateTime $lastModified, TokenInterface $user)
    {
        $source =<<<STRING
            {$lastModified->format('c')}
            {$this->rolesToString($user)}
            {$this->appVersion}
            {$a->getTitle()}
            {$a->getContent()}
STRING;
        return md5($source);
    }

    private function rolesToString(TokenInterface $user)
    {
        return array_reduce(
            $user->getRoles(),
            function ($previousRoles, Role $r) {
                return $previousRoles . ',' . $r->getRole();
            },
            ''
        );
    }
}

Don't sniff the network (and install multiple webservers in X versions) to verify that ETag is different when you deploy new version of your app. Isolated tests are good enough: