If you run a busy Drupal site, there's a good chance you've set up the usual performance boosts: Varnish, Memcache, APC. If not that, maybe you host on Pantheon or Acquia where they take care of all that stuff for you. What if it is a smaller site, or a hobby site, or what if you're on gasp shared hosting?
I think the best bang for the buck in Drupal caching performance comes with File Cache (https://drupal.org/project/filecache) a module that takes the page cache out of the database and puts it on the filesystem.
The problem with Drupal's default caching mechanism is that a cached page request will be read from the database. The database is already busy storing all the content, configuration, session and user data. Anything you can offload from the database is an improvement. What's more, the cache has to get in the database in the first place. That means an insert operation for each cached page... Expensive.
Reportedly it gives a boos in speed close to that of Memcache, but it's a lot easier to set up. I've seen response times approximately 35% quicker. That's pretty good and if your site is busy, it means you can server more requests without slowing down.
Installation is easy... just install the module and add a couple of lines to your settings.php. See the module's README.txt