This is useful if you start in a folder (e.g. vi ~/Sites/myproject/app/controllers/), and then select a file and want to get back to the folder browse window, simply use:
Ctrl + o
<c-o> (as well as <c-i>) is actually for jumping. It won't always take you to the previous buffer, because it is based on a jump list. You jump when you perform a search, or move around with marks, scroll block wise with braces, use % key... and when switching buffers. The complete list of "jump commands" is in the doc
So this trick can work, only if your last jump was a buffer switching (example your describe). But I think the most efficient way to switch from one buffer to another is use :bp and :bn -- or map them to something more comfortable.