Set language for a GitHub Gist

Lately I noticed my GitHub Gists were being saved as plain text files. Intuitively I click edit, change the language to something appropriate before updating the Gist.
Only to find nothing has changed and my Gist is still in plain text.

I was stumped.

After some digging it turns out you need to give your filename a proper extension and automagically the correct language is set.