Sometimes (not very often), you find yourself in a position where you need to delete a tag from your local repository and a remote repository. In my professional life, this has happened twice so far.
First you want to make sure that you remove the tag from your local repository by doing the following:
git tag -d v1.2.3
This will delete your tag v1.2.3 from the local repository.
Then you want to do a push with :refs to delete it from your remote repository.
git push origin :refs/tags/v1.2.3
This is a special push where you have no content and a empty tag which will delete it.
Hope this helper someone!