When you remove a virtual hard disk drive from within Virtual Box, the file itself is not removed. Even worse, deleting the file manually and then trying to add a new virtual disk with the same file name in the same path will just yield an error. Virtual Box complains about the file name already being associated with a specific UUID.
All your virtual hard disk drives are stored in the global Virtual Box configuration. You can retrieve a full list using the following command:
vboxmanage list hdds
This will give you the file names and UUIDs to each virtual disk. In order to remove an entry from the list and make the file name available again, simply execute the following command:
vboxmanage closemedium disk <uuid> --delete
After that, both the file and the UUID entry in the configuration should be be purged from the list and you can create a new virtual hard disk drive with the same file name again.
This was very helpful, thanks.