In your directory, type the following :
The hard way
git rm -r --cached .
The soft way
git rm --cached filename
This will remove everything (hard way) or only the specified file (soft way) from your index (not your actual files, don't worry)
git add -A git commit -m "yeah my .gitignore is now working \o/"