Here's a quick (and in hindsight, obvious) tip for when your factories start misbehaving and you're tearing your hair out trying to determine the root cause.
Include FactoryGirl and its related gems in your Gemfile's development group:
group :development, :test do
Now when you fire up rails console you have access to all your factories and can debug them as needed. This technique is additionally useful if you need to create some populated, dummy data.
However, if you want to ensure that the data isn't saved during your debugging session, be sure to start up console in sandbox mode:
rails c --sandbox