Since March 2015, Compose has supported MongoDB 3.0. One of the nicer features of Compose is its automatic backups. However, when I needed to restore from a backup the other day, it took some back-and-forth with Compose's support team to resolve, and there were a few surprises along the way.
My hope is that this article will be outdated soon, but here's what you need to know today.
The short version
As recommended by the Compose support team:
- Install MongoDB 2.6 on your local machine
- Download and extract the backup
mongodump --dbpath path/to/extracted/dir
mongorestoreusing your Heroku credentials on the newly created
The rest of this article will go into more detail. But first, some key information:
Backups aren't 3.0-friendly
Each of these downloads is in a format that's not easy to import into MongoDB 3.0.
Why MongoDB 2.6?
I was surprised when Compose's support team asked if I could install MongoDB 2.6.
The short version is, you need to produce a set of BSON/JSON files from your backup in order to restore it. Support for this was removed in 2.8:
The 2.8 versions of MongoDB tools,
mongooplog, must connect to running MongoDB instances and cannot modify MongoDB data files (i.e. with
--dbpath) as in previous versions.
1. Install MongoDB 2.6
I'm using Homebrew on OS X. I found it easiest to uninstall mongodb and install it using homebrew-versions.
Important: Make sure you back up before you do this.
Install Homebrew versions, if you don't already have it:
brew tap homebrew/versions
Uninstall your local MongoDB:
brew uninstall mongodb
Install MongoDB 2.6:
brew install homebrew/versions/mongodb26
(Once you're finished, you can run
brew uninstall homebrew/versions/mongodb26 and re-run
brew install mongodb.)
2. Download and extract the backup
Extract tarball. In my case, I ended up with a directory that looked like this:
. ├── admin.0 ├── admin.ns ├── app12345678.0 ├── app12345678.1 ├── app12345678.2 ├── app12345678.3 ├── app12345678.4 ├── app12345678.ns ├── local.0 ├── local.1 ├── local.2 ├── local.3 ├── local.4 ├── local.ns ├── mongod.lock ├── raw_parse.txt └── storage.bson
Inside the directory above, run:
mongodump --dbpath .
--dbpath option is only supported in MongoDB 2.6, hence why we had to install it.
This will produce a
dump directory containing the data:
dump ├── admin │ ├── system.indexes.bson │ ├── system.users.bson │ ├── system.users.metadata.json │ ├── system.version.bson │ └── system.version.metadata.json └── app12345678 ├── my_collection_name.bson ├── my_collection_name.metadata.json └── system.indexes.bson
You'll need your Heroku app's
MONGOHQ_URL environment variable for the credentials:
heroku config:get MONGOHQ_URL
You should get something like this:
Note that there's two hosts and ports listed. In my case, the right one was the one listed second, but you can double-check by looking at the Admin section of the Compose panel (look for “Mongo Console”).
mongorestore should look like this:
mongorestore \ --host candidate.1.mongolayer.com \ --port 12345 \ --db app12345678 \ -u heroku \ -p averylongstring \ dump/app12345678/
That will restore all the files in the
dump/app12345678 directory created by
mongodump in the step above.
Why the rigamarole?
From the support team:
The reason for this change is MongoDB's move to pluggable storage systems. We haven't yet decided how best to package backups for these scenarios: with any backup you download from us, you can use the extracted files directly to stand up a mongo instance locally, against which you can then perform mongodump/mongorestore operations. For your particular case, it's easiest to use pre-2.8 utils to create the type of files that mongorestore can work with.
Why isn't there a “restore” button next to each backup?
I suggested this along the way, and got this response:
We actually do have an automagic restoration button for each and every backup taken on a deployment... but only for deployments that are provisioned through a direct Compose.io account. The TL;DR on why that has to be is because of how the Heroku partner program integration works. There just wasn't a reasonable and reliable way that we could bring that same data restoration functionality to Heroku-provisioned databases. Sadness all around. For heroku databases, using standard mongodump / mongorestore tools is the only way to restore a backup.