Last Updated: February 25, 2016
· jrschumacher

Design MongoDB heavy then put 'em on a diet.

I found myself optimizing my MongoDB database well before I knew what to optimize. Now I am stuck continually writing scripts to add more data to documents and handling complex queries because I didn't want to store too much data. In short, it's far more cost effective to clean up unused data than writing scripts to add missing data.

So for flexible applications I design heavy: store as much potentially necessary data until it is determined unnecessary.

He's chunky, slow and smelly but he has potential. The heavy database isn't going to be running fast or index well and will surely put a load on your storage and memory. That said he will have all the information you will need while your application comes to it's completion. You won't find your self in a situation where you wish you had added the event endDate rather than calculating what it will be every time.

Wow look at him run! Once you have a heavy system and most of your core functionality done you will start seeing a pattern and know where you can optimize. Plus with a simple query clean up deprecated data in older documents.

db.User.update({}, {$unset: {fullName: 1, createdOnCentury: 1}}, {multi: true});

Now you aren't spending the time to update older documents with data you should have added ahead of time.

He will be slim in time for the marathon. The truth is, I will find the time to optimize the database and worse case spend a little more money until I can. The flip side is having down time while I write scripts to migrate old documents. In the first stages of an application what is the difference in 200KB to 2MB? Not much when the server has 4GB memory and 40GB disk space. The real question is will I have more time to work on adding data or removing data?

Note: There are a lot of very good alternatives to this approach and some may argue this approach is a symptom of not enough design. These are valid arguments, however in our case we didn't have the luxury of enough design time. Plus our client was still discovering their needs during the project – and still is.

Say Thanks

1 Response
Add your response

Dsc 0105 0

I agree completely. It is easier for me to slaught unnecessary fields rather adding them later.

over 1 year ago ·