Last Updated: February 25, 2016
·
5.375K
· comerford

Avoiding slow insert performance with MongoDB 2.6 shell

With the 2.6 release of MongoDB, the default shell behavior has changed. Previously, with 2.4 and below, a quick-and-dirty for loop to insert 100,000 records would have only checked the status of the last operation in the loop. This is fast, but not exactly what you would like for a default behavior in general, there is no feedback if 999,999 inserts fail but the last one succeeds (not a common case, granted, but you get the point).

As a quick comparison, here are the relative speeds for this loop which does 100,000 inserts:

2.4 = 2.246 seconds
2.6 = 37.169 seconds

Ouch! But, as I said, this is a better default in general so we can forgive version 2.6 as long as there is a way to get back to the ~2 second performance. There are 2 ways to do this, the first is simply use the 2.4 shell which will default back to the old behavior, but that has drawbacks and it's annoying to have to keep multiple versions around for something like this.

The second way is the better one going forward - using the new unordered bulk insert API gets us back to 2.246 seconds, and even better, gives us real feedback about what succeeded and what failed:

var bulk = db.timecheck.initializeUnorderedBulkOp(); 
for(var i = 0; i < 100000; i++){
    bulk.insert({"_id" : i})
}; 
bulk.execute({w:1});
   BulkWriteResult({
   "writeErrors" : [ ],
   "writeConcernErrors" : [ ],
   "nInserted" : 100000,
   "nUpserted" : 0,
   "nMatched" : 0,
    "nModified" : 0,
    "nRemoved" : 0,
    "upserted" : [ ]
    })

This can all be run as a single line command, I just split it out to help with readability. You can see the full test results and the full write up of this on both the Stack Overflow Q&A that I wrote up, or my original blog post on the topic

2 Responses
Add your response

In version 2.4.8, I am getting this error:
mongos> var bulk = db.PerfTest.initializeOrderedBulkOp();
++++++++++++++++
Wed Apr 23 08:16:23.686 TypeError: Property 'initializeOrderedBulkOp' of object GoodFriday.Perf
Test is not a function
++++++++++++++++
Is "initializeOrderedBulkOp()" a valid function call in 2.4.8 ?
Any help would be greatly appreciated.
Thank you,
vad.roytman@gmail.com

over 1 year ago ·

Right - the bulk ops are 2.6+ and this is specifically for 2.6 testing - in 2.4 you can just use a straight for loop, but you lose the nice results summary - see the linked loop in the tip (https://gist.github.com/comerford/9834062)

over 1 year ago ·