Tailable cursors in MongoDB

This took an hour or so of googling and reading the source of the Node driver to figure out, so I'm putting it here to save others the pain.

There's a neat feature of MongoDB wherein you can keep a cursor open on capped collection and waiting for data for the lifetime of the application. It's especially useful for creating a simple PubSub system without adding a dependency on Redis.

First you need to set up the capped collection:

db.createCollection('messages', {capped: true, size: 100000})

The "size" field is the maximum size of the collection in bytes. The above is 100kb, which is more than enough for the pubsub system (probably too much!)

Then, to get messages from that collection when they are inserted:

function listen(conditions, callback) {
  coll = db.collection('messages')
  latestCursor = coll.find(conditions).sort({$natural: -1}).limit(1)
  latestCursor.nextObject(function(err, latest) {
    if (latest) {
      conditions._id = {$gt: latest._id}
    }
    options = {
      tailable: true,
      await_data: true,
      numberOfRetries: -1
    }
    stream = coll.find(conditions, options).sort({$natural: -1}).stream()
    stream.on('data', callback)
  })
}

The summary is as follows:
1. get the latest document and add it's ID to the conditions
2. create a cursor which is tailable, awaits data, and always retries
3. keep calling cursor.nextObject, passing results to the callback

The main thing that wasn't clear elsewhere is the option await_data, as it is usually written awaitdata.

Hope this helps someone!