c8cr1q
Last Updated: February 25, 2016
·
5.495K
· richthegeek
13027845faeb65d26acd203f755ceb21

Tailable cursors in MongoDB

This took an hour or so of googling and reading the source of the Node driver to figure out, so I'm putting it here to save others the pain.

There's a neat feature of MongoDB wherein you can keep a cursor open on capped collection and waiting for data for the lifetime of the application. It's especially useful for creating a simple PubSub system without adding a dependency on Redis.

First you need to set up the capped collection:

db.createCollection('messages', {capped: true, size: 100000})

The "size" field is the maximum size of the collection in bytes. The above is 100kb, which is more than enough for the pubsub system (probably too much!)

Then, to get messages from that collection when they are inserted:

function listen(conditions, callback) {
  coll = db.collection('messages')
  latestCursor = coll.find(conditions).sort({$natural: -1}).limit(1)
  latestCursor.nextObject(function(err, latest) {
    if (latest) {
      conditions._id = {$gt: latest._id}
    }
    options = {
      tailable: true,
      await_data: true,
      numberOfRetries: -1
    }
    stream = coll.find(conditions, options).sort({$natural: -1}).stream()
    stream.on('data', callback)
  })
}

The summary is as follows:
1. get the latest document and add it's ID to the conditions
2. create a cursor which is tailable, awaits data, and always retries
3. keep calling cursor.nextObject, passing results to the callback

The main thing that wasn't clear elsewhere is the option await_data, as it is usually written awaitdata.

Hope this helps someone!

Say Thanks
Respond

1 Response
Add your response

19303
6e385c63ee0f189bb0e154ab691bd376
  1. It seems to actually be that the option is "awaitdata" nowadays, not "await_data" like it currently says in the post.
  2. It should be noted that the tailable cursor wont work if the collection is empty, so if you're unit testing, and clearing your database, you need to create an initial row or something before setting up the tail.
over 1 year ago ·