How promises mad my day

I thought I might as well share my first blog post with you right here:

Recently I reviewed some code I had written for my first somewhat bigger node.js project. I was migrating the site to another platform and had some issues with file uploading. So I tried to figure out what went wrong...

The following code basically takes an imageStream, resizes the image with node-gm (two sizes) and attaches them to a couchdb document. I wrote it in CoffeeScript so beware!

My personal Pyramid of Doom

handleImage = (file, data, callback) ->
  if file.size != 0
    imageStream = fs.createReadStream(file.path)
    resizeImage imageStream, '400', (gmErr, thumbStream1) ->
      if !gmErr
        attachImage data, thumbStream1, 'flyer.jpg', (attErr, attData) ->
          if !attErr
            imageStream = fs.createReadStream(file.path)
            resizeImage imageStream, '250', (gmThumbErr, thumbStream2) ->
              if !gmThumbErr
                data.rev = attData.rev
                attachImage data, thumbStream2, 'flyer_thumb.jpg', (attThumbErr, attThumbData) ->
                  if !attThumbErr
                    callback(null, 'OK')
                  else
                    callback('failed to attach image to document ' + JSON.stringify(data))
              else
                callback('failed to resize image 250')
          else
            callback('failed to attach image to document ' + JSON.stringify(data))
      else
        callback('failed to resize image 400')
  else
    callback(null, 'OK')

Holy moly, that's a nice pyramid I built there! Besides the crappy code (with no comments at all!) the callback hell prevents to easily figure out what exactly is going on there. Luckily, having written this in CoffeeScript brings some light into this mysterious jungle. It's full of ifs and elses and doesn't look a bit beautiful at all.

Promises to the rescue!

I've been kind of into promises lately. I like the beautiful code style this technique generates and it's a powerful tool when it comes to asynchronous code. I'm not going into the details of the theory and history of promises; i think this is pretty much covered. I just going to frame the part on how they helped me on my particular problem.

First things first

To use promises it's really helpful to rely on some third party libraries on this matter. Popular ones are Q, RSVP.js and when.js. I chose Q because it did a good job for me lately and is well documented.

Let's go:

npm install q

Don't forget to require it:

Q = require('q')

Now I could start using promises. So I had all my asynchronous functions with callbacks and whatnot, that needed to be changed to correspond to the Promise/A proposal. Easier said, they need to talk the promise-language.

Luckily, Q provides a way to generate deferreds (which return promises) from existing callback functions. I used an existing pattern to promisify my functions:

promisify = (asyncFunction, context) ->
  ->
    defer = Q.defer()
    args = Array.prototype.slice.call(arguments)
    args.push((err, val) ->
      if err
        return defer.reject(err)
      defer.resolve(val)
    )
    asyncFunction.apply(context || {}, args)
    defer.promise

This function wraps your asynchronous callback function and returns a promise created by the deferred callback. It basically resolves the val and rejects the err of the callback applied to the asyncFunction. Note that it only works with the standard style callback functions callback(err, val).

Now it's time to wrap the functions:

resizeImage = promisify((readStream, size, callback) ->
  gm(readStream, 'img.jpg').resize(size, ' ').stream((err, stdout, stderr) ->
    if !err
      callback(null, stdout)
    else
      callback(err)
  )
)

Yes, it's that easy. Just add promisify() around the callback functions and you're good to go. Now the functions are returning promises and it's possible to use Q's wonderful set of tools on them.

The promised way

With no further ado, this is the resulting handleImage() function done with my new gained promise-returning functions:

handleImage = (file, data) ->
  if file.size != 0
    imageStream = fs.createReadStream(file.path)
    Q.all([
      resizeImage(imageStream, '400'), # bigThumb
      resizeImage(imageStream, '250') # smallThumb
    ]).spread((bigThumbStream, smallThumbStream) ->
      smallThumbStream.pause() # The stream has to wait for the first attachImage to be done
      attachImage(data, bigThumbStream, 'flyer.jpg').then((bigThumbData) ->
        data.rev = bigThumbData.rev
        smallThumbStream.resume() # Yeah. Node version 0.8.x
        attachImage(data, smallThumbStream, 'flyer_thumb.jpg')
      )
    )
  else return true

Aaah, much better! Some notes on this:

• Q.all() provides a way to execute multiple functions in parallel. The returned promise is resolved, when all promises inside the array are resolved and it is rejected immediately if one of them get's rejected. If all promises are resolved, an array of the resolved values is passed to the next chained then()-Function.
• The spread() function can be used instead of then() if the last promise passes an array of resolved values. It spreads the array into the arguments of it's inner function. The corresponding way doing it with then() should make it a bit clearer:

...
Q.all([
  resizeImage(imageStream, '400'),
  resizeImage(imageStream, '250')
]).then((thumbStream) -> # thumbStream is an array!
  thumbStream[1].pause()
  attachImage(data, thumbStream[0] 'flyer.jpg').then((bigThumbData) ->
    data.rev = bigThumbData.rev
    thumbStream[1].resume()
    attachImage(data, thumbStream[1], 'flyer_thumb.jpg')
  )
)
...

• The attachImage() part has to be done in serial because the second image needs to know the document revision after the first image upload (which changes the revision).
• In case you're wondering why we're just returning a static true when there's no image attached (file.size === 0) that's because we're using Q.when to determine if there is something that has to be resolved in the future or not.

Q.when(handleImage(uploadedFile, data), () ->
  # everything's fine.
, (err) ->
  # there was an error.
)

when() is used when a function might or might not return a future object (promise). The function in the second argument is executed if the handleImage() function returns a resolved promise or if it returns a static value. The function in the third argument is executed if handleImage() returns a rejected promise.

(All) done

I'm pretty happy with the results (even there is plenty of room for optimization, but I think it's okay for the time being) and even I could not find the error as fast as I hoped afterwards, I think the refactoring was worth the time.

So I hope I could make the use of promises in practice a bit clearer and if you read the post to this point, thank you very much, you have been a very patient reader!