Last Updated: February 25, 2016
· amo__

RabbitMQ : Tip to build a consumer scheduler

RabbitMQ and any Messaging servers are great.
But how do you manage to start a consumer when it's needed ? You can't keep every kind of consumer alive for ever ! Hopefully RabbitMQ provides a built-in API that can help solve the problem !

curl -i -u guest:guest http://localhost:15672/api/queues/your_vhost

This api return an json of items containing, among others:

{ "name" : "generateThumbnails", "consumers": 3, "messages_ready" : 23 }

By calling this api every minutes (Cron task ?), you'll be able to know when a queue have messages waiting to be consumed without any consumers to handle them, from there you will only have to know how to start a consumer, based on queue's name.

An additionnal feature for you scheduler : when you start a consumer, you might include a mecanism to limit the life-time of your consummer :

  • Maximum of messages to be handled
  • Maximum duration of the consumer (don't execute for more than 2 hours, no matter how many message are handled)
  • or both :)

This should lead you to an efficient consumer scheduler solution :)