RabbitMQ : Tip to build a consumer scheduler
RabbitMQ and any Messaging servers are great.
But how do you manage to start a consumer when it's needed ? You can't keep every kind of consumer alive for ever ! Hopefully RabbitMQ provides a built-in API that can help solve the problem !
curl -i -u guest:guest http://localhost:15672/api/queues/your_vhost
This api return an json of items containing, among others:
{ "name" : "generateThumbnails", "consumers": 3, "messages_ready" : 23 }
By calling this api every minutes (Cron task ?), you'll be able to know when a queue have messages waiting to be consumed without any consumers to handle them, from there you will only have to know how to start a consumer, based on queue's name.
An additionnal feature for you scheduler : when you start a consumer, you might include a mecanism to limit the life-time of your consummer :
- Maximum of messages to be handled
- Maximum duration of the consumer (don't execute for more than 2 hours, no matter how many message are handled)
- or both :)
This should lead you to an efficient consumer scheduler solution :)