For a side project, I’ve been looking for a batch queueing system – i.e. a system that I simply submit a command script to, and the script gets executed on the next available server. The output is logged – while running – into a central database. Bonus points if there’s fine-grained control over what servers get chosen.

And it turns out, solutions to that problem don’t really exist. Yes, I can “roll my own” with duct tape and a bunch of shell scripts – but is this truly a system that nobody ever saw the need to create and share with the world?

In case it is shared, I’d really like to hear about it. If not, it’s probably ending up on my projects list sooner or later… And in case it does, here’s a set of links to get started.

OSS packages

Somewhat close to what I need, usually with lots of overkill in areas I don’t need:

  • Condor
  • TORQUE – this one looks closest to what I need, but again, is huge
  • Maui Scheduler – a scheduling counterpart to TORQUE
  • GNU Queue – seems pretty abandoned
  • DrQueue – Mostly for scheduling render tasks, but can maybe be bent to my will

Discussions & articles on the net:

Leave a reply