cron question / process queue

[Michael ODonnell]

> I don't see it.  I have monthly, weekly and daily jobs that I want
> to run sequentially, in that order, starting at 8pm.  How would
> I do that using 'at'?  Preferably without making the actual jobs
> know about each other, e.g.  not teaching the weekly job that it
> should be followed by the daily job.

That's why I like the approach I suggested - the only thing that
your daily/weekly/monthly processes have to do when they decide
it's time for a run is something like:

   cp appropriateScriptFile.sh /yourQueueDir/`date '+%Y%m%d%H%M%S'`-$$+duration.sh

...and then exit.  Those scripts would end up having names like
20080307104617-23987+2h.sh which would cause them to sort lexically
in time order, resulting in FIFO behavior.  Then your hourly
job would only have to decide whether it was appropriate to run
anything it found there, based on the current time and maybe also
the estimated duration as encoded in the filename.  Most hours,
it would just exit because there'd be nothing to do or because
you don't want to run something during business hours.  As each
script completed your hourly job could then determine whether it
was appropriate to start another one immediately (if present) or to
instead simply exit, deferring any remaining scripts until later.

No task would know any of the others even exist and your queue-runner
only has to consider one script at a time, independent of the others.

Of course, synchronization-wise things get interesting as soon
as any job takes longer than the hourly interval because the next
hour's queue-runner has to know not to interfere with the one that's
still active.  A lock-file should be enough to handle that, though.

It's a SMOP - here's the whole thing in Perl:

   $_()%@queue#=.;time~[HiMom]%^$!!__chomp<backup>|\+{&};