agenda / agenda Goto Github PK

I know this isn't the most helpful issue opened, but after the last update pushed to npm I am seeing daily jobs get ran every few hours.

I'm going to sit down tomorrow and get some test cases going, but just a heads up that something is up.

I am considering porting my kue code over to agenda. Reading the docs I do not see any way to schedule a job, wait for it to complete and then have an complete job event that can tell you the status of the job that just ran.

I realize that this functionality is not needed in the case of recurring jobs, but it is very useful in apps that use jobs to do some heavy lifting during a request and then return some result of the job to the user.

One other question - or maybe feature request: If this type of event is supported, it is possible to pass back data from the job back to the code that initiated the job?

It seems as though Agenda reruns jobs whenever they are defined, regardless if they were ran before or not.

Take this example:

var mongoose        = require('mongoose');
var Agenda          = require('agenda');

var config = require('./configuration').env();

var agenda = new Agenda();
agenda.database(config.mongo);

agenda.define('testJob', function(job, done){
  console.log('testJob Running');
});

agenda.every('1 days', 'testJob');

agenda.start();

console.log('Agenda started');

When you start this program, the job runs once and waits. If you restart the server, the job gets ran again, even though it was defined to be ran daily and was marked as ran and finished.

TestJob MongoDB Entry:

{
    "_id" : ObjectId("53149a7b6ad28fe1395c487e"),
    "data" : null,
    "failReason" : null,
    "failedAt" : null,
    "lastFinishedAt" : ISODate("2014-03-03T15:28:44.887Z"),
    "lastRunAt" : ISODate("2014-03-03T15:28:44.886Z"),
    "lockedAt" : null,
    "name" : "testJob",
    "nextRunAt" : ISODate("2014-03-04T15:28:44.886Z"),
    "priority" : 0,
    "repeatInterval" : "1 days",
    "type" : "single"
}

As we can see, the job is marked as ran, and should be ran next on March 4th (the next day). Is there a way of defining and setting a job up so that it doesn't automatically run?

I've also tried using the Crontab syntax ('0 5 * * 1-5'), but this job also gets ran every time the server starts.

I want to defined a job or two to run at a certain time every day and not have to worry about the server rerunning these daily jobs if the server restarts for some reason.

I had a look through the documentation and code but couldn't find any way to remove jobs?

I created a task and had it running every 10 seconds but decided I want to call it manually, however it keeps running!

I tried Job#remove as mentioned in #26 but that doesn't exist:

agenda.jobs({name: 'Update Instagram'}, function(err, jobs) {
  if(err) throw err;
  jobs.forEach(function(job) {
    job.remove();
  });
});

[TypeError: Object #<Job> has no method 'remove']

When I decide to change one job name it happens that the old job still exists in Mongo and if I forgot to change the execution name on an event (let's imagine I have this job triggered from several places at my app), the old job gets executed instead of the new one.

I'm not quite sure how to structure this, but I'd like to be able to look up the job records, and see which server a job was last run on.

I'd like the hostname (os.hostname) and process id (process.pid) records. Perhaps to keep this flexible a callback could be provided which sets the server identifier for the record update?

This will help with troubleshooting - tracking what was run where.

...that way we don't have to guess which value to set lock expiry.

for the testing purpose, I randomly created a job, later in the code, I rename the job to another, then the I encountered a error : uncaught exception: Error: Attempted to run job undefined.

I don't see any API to remove the job or list all the jobs, or I missed it

according to this http://nodejs.org/api/globals.html#globals_settimeout_cb_ms a timer can only span to 24.8 days.

maybe create a handler when the time set overflows this limit.

Having agenda be aware of other nodes in the system that connect to the jobs database will be helpful in the scenario you don't want a dedicated batch machine in your environment. This will prevent the same jobs being run twice in the same environment on different machines. (some sort of distributed lock mechanism with mongo?).

Hi, i took the example in readme and wrote a tryAgenda.js very simple

Agenda = require('agenda');
var agenda = new Agenda({db: { address: 'localhost:27017/agenda-example'}});
agenda.define('a task', function(job, done) {
console.log("a task is running ... say hello");
});
agenda.every('1 minutes', 'a task');
agenda.start();

but at runtime it exits with error ...see trace http://pastebin.com/QC4JMK2t

any hints?

I have tests locally failing. It look strange since travis ci shows everything is green. Will appreciate suggestions:

> [email protected] test /Users/alexanderbeletsky/Development/Projects/agenda
> mocha


  ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․

  63 passing (768ms)
  2 failing

  1) Agenda configuration methods database sets the database:
     TypeError: Cannot read property '_connect_args' of undefined
      at Context.<anonymous> (/Users/alexanderbeletsky/Development/Projects/agenda/test/agenda.js:26:33)
      at Test.Runnable.run (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runnable.js:211:32)
      at Runner.runTest (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:358:10)
      at /Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:404:12
      at next (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:284:14)
      at /Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:293:7
      at next (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:237:23)
      at Object._onImmediate (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:261:5)
      at processImmediate [as _immediateCallback] (timers.js:330:15)

  2) Agenda configuration methods database sets the collection:
     TypeError: Cannot read property '0' of undefined
      at Context.<anonymous> (/Users/alexanderbeletsky/Development/Projects/agenda/test/agenda.js:30:41)
      at Test.Runnable.run (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runnable.js:211:32)
      at Runner.runTest (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:358:10)
      at /Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:404:12
      at next (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:284:14)
      at /Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:293:7
      at next (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:237:23)
      at Object._onImmediate (/Users/alexanderbeletsky/Development/Projects/agenda/node_modules/mocha/lib/runner.js:261:5)
      at processImmediate [as _immediateCallback] (timers.js:330:15)



npm ERR! Test failed.  See above for more details.
npm ERR! not ok code 0

I need to have a job, that scheduled for particular time.. but then repeated again next time. Something like

agenda.schedule('every saturday 12:00', 'send email');

So, it would run every saturday 12:00. Is it possible to do that?

Hi,

Thanks for the great library.

I am using agenda for sending reminder email after an interval of 2 days of adding some activity. Everything is working fine on localhost but after deploying application with nodejitsu, I am getting following error on calling an API that uses agenda,

502 socket hang up

After removing agenda from the API that uses agenda, everything works fine. Any idea why this error is occurring on deploying application?

So, is the package standalone? Do I need to run a mongodb service on the system?

var agenda = new Agenda({db: { address: 'localhost:27017/agenda-example'}});

agenda.define('delete old users', function(job, done) {
  User.remove({lastLogIn: { $lt: twoDaysAgo }}, done);
});

agenda.every('3 minutes', 'delete old users');

Trying the examples results in errors all over the place and so I'm wondering what I'm missing, what isn't explained in the dox.

Running into failed to connect to [localhost:27017], undefined function calls when I try different ways etc.

if you register on gittip.com ill buy you a beer for making this awesome lib

from README:

success - called when a job finishes successfully
success:job name - called when a job finishes successfully

agenda.once('success:send email', function(job) {
console.log("Sent Email Successfully to: %s", job.attrs.data.to);
});

I use something like:
agenda.on('success', function(job) {
console.log("Sent Email Successfully to: %s", job.attrs.data.to);
});

and this is not being executed even if my jobs are not failing and I receive on agenda.on('complete', ...

• there is typo in README since we should use agenda.on not agenda.once

Hi there

Is it possible to run a specific file at first of every month at midnight??Any help will be appreciated..

thanks

Is there any way to determine what jobs are running right now?

I'm trying to understand the real need to save method. With save method, you write smth like

var job = agenda.schedule('some', 'job');
job.repeatEvery('week');
job.save()

it's to verbose, as for me. It would look better with kind of fluent inteface.

agenda.schedule('some', 'job').repeatEvery('week');

Looks like methods repeatEvery, schedule, priority could do save operation inside, so you don't have to call it manually. What do you think?

I noticed that we cannot remove a job on complete or success callback like

agenda.on('complete', function(job){
    job.remove(function(err){});
});

Is this intended behaviour?

When I create a job to run every 10 minutes from a running node instance, it works fine. However when I restart the node server, the jobs are not started again.

While my app is initializing I do:

global.agenda = new require('agenda')({
    db: {
        address: 'USER:PASSWORD@localhost:27017/agenda',
        collection: 'jobs'
    },
    processEvery: '30 seconds',
    maxConcurrency: 100
});

global.agenda.start();

I bind a job to a model of my app when the model is created:

agenda.define(model.id, function(job, done) {
    //do something
    done();
}); 
agenda.every('10 minutes', model.id);

This works when the job is created and while nodejs is running. However when nodejs is restarted, the jobs do not run.

Am I missing something?

Sample code:

var Agenda, queue;

Agenda = require('agenda');

queue = new Agenda({
  db: {
    address: 'localhost:27017/test_DB'
  }
});

queue.purge(function(err, num) {
  if (err != null) {
   console.log("Error purging Agenda queue: " + err);
  }
});

Output of the script: key $not must not start with '$'.

Hello, thanks so much for building this. Its really a big advantage over cron. I am not sure what happened, maybe this is somehow an issue with my time zone or something? OR maybe I did not quite understand how it works.

But I could swear that I scheduled something for Friday at 11:30pm.. unfortunately the server restarted on Wednesday at 6:30pm. Not sure how a time zone configuration could account for that, although it does end in :30.

Anyway I really appreciate a hint about how to fix this if anyone has a quick second. This is what is in my DB:

 db.agendaJobs.find()
{ "_id" : ObjectId("530430bd810cd93400000001"), "data" : {  }, "failReason" : null, "failedAt" : null, "lastFinishedAt" : ISODate("2014-02-19T23:30:08.737Z"), "lastRunAt" : ISODate("2014-02-19T23:30:04.706Z"), "lockedAt" : null, "name" : "restart scripto", "nextRunAt" : null, "priority" : 0, "repeatInterval" : null, "type" : "normal" }
{ "_id" : ObjectId("530430bd810cd93400000002"), "data" : {  }, "failReason" : null, "failedAt" : null, "lastFinishedAt" : ISODate("2014-02-19T23:30:04.737Z"), "lastRunAt" : ISODate("2014-02-19T23:30:04.710Z"), "lockedAt" : null, "name" : "restart scripto", "nextRunAt" : ISODate("2014-02-26T23:30:04.710Z"), "priority" : 0, "repeatInterval" : "1 week", "type" : "normal" }
{ "_id" : ObjectId("530430bd810cd93400000003"), "data" : {  }, "failReason" : null, "failedAt" : null, "lastFinishedAt" : ISODate("2014-02-19T23:52:44.436Z"), "lastRunAt" : ISODate("2014-02-19T23:50:04.958Z"), "lockedAt" : null, "name" : "backup", "nextRunAt" : null, "priority" : 0, "repeatInterval" : null, "type" : "normal" }
{ "_id" : ObjectId("530430bd810cd93400000004"), "data" : {  }, "failReason" : null, "failedAt" : null, "lastFinishedAt" : ISODate("2014-02-19T23:52:44.237Z"), "lastRunAt" : ISODate("2014-02-19T23:50:04.962Z"), "lockedAt" : null, "name" : "backup", "nextRunAt" : ISODate("2014-02-20T23:50:04.962Z"), "priority" : 0, "repeatInterval" : "1 day", "type" : "normal" }

And this is my code:

Agenda = require 'agenda'
childproc = require 'child_process'

restart = (job, done) ->
  e, o, er = childproc.exec! './stop'
  console.log "#{e}\n#{o}\n#{er}"

  prog = childproc.spawn 'bin/run.sh', [], {}
  prog.stderr.on 'data', (d) -> console.log d.toString()
  prog.stdout.on 'data', (d) -> console.log d.toString()
  console.log "started scripto"
  done?()
  null

backup = (job, done) ->
  e, o, er = childproc.exec! 'toffee backup.coffee'
  console.log "#{e}\n#{o}\n#{er}"
  done()
  null

restart {}

agenda = new Agenda {db: { address: 'localhost:27017/agenda' }}

agenda.define 'restart scripto', restart
agenda.define 'backup', backup

weeklyRestart = agenda.schedule 'Friday at 11:30pm', 'restart scripto', {}
weeklyRestart.repeatEvery('1 week').save()

nightlyBackup = agenda.schedule "at 11:50pm", 'backup', {}
nightlyBackup.repeatEvery('1 day').save()

agenda.start()

process.on 'uncaughtException', (er) ->
  console.log er

Thanks so much for any advice you have.

Hey guys, Love the library. Thank you.

I just had a question about agenda and how it saves jobs.

does saving a job mean that onces the node server is shutdown and restarted, it will continue to process the jobs saved in the database? if not, the what is the alternate solution?

I have tried saving jobs, but after restarting my node script, it does not run my old jobs. Here is my script

var express = require('express')
var app = express()
var server = require('http').Server(app)
var io = require('socket.io')(server)


var Agenda = require('Agenda')
var agenda = new Agenda({ db: { address: 'localhost:27017/jobs' } })
agenda.start() // start agenda

io.on('connection', function (socket) {

    socket.on('createJob', function (data, callback) {
        var jobName = data.name;

        // create the job
        agenda.define(jobName, function (job, done) {

            // do stuff
            getReport();
            sendEmail();

            // save the job
            job.save(function (err) {
                if (!err) {
                    console.log("Saved");
                    done();
                }
            })

        })

        agenda.every('3 minutes', jobName, { data: 'random' });

    })

})

// start server
server.listen(3000, function () {
    console.log('Server started');
})

Right now the server is locking jobs and then not starting them with a setTimeout so that we get the exact run time. If you close agenda while it's in limbo, the job will likely be locked.

References #58 (I suspect).

I know this is a total noob question, but how do I include this , I keep getting

ReferenceError: Agenda is not defined

var agenda = new Agenda();

Hello,

I have the following scenario: my queue contains three scheduled items (i.e. A, B, C). The external process that handles these items cannot be launched multiple times at the same time. For this reason, I would need Agenda to process A, wait until it's completed (or failed), then process B, wait, then finally C.

If I understand the documentation correctly, I would have to invoke Agenda like so using your example in the documentation:

agenda.define('some long running job', function(job, done) {
  doSomelengthyTask(function(data) {
    formatThatData(data);
    sendThatData(data);
    // done();
  });
});

I should remove done() because I want Agenda to process the scheduled items synchronously, correct?

Also, what is the content type and value of job and done? What should I be expecting back?

Thanks!

Hi

I was testing agenda to use it in my project. to kick it off, I tried with agenda.now() and I encountered too many issues

The sample test case i used is to print the input string. I tried printing from 1 - n and i observed these things

1. It did not print all the numbers, it missed some
2. Sometimes its printing numbers twice
3. After 10 calls (approx), the scheduler stops forever, i dont see any numbers printing, even if I input
   4.There is atleast a 2 second delay in printing the numbers

I want to run this jobs:

_agenda.every('1 minute', 'heartbeat', {}); // every minute
_agenda.every('35,36,37 * * * *',   'anotherjob', {}); // at 35 mins, at 36 mins, at 37 mins of each hour

Then I add a logger to monitor output:

_agenda.on('start', function (job) {
    console.log("> "+(new Date)+" : "+job.attrs.name+" // "+JSON.stringify(job.attrs.data));
});

So the output is:

> Mon Apr 07 2014 15:34:54 GMT-0300 (BRT) : heartbeat // {}
> Mon Apr 07 2014 15:35:54 GMT-0300 (BRT) : heartbeat // {}
> Mon Apr 07 2014 15:36:54 GMT-0300 (BRT) : heartbeat // {}
> Mon Apr 07 2014 15:37:54 GMT-0300 (BRT) : heartbeat // {}
> Mon Apr 07 2014 15:38:54 GMT-0300 (BRT) : heartbeat // {}

This means that my scheduled job cron-style doesn't work.

Am I doing something wrong?

I'm not 100% sure yet, but it seems after the latest update .every() jobs never properly save being completed. Take this example:

var Agenda          = require('agenda');
var log             = function() { console.log.apply(this, arguments); };

var agenda = new Agenda();
agenda.database('mongodb://localhost/some-db').processEvery(100);
agenda.start();

agenda.define('testJob', function(job, done){
  done();
  log('testJob Running');
});

agenda.define('anotherJob', function(job, done) {
  done();
  log('anotherJob Running');
});

agenda.every('2 seconds', 'testJob');
agenda.now('anotherJob');


log('Agenda started');

log('finding jobs');

On a blank DB, you'll find these in your agendaJobs collection:

{
    "_id" : ObjectId("5319eb8f3b96f798eb0a3760"),
    "data" : null,
    "failReason" : null,
    "failedAt" : null,
    "lastFinishedAt" : null,
    "lastRunAt" : null,
    "lockedAt" : ISODate("2014-03-07T15:53:51.388Z"),
    "name" : "testJob",
    "nextRunAt" : ISODate("2014-03-07T15:53:51.376Z"),
    "priority" : 0,
    "repeatInterval" : "2 seconds",
    "type" : "single"
}
{
    "_id" : ObjectId("5319eb8ff86ba55343000001"),
    "data" : null,
    "failReason" : null,
    "failedAt" : null,
    "lastFinishedAt" : ISODate("2014-03-07T15:53:51.445Z"),
    "lastRunAt" : ISODate("2014-03-07T15:53:51.445Z"),
    "lockedAt" : null,
    "name" : "anotherJob",
    "nextRunAt" : null,
    "priority" : 0,
    "repeatInterval" : null,
    "type" : "normal"
}

"anotherJob" gets marked as complete, however "testJob" does not. It's pretty odd, I looked into the callback function to ensure the .save() actually updated the document.

    var jobCallback = function(err){
      if(err){

        self.fail(err);
        agenda.emit('fail', err, self);
        agenda.emit('fail:' + self.attrs.name, err, self);
      }else{
        agenda.emit('success', self);
        agenda.emit('success:' + self.attrs.name, self);
      }

      self.attrs.lastFinishedAt = new Date();
      self.attrs.lockedAt = null;
      self.save(function(saveErr, job){

        console.log('===     self.save     ===');
        console.log(arguments);

        cb && cb(err || saveErr, job);
      });
    };

You'll see the .save() callback's returned job is actually marked as completed, which is slightly baffling. Sorry for opening to many issues, I hope this helps!

If a mongodb instance is secured with username/password, how do I setup this in agenda?

Redis support would be fantastic. I run for a 100% Redis outfit, and I'd like to use your module for building our scheduler system.

I know it muddies the waters somewhat, so it may be worth looking into JugglingDB (an ORM for Node with support for many databases, including Redis and Mongo). Usually I'd be loathe to recommend an ORM, but this is one case that may suit such a suggestion.

Hi,

'use strict';

var agenda = require('agenda'),
    task = agenda({ db: { address: 'localhost:27017/agenda-example' } }),
    util = require('util');

task.define('FindInaPage', function(job, done) {
  util.log('found!');
  done();
});

task.every('0 35 * * * *', 'FindInaPage');
task.start();

Turns into:

$ node runner.js
3 Apr 20:34:59 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:00 - found!
3 Apr 20:35:01 - found!

That is so weird.

FWIW: I have created date-frequency a while ago.

If you ever bump into limitations with the cron notation in the every method you offer, maybe it is worthwhile having a look at it. It is far from finished, but its goals are:

• An ISO 8601 Durations based string notation. This is more readable than cron and is more extensible. For instance, cron doesn't offer n-th week of the year.
• Allow "next occurence after" and "occurences between" lookups.

The project is of course open for changes/improvements should you ever consider it. It now only does what it needs to do for my own needs.

Hello, if you see my previous issue, I did some testing today and found that whenever I used syntax specifying the day and time like "Friday at 11:30pm" what agenda put in mongo was actually [today] at whatever time. So 'at 11:30pm' works, but specifying the day seems equivalent to not specifying it, so with 'Friday at 11:30pm' or 'AnyDay at 11:30pm' it just gets scheduled for today.

But that syntax would be an awesome feature and is in the Readme example. So if you have a moment, can you confirm that is not actually supported [yet], if you plan to support it in the future, or if you are actually certain it IS supported and somehow my test was not able to reproduce that functionality?

Thanks so much for your time.

It doesn't look like jobs are spun out as their own child processes. Was mainly wondering if this was a feature, or maybe I'm misreading the code somewhere. Thanks for the work!

I am considering creating a small express middleware that will give you a big readonly view of the current jobs and their statuses.

Do you know if anything like this already exists?

I'm trying to run same job with different params.

_agenda.every('15,35,55 * * * *',   'myjob', {name:"john"});
_agenda.every('40 minutes',         'myjob', {name:"mark"});
_agenda.every('0 17,13,19 * * *',   'myjob', {name:"bill"});
_agenda.every('0 2,14 * * *',       'myjob', {name:"steve"});

I know every() saves only one doc in database. but why?

Is there any way to do what i'm trying without re-define same job function many times?

I have two jobs running and they both got stuck at 08:31:13 today running for about 9 hours. It is strange that the lockedAt value is 10:41 while lastRun was 08:31. Do you have any clue whats going on?

{
    "_id" : ObjectId("53867e6ea27e17ac40300357"),
    "data" : null,
    "lastFinishedAt" : ISODate("2014-05-29T08:21:18.834Z"),
    "lastModifiedBy" : null,
    "lastRunAt" : ISODate("2014-05-29T08:21:18.829Z"),
    "lockedAt" : ISODate("2014-05-29T10:41:22.142Z"),
    "name" : "53865113fd1bb38c07329133",
    "nextRunAt" : ISODate("2014-05-29T08:31:13.829Z"),
    "priority" : 0,
    "repeatInterval" : "595 seconds",
    "type" : "single"
}

{
    "_id" : ObjectId("53867e6ea27e17ac40300356"),
    "data" : null,
    "lastFinishedAt" : ISODate("2014-05-29T08:21:18.849Z"),
    "lastModifiedBy" : null,
    "lastRunAt" : ISODate("2014-05-29T08:21:18.846Z"),
    "lockedAt" : ISODate("2014-05-29T10:41:22.141Z"),
    "name" : "53862eb2599aed4707f7c66f",
    "nextRunAt" : ISODate("2014-05-29T08:31:13.846Z"),
    "priority" : 0,
   "repeatInterval" : "595 seconds",
   "type" : "single"
}

processEvery is set to 30 seconds.

var job = agenda.now('foo');
Inspecting the returned job shows that a nextRunAt is set accordingly.

Line #219 in test/agenda.js
describe('jobs', function() also verifies the nextRunAt is returned. However this test does not verify whats written to the database only what has been setup in the job before its saved.

Line #179 in lib/agenda.js
When creating a job with .now() nextRunAt will always be <= now so props.nextRunAt will be deleted from the props object.

Line #184 in lib/agenda.js
update = { $set: props }; will never have a nextRunAt property with job.now()

Line #196 in /lib/agenda.js
Will insert this job with no nextRunAt properties.

This is currently what I am experiencing, all the job.now()'s are being entered into the database with no nextRunAt property.

** Edit - just wanted to say this is a great module and I appreciate the hard work put into it. I suspect this bug might have snuck in with the commits on 6/11 with the .every fix?

var Agenda = require('agenda');

setInterval(function() {
  console.log(new Date)
}, 1000);

var agenda = new Agenda({db: { address: 'localhost:27017/agenda-example'}});
agenda.define('check this out yow', function(job, done) {
  console.log('checked!');
  done();
});
agenda.every('0 * * * *', 'check this out yow');
agenda.start();

I suppose the check should run every hour (00:00, 01:00... 23:00). Unfortunately at first run the definition gets ran first then when the cron pattern has matched, gets ran. Is there a way to make the job not run when the scripts executed?

AFAIK, it looks like human-readable only supports intervals. It would be really nice to add support for simple cron syntax to allow for other use cases that require time precision

https://github.com/ncb000gt/node-cron

In some use cases I need to update a job, to repeat in another intervall:

agenda.jobs({
    name: 'some job'
}, function(err, jobs) {
    if(jobs.length == 0){
        //define job
        agenda.define('some job', { lockLifeTime: 1000 }, function(job, done) {
             // do something and call done();
        });
        agenda.every((interval * 60) - 5 + ' seconds', 'some job');
    }else{
        jobs[0].repeatEvery(repeatInterval + ' seconds').save();
        jobs[0].schedule('in ' + repeatInterval + ' seconds');
   }
});

However the job is updated it is executed only once and stuck in following state:

{
    "_id" : ObjectId("53725383a27e17ac40300276"),
    "data" : null,
    "lastFinishedAt" : ISODate("2014-05-13T17:18:41.625Z"),
    "lastRunAt" : ISODate("2014-05-13T17:18:41.624Z"),
    "lockedAt" : ISODate("2014-05-13T17:19:09.744Z"),
    "name" : "5370c7e94f8ccf6913c87515",
    "nextRunAt" : ISODate("2014-05-13T17:19:36.624Z"),
    "priority" : 0,
    "repeatInterval" : "55 seconds",
    "type" : "single"
}

Why does the job get locked?

I am considering moving agenda into a networked architecture that handles things like job delegation among multiple workers. Currently this is handled by mongo and selective querying by workers.

This is spurred by comments by @coreybutler in #24.

Current System

Advantages

• simplified architecture: no distinction between a master/slave, configuring of multiple workers, etc. Extremely easy to get started (single worker)
• Robust: doesn't break if single worker goes down, or even if 3 workers go down. Each worker is independent

Disadvantages

• Highly reliant on Mongo: Because things like locking, grabbing jobs, etc are all done with queries, it makes it difficult for new adapters to be written. This continues to escalate if we wanted to add things like progress reporting.
• Not as optimized: Currently we grab all the jobs that could be run by a single worker, which cues them and then runs them one at a time (obeying its concurrency). This means that a worker could queue (and grab) 5 jobs, while only being able to run 3. If timing happened correctly, a second worker wouldn't grab those jobs. This could be fixed with better queries, locking, etc. but we further commit to mongo
• Events tied to workers: job events happen on the worker, but not ALL of the workers. This means that every worker must be able to handle the events, and that there's no way to handle an event for a job if you're not the worker that processed the job.

New System

I am envisioning all workers communicating with each other via a socket-like interface. The first worker to connect will basically take a "master" role and handle all of the delegation, database querying, managing state, etc.

Workers can specify which jobs they will take, eg.

var agenda = new Agenda({server: 'agenda://127.0.0.1:29999'})

agenda.define('jobA', doJobA);
agenda.define('jobB', doJobB);
agenda.define('jobC', doJobC);

// worker A
agenda.start({only: ['jobA', 'jobB']})

// worker B
agenda.start({except: ['jobA']})

Advantages

• simplified db adapters : With this, the database goes back to just storing data, not handling things like locking, messaging, etc. With this, we would (hopefully) be able to abstract it away into multiple adapters, allowing support for mongo, redis, postgres, and whatever people want
• unified state : With things like writing a web-interface, it is currently difficult to get a full picture (how many workers are there, what are they working on, etc). This would create a "master" with an answer. This could then have its own query API (sockets would be fine) which would allow a web interface to connect to it and issue commands to get state, force processing of jobs, etc

Disadvantages

• conceptually more complex: There's potentially more to understand, which could make it harder for new contributors to come into the project, as well as make it harder for people to use. Right now agenda seems pretty simple at first glance, and this could lose some of that
• Massive Rewrite: Most of agenda would need to be re-written from scratch. It can be worth it, but it is a big undertaking
• Could be more fragile: What happens when the master goes down? Yes we can recover, but I envision it being a big pain.

Keep in mind this isn't even a proposal, just a half-baked first idea. I am hoping that people will chime in and help shape this, or tell me that it's a bad idea.

Feel free to chime in @coreybutler @bars3s @TomKaltz @mkoryak @moudy and anyone else who wants to make their opinion heard!

Can you allow extensibility to this package to hook up different data stores, so that we can add additional database adapters to it. I am looking for MYSQL support and if it was allowed, then we could use this package for different data sources.

Would it be practical to have a mechanism to report progress on a job to the database or via event emission?

I noticed you have a custom function generateUUID. Why not use node-uuid instead?

Math.random sometimes causes collisions, and node-uuid avoids them while taking advantage of the Node-JS crypto API to be more secure. It also supports timestamp based UUIDs, which might be more applicable to a scheduling library.

See the below, starting at line 169 in https://github.com/rschmukler/agenda/blob/master/lib/agenda.js

function generateUUID() {
   'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
     var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
     return v.toString(16);
    });
}

Hey,

Agenda.prototype.every currently returns the job attributes, but not the id, which makes it hard to save a reference to that particular job. That could easily be remedied by creating the _id beforehand with new Mongo.ObjectID — possibly in the Job constructor.

Cheers

Have job buffer grab for jobs ahead of its window, instead of behind it. Then, use setTiemout to handle these jobs. This would guarantee that jobs run "exactly" when scheduled.