redibox / job Goto Github PK

A Job should be able to be delayed once created, and only run after the delayed time has expired.

At the moment we've added logging for processes running in Kubernetes pods everywhere, for creation, completion and failures of jobs. Instead this should be down to the user to add - to do this we need to add in lifecycle events for jobs, I propose the following:

• beforeCreate
  • async with callback - return original job or modified job.
• afterCreate
  • Just plain events, no interaction with jobs.
• onSuccess
  • Just plain events, no interaction with jobs.
• onRelayCancelled
  • Just plain events, no interaction with jobs.
• onRetry
  • Just plain events, no interaction with jobs.
• onFailure
  • Just plain events, no interaction with jobs.

Trying to build a test for the Job rety() method, however hitting an issue:

  it('Should allow the instance of the job to retry itself', (done) => {
    let count = 0;
    global.singleJob = function queueHandler() {
      console.log('doing job')
      count++;
      if (count === 2) {
        return done();
      }
      return Promise.reject();
    };

    const job = Hook.create('test', {
      runs: 'singleJob',
    }).onFailure(() => {
      job.retry();
    });
  });

job.retry(); is a function, but since it's a multi-exec it's async, but doesn't seem to be triggering the second job.

Jobs should be able to be tracked throughout their lifecycle. Either by Job ID or specific data matching. A tracked job should also show any jobs which created it along with any jobs it has created itself.

Use case:

If debugging on a cluster setup with multiple users using it, then jobs will be shared across them. Having a sticky option on a job which always runs the subsequent jobs on the server it was created from would be really handy.

When forcing a job to timeout, the expected result of the onFailure result should have a timeout property in the error object, in this case it doesn't:

  it('Should emit a failure if job timeout is set and hit', function (done) {
    global.singleJob = function singleJob() {
      return new Promise((resolve) => {
        setTimeout(resolve, 2000);
      });
    };

    Hook.create('test', {
      runs: 'singleJob',
    }, {
      timeout: 1000,
    }).onFailure(result => {
      assert.equal(result.error.timeout, true);
      done();
    });
  });

result:

{ job: 
  { id: 'test-citvo32bg0003uhvdl3i3k3e7',
    worker_id: 'citvo31ie0000uhvd9hv0z3dx',
    status: 'failed',
    runs: 'singleJob' },
 error: {},
 output: null }

Each queue should provide stats on metrics of it's Jobs. E.g. process times, average job waiting length etc.

When a job time out occurs AND retries is set, the job does timeout but does not retry again. Redis also seems to close the connection.

  it('Should retry the job if timeout is reached and retries is set', function(done) {
    this.timeout(10000);
    let count = 0;
    global.singleJob = function queueHandler(job) {
      count++;
      if (count === 1) {
        return new Promise((resolve) => {
          setTimeout(() => { resolve() }, 2000);
        });
      } else {
        return done();
      }
    };

    Hook.create('test', {
      runs: 'singleJob',
    }, {
      retries: 5,
      timeout: 1000,
    });
  });