Git Product home page Git Product logo

adonis-bull-queue's Introduction

@rlanz/bull-queue

Download Version License

@rlanz/bull-queue is a queue system based on BullMQ for AdonisJS.

Note

You must have a Redis server running on your machine.


Getting Started

This package is available in the npm registry.

npm install @rlanz/bull-queue

Next, configure the package by running the following command.

node ace configure @rlanz/bull-queue

and... Voilร !

Usage

The Queue provider gives you access to the dispatch method. It will dispatch the linked job to the queue with the given payload.

import { Queue } from '@ioc:Rlanz/Queue';

Queue.dispatch('App/Jobs/RegisterStripeCustomer', {...});

Queue.dispatch('App/Jobs/RegisterStripeCustomer', {...}, {
  queueName: 'stripe',
});

You can create a job by running node ace make:job {job}. This will create the job within your app/Jobs directory.

The handle method is what gets called when the jobs is processed while the failed method is called when the max attempts of the job has been reached.

You can remove the failed method if you choose as the processor checks if the method exists. Since the job instance is passed to the constructor, you can easily send notifications with the failed method. See this page for full documentation on the job instance.

Example job file:

// app/Jobs/RegisterStripeCustomer.ts
import type { JobHandlerContract, Job } from '@ioc:Rlanz/Queue';

export type RegisterStripeCustomerPayload = {
  userId: string;
};

export default class RegisterStripeCustomer implements JobHandlerContract {
  constructor(public job: Job) {
    this.job = job;
  }

  public async handle(payload: RegisterStripeCustomerPayload) {
    // ...
  }

  /**
   * This is an optional method that gets called if it exists when the retries has exceeded and is marked failed.
   */
  public async failed() {}
}

Job Attempts

By default, all jobs have a retry of 3 and this is set within your config/queue.ts under the jobs object.

You can also set the attempts on a call basis by passing the overide as shown below:

Queue.dispatch('App/Jobs/Somejob', {...}, { attempts: 3 })

Delayed retries

If you need to add delays inbetween retries, you can either set it globally via by adding this to your config/queue.ts:

// config/queue.ts
  ...
  jobs: {
    attempts: 3,
    backoff: {
      type: 'exponential',
      delay: 5000,
    },
  }

Or... you can also do it per job:

Queue.dispatch('App/Jobs/Somejob', {...}, {
  attempts: 3,
  backoff: { type: 'exponential', delay: 5000 }
})

With that configuration above, BullMQ will first add a 5s delay before the first retry, 20s before the 2nd, and 40s for the 3rd.

You can visit this page on further explanation / other retry options.

Running the queue

Run the queue worker with the following ace command:

node ace queue:listen

# or

node ace queue:listen --queue=stripe

# or

node ace queue:listen --queue=stripe,cloudflare

Once done, you will see the message Queue processing started.

Typings

You can define the payload's type for a given job inside the contracts/queue.ts file.

import type { RegisterStripeCustomerPayload } from 'App/Jobs/RegisterStripeCustomer';

declare module '@ioc:Rlanz/Queue' {
  interface JobsList {
    'App/Jobs/RegisterStripeCustomer': RegisterStripeCustomerPayload;
  }
}

adonis-bull-queue's People

Contributors

ammezie avatar brkn avatar dmdboi avatar ikudosi avatar kabbouchi avatar paulocastellano avatar romainlanz avatar simlej avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

adonis-bull-queue's Issues

readme link to bullmq is outdated

Hello,

The link to the bullMQ is outdated. I will push a PR to update it.

In the same time, I will try to use the repeatable feature. If it works, I will create another PR to document it.

SSL/TLS

is possible to conect to a redis server over ssl/tls? i have a digital ocean DB and i only works with ssl.

Run node ace queue:listen in production

I have an API built with Adonis v5 that has jobs for sending messages, etc after an order has been placed. How would I go about runing the queue listener on render.com where the API is deployed?

Any assistance would be highly appreciated.

Cannot specific queueName when dispatch job.

`
import { Queue } from '@IOC:Setten/Queue';

Queue.dispatch('App/Jobs/RegisterStripeCustomer', {...}, {
queueName: 'stripe',
});
`

This produce errors when compiling project

[ error ] typescript compiler errors app/Controllers/Http/CronJobController.ts:75:17 - error TS2769: No overload matches this call. Overload 1 of 2, '(job: "App/Jobs/CalculateUserStat", payload: CalculateUserStatPayload, options?: JobsOptions | undefined): Promise<string>', gave the following error. Argument of type '{ queueName: string; }' is not assignable to parameter of type 'JobsOptions'. Object literal may only specify known properties, and 'queueName' does not exist in type 'JobsOptions'. Overload 2 of 2, '(job: "App/Jobs/CalculateUserStat", payload: CalculateUserStatPayload, options?: JobsOptions | undefined): Promise<string>', gave the following error. Argument of type '{ queueName: string; }' is not assignable to parameter of type 'JobsOptions'. Object literal may only specify known properties, and 'queueName' does not exist in type 'JobsOptions'.

AdonisJS 6 Support

We trying to use bull-queue after init adonis with "API Starter Kit", but there is an error when we try to run node ace configure @rlanz/bull-queue.

node -v
# v21.6.1

How to reproduce:
npm init adonisjs@latest hello-world

Select "API Starter Kit".

Run:

npm install @rlanz/bull-queue
# and
node ace configure @rlanz/bull-queue

Get this following error: [ error ] Cannot configure module "@rlanz/bull-queue". The module does not export the configure hook

Cannot find module 'node:path'

Running node ace configure @setten/bull-queue returns:

Cannot find module 'node:path'
Require stack:
- D:\wamp64\www\quiz\node_modules\@setten\bull-queue\build\instructions.js
- D:\wamp64\www\quiz\node_modules\@poppinss\utils\build\src\esmRequire.js
- D:\wamp64\www\quiz\node_modules\@poppinss\utils\build\index.js
- D:\wamp64\www\quiz\node_modules\@adonisjs\config\build\src\Config.js
- D:\wamp64\www\quiz\node_modules\@adonisjs\config\build\index.js
- D:\wamp64\www\quiz\node_modules\@adonisjs\application\build\src\Application.js
- D:\wamp64\www\quiz\node_modules\@adonisjs\application\build\index.js
- D:\wamp64\www\quiz\node_modules\@adonisjs\core\build\standalone.js
- D:\wamp64\www\quiz\ace

 at Function.Module._resolveFilename internal/modules/cjs/loader.js:880

   1  Function.Module._load
     internal/modules/cjs/loader.js:725

   2  Module.require
     internal/modules/cjs/loader.js:952

   3  require
     internal/modules/cjs/helpers.js:88

   4  Object.<anonymous>
     D:/wamp64/www/quiz/node_modules/@setten/bull-queue/build/instructions.js:9

   5  Module._compile
     internal/modules/cjs/loader.js:1063

   6  Object.Module._extensions..js
     internal/modules/cjs/loader.js:1092

   7  Module.load
     internal/modules/cjs/loader.js:928

   8  Function.Module._load
     internal/modules/cjs/loader.js:769

   9  Module.require
     internal/modules/cjs/loader.js:952

How to deploy on Heroku

First of all, thanks for making this great queue wrapper. It was quite easy to set up and I'm very happy with it ๐Ÿ‘
Now for my question: I have an Adonis web app running on Heroku and would like to run a queue in a worker dyno. I can run it locally and it works great, but when I deploy to Heroku, I get the following error: Error: AdonisJS requires "@adonisjs/assembler" in order to run typescript source directly

This is my Procfile:

web: node build/server.js
worker: node ace queue:listen

I noticed when I run the build command (node ace build --production), it doesn't make a new JS file for the queue/worker. Is there some way to make it generate something I can run on Heroku, like a build/queue.js?

Access to the queue object

I was evaluating the queue packages for Adonis, and I really liked the structure you've built. However, I would need to have access to the queue in order to, for example, delete all jobs. Do you think it would be interesting to have a getQueue(queueName:string) method that could retrieve the queue itself?

Thank you!

How to use this package in the real world

Hello,

I'm trying to run some cron actions. These actions a super simple, it's just a trigger of a command build using ace.

First, I do not know if i should use crontab from linux, a package like adonis5-schedule or this package.

I've tried adonis5-schedule and I do not know how to use it in production (i.e.: how to stop jobs when reploying my application.
For this package, I can use the feature repeat from bull-mq but how to setup these in Adonis? I tried a prldfile but each time the server wake up, a new worker is created which is not what I want and I do not know how to manage this in production? Should I run node ace queue:listen using pm2? That's hard to understand the purpose of this package and the problem solved.

So I think crontab is the best solution actually?

Laravel have a tasks scheduler with a lot of feature and documentation. Could we expect the same with Adonis6?

Queue.dispatch does not seem to trigger when fired from a command

I have a command created by node ace make:command where it does a Queue.dispatch('App/Jobs/ImportVariants', {id: "1"}).

The problem I have is neither the console.log is displayed nor does the record gets deleted (for testing purposes).

If I do the same Queue.dispatch('App/Jobs/ImportVariants', {id: "1"}) in node ace repl it works.

Within the app/Jobs/ImportVariants.ts I have:

import ShopifyStore from "App/Models/ShopifyStore"

export type ShopifyStorePayload = {
  id: string
}

export default class ImportVariants {
  public async handle(payload: ShopifyStorePayload){
    console.log(payload)
    let user = await ShopifyStore.findByOrFail('id', payload.id)
    user.delete()
  }
}

And contracts/queue.ts:

import type {ShopifyStorePayload} from 'App/Jobs/ImportVariants'

declare module '@ioc:Setten/Queue' {
  interface JobsList {
    'App/Jobs/ImportVariants': ShopifyStorePayload
  }
}

When we add queue with delay then queue is not working

Hi , when I am adding the queue with delay like this

await Queue.dispatch("App/Jobs/TestQueueJob", { type:"testing" }, { delay:5000 });

Then It's not working queue added in the Redis database but after that It's not processing. could you please help me in this .

but when we do normal dispatch without any delay then It's working fine .

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.