felixmosh / bull-board Goto Github PK

Due to CSP browser restrictions UI is not working any more in Chrome/FF. Please use window.location.pathname instead of passing basePath via window object (this would allow us to skip inlined <script> tag).

We're using Bull Board on a project, and it's been really helpful to visualize what's happening in our queue. However, we're finding that the completed and failed counts don't seem to reflect the current status of the queue. Our code gets all kinds of completed and failed events, which we use to drive our app. But we see nothing in the UI:

I do find that failed events in Bull don't always have the right err, which I've filed upstream in OptimalBits/bull#1326. But this looks like a Bull Board issue vs Bull.

import {  router } from 'bull-board';

app.use('/ui', router)

trace:
at Function.render (/Users/dev/platform/node_modules/express/lib/application.js:580:17) at ServerResponse.render (/Users/dev/platform/node_modules/express/lib/response.js:1012:7) at exports.entryPoint (/Users/dev/platform/node_modules/bull-board/src/routes/index.ts:8:7) at Layer.handle [as handle_request] (/Users/dev/platform/node_modules/express/lib/router/layer.js:95:5) at next (/Users/dev/platform/node_modules/express/lib/router/route.js:137:13) at Route.dispatch (/Users/dev/platform/node_modules/express/lib/router/route.js:112:3) at Layer.handle [as handle_request] (/Users/dev/platform/node_modules/express/lib/router/layer.js:95:5) at /Users/dev/platform/node_modules/express/lib/router/index.js:281:22 at Function.process_params (/Users/dev/platform/node_modules/express/lib/router/index.js:335:12) at next (/Users/dev/platform/node_modules/express/lib/router/index.js:275:10)

Ref.: https://keepachangelog.com/en/1.0.0/

Context

We're currently declaring the types for redis-info manually at src/declarations.d.ts, but recently we've got access to more complete types through the @types/redis-info package.

Proposal

We should change to use the types in DefinetilyTyped and update some of the types we declare locally to be defined in terms of them.

TypeError: queue.client.info is not a function
at getStats (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/getDataForQeues.js:12:22)
at getDataForQeues (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/getDataForQeues.js:71:23)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async handler (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/queues.js:5:5)
TypeError: queue.client.info is not a function
at getStats (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/getDataForQeues.js:12:22)
at getDataForQeues (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/getDataForQeues.js:71:23)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async handler (/Users/macbookpro/Documents/data/node_modules/bull-board/routes/queues.js:5:5)

//////////////////////////////

const { Queue, Worker, QueueEvents } = require('bullmq');

var cluster = require('cluster');
const { setQueues, UI } = require('bull-board')
const app = require('express')()
const port = 3010

var numWorkers = 8;
var queue = new Queue("Paint");
const queueEvents = new QueueEvents('Paint');

setQueues([queue])

if(cluster.isMaster){
for (var i = 0; i < numWorkers; i++) {
cluster.fork();
}

cluster.on('online', function(worker) {
// Lets create a few jobs for the queue workers
for(var i=0; i<500; i++){
queue.add('cars', { color: 'blue' });
};
});

cluster.on('exit', function(worker, code, signal) {
console.log('worker ' + worker.process.pid + ' died');
});
}else{
const worker = new Worker('Paint', async job => {
if (job.name === 'cars') {
await console.log(job.data.color);
}
});
queueEvents.on('waiting', ({ jobId }) => {
console.log(A job with ID ${jobId} is waiting);
});

queueEvents.on('active', ({ jobId, prev }) => {
    console.log(`Job ${jobId} is now active; previous status was ${prev}`);
});

queueEvents.on('completed', ({ jobId, returnvalue }) => {
    console.log(`${jobId} has completed and returned ${returnvalue}`);
});

queueEvents.on('failed', ({ jobId, failedReason }) => {
    console.log(`${jobId} has failed with reason ${failedReason}`);
});

}
app.use('/ui', UI)
app.listen(port, () => console.log(Example app listening at http://localhost:${port}))

Just trying your nice board! Well done 👏

Is it possible to see job logs logged with job.log()?

It would be nice to know what was logged during jobs runs.

Cheers! 🍻

This would be a breaking change, basically removing the createQueues method and instead of keeping track of the queues internally, we would rather accept queues as a parameter (probably to UI or a config method).

Major benefit would be that users have more control on how to use Bull on their own and maybe even supporting integration with other libs

A migration from JS to TS would mean type exposure, which is nice.

I want to use it in an application built by koa2，but I don't get it.
my code like:

const Koa = require('koa');
const app = new Koa();
const { UI } = require('bull-board');

app.use((ctx, next) => {
    if(ctx.path.startWith('/monitor')) {
        if (ctx.status === 404 || ctx.status === '404') {
            delete ctx.res.statusCode
        }
        ctx.respond = false;
        UI(ctx.req, ctx.res);
    } else {
        next();
    }
});

This is from Can we mount express app into koa?

Hi, I'm trying to test bull-board but using the example but I'm getting a screen with:

loading...

and not bull-board management screen:
Is the same as the example

const { createQueues } = require('bull-board')
 
	const redisConfig = {
  		redis: {
    		port: 6379,
    		host: "127.0.0.1"
  	},
	}
 
	const queues = createQueues(redisConfig)

	const helloQueue = queues.add('helloQueue') // adds a queue
	const helloQueueWithOpts = queues.add('helloQueue', {
  		prefix: 'hello',
	}) // adds a queue with QueueOptions https://github.com/OptimalBits/bull/blob/master/REFERENCE.md#queue
 
	// defines how the queue works
	helloQueue.process(async job => {
  		console.log(`Hello ${job.data.hello}`)
	})
 
	helloQueue.add({ hello: 'world' }) // adds a job to the queue

And later

const { UI } = require('bull-board')
 
	app.use('/admin/bullDash', UI)

Someone knows what is happening?

Testing out 1.0.0-alpha.5...

Current package version

• Bull => 3.13.0
• BullMQ => Not installed

Steps Taken

1.npm i [email protected]
2. tsc -p project_src

However tsc fails and I get the error below

node_modules/bull-board/dist/index.d.ts:2:34 - error TS2307: Cannot find module 'bullmq'. 2 import { Queue as QueueMq } from 'bullmq';

node_modules/bull-board/dist/index.d.ts:3:30 - error TS2307: Cannot find module 'express-serve-static-core'.
3 declare const router: import("express-serve-static-core").Express;

Note I tested 1.0.0-alpha.4 to confirm and got the same error

Also manually installed bullmq package and error did go away however I received this new error

node_modules/bullmq/dist/interfaces/sandboxed-job.d.ts:3:57 - error TS2304: Cannot find name 'Omit'.

3 export interface SandboxedJob<T = any, R = any> extends Omit<JobJson, 'data' | 'opts' | 'progress' | 'log' | 'returnValue'> {

When i tried to use bull-board, nothing happens.
With no queues i have "no stats to display" page, but if I add some existing queue like

import { createQueues } from 'bull-board'
const queues = createQueues(myConfig)
export const myQueue = queues.add('MyQueue')

And I try to use it, there is empty white page on '/admin/queues' and I have following errors in browser's console

TypeError: Expected a finite number, got number: NaN
    at e.exports (bundle.js:1)
    at $e (bundle.js:30)
    at Ai (bundle.js:22)
    at Vl (bundle.js:22)
    at Fl (bundle.js:22)
    at Ll (bundle.js:22)
    at Rl (bundle.js:22)
    at bundle.js:22
    at t.unstable_runWithPriority (bundle.js:30)
    at ca (bundle.js:22)

updated: I used bull-board api.

Seems to me there's some issue with the CSS for the pre blocks where the monospace font is not properly applied.

Board shows the data, but not the job name, can we get it to show this?

Hello, I'm wondering what "Blocked client" indicates on the UI, and if there is something I should do about them.

I'm trying to run bullboard but he just returning me a blank page with HTML code and i dunno how to solved it, can you guys help me:

This is the code i'm trying to run:

// bullboard.js

import {UI, setQueues} from 'bull-board'
import Queue from '../services/queueService'

setQueues(Queue.queues.map((queue) => queue.bull))

export default UI

// setup-app.js
import cors from '../middlewares/cors'
import jsonParser from '../middlewares/json-parser'
import rawParser from '../middlewares/raw-parser'
import contentType from '../middlewares/content-type'
import bodyParser from '../middlewares/body-parser'
import logger from '../middlewares/logger'
import bullboard from '../middlewares/bullboard'
import voyager from '../middlewares/voyager'

export default (app) => {
	app.disable('x-powered-by')
	app.use(cors)
	app.use(jsonParser)
	app.use(rawParser)
	app.use(contentType)
	app.use(bodyParser())
	app.use(logger)
	app.use(bullboard)
	app.use('/voyager', voyager)
}

// queueService.js
import Queue from 'bull'

import env from 'env-cat'
import * as jobs from '../jobs'
import * as logger from '../core/logger'

const queues = Object.values(jobs).map((job) => ({
	bull: new Queue(job.key, env.get('REDIS_URL')),
	name: job.key,
	handle: job.handle,
}))

export default {
	queues,
	add(name, data) {
		const queue = this.queues.find((queue) => queue.name === name)
		return queue.bull.add(data)
	},
	process() {
		return this.queues.forEach((queue) => {
			queue.bull.process(queue.handle)

			queue.bull.on('failed', (job, err) => {
				logger.info('Job failed', queue.key)
				logger.info(err)
			})
		})
	},
}

// jobs.js
export {default as Mail} from './Mail'
export {default as Task} from './Task'
export {default as Pipeline} from './Pipeline'
export {default as SegmentUpdate} from './SegmentUpdate'
export {default as SegmentDelete} from './SegmentDelete'

Node version: 12.16.1
NPM Version: 6.13.7

Pipeline doesn't run on forked PRs.

Solution

are you guys planning on supporting BullMQ 4 ?

We can start with a button to clean all the jobs in the delayed status, can probably be expanded to all the statuses

There are alredy two retry buttons. It would be nice to have some buttons to clean up all failed jobs and single jobs.

I was trying to run bullboard with Apollo Server, but somehow is not showing anymore the dashboard, when i try to access the is returning this to me.

I was starting the bullboard with a middleware and adding the queue with this code, after i debug aparently is normally working.

And adding to my application with this code.

Node version: v10.15.2
Bullboard version: ^0.5.0
Bull version: ^3.12.1

Is it possible to add the result (json) of a completed job into the dashboard to see what the output was?

Hello!

Love the screenshot look and tried setting this up for a redis hosted on Heroku.

It's very hard because there are no error messages anywhere and the documentation seems incomplete.

How do I know if my credentials are correct?
How do I specify queues?
When it "fails" it just ends up pending forever and then fails with MaxRetries, is that intended?

After reading around this is what I ended up with

const { createQueues, UI } = require('bull-board');
const app = require('express')();

const redisConfig = {
  redis: {
    port: process.env.REDIS_PORT,
    host: process.env.REDIS_HOST,
    password: process.env.REDIS_PASSWORD,
  },
}

const queues = createQueues(redisConfig);

queues.add('posts');

app.use('/admin/queues', UI)

app.listen( 3000, () => {
    console.log( `Server up and running on port 3000` );
});

I'd gladly help with PRs but right now I don't even know where to start 💃

It would be nice to have detailed under the Delayed jobs the information for when it's going to run again, if Bull can provide it

Also, very nice lib 😄

I was really excited about this project when I saw it, but I had trouble getting it to work. We have Bull queues running in production in a Docker Swarm with dozens of Fastify worker containers and several different redis endpoints. I tried using fastify.use() but it's not compatible I don't think.

But this kinda sucks anyways — we would have to embed the bull-board and all its dependencies (express, react, react-dom, ramda, etc) directly into our server images. I was hoping this would be more of a standalone server that you can point to a redis process and give it a prefix and it'll find your queues for you (via KEYS bull:* every so often). That would mean just adding one more container instead of embedding an instance of it to every worker image.

I think you already have the basis of it here, an UI + Express server, what do you think of modifying it to work without any required integration into your node app that is running Bull?

does all the orders appears in one page, no pagaination ?
i could implement one if no any

Explore Azure Pipelines.
Otherwise, a suggestion would be using Travis

Hey folks! first of all, I hope to all of you having a good and stay healthy time on this covid-19 hard times.

I don't know if this could be considered as an issue but I was reading the source code and found that there a few extra definitions that are missing on the README

router.put('/queues/:queueName/retry', wrapAsync(retryAll))
router.put('/queues/:queueName/:id/retry', wrapAsync(retryJob))
router.put('/queues/:queueName/:id/promote', wrapAsync(promoteJob))
router.put('/queues/:queueName/clean/:queueStatus', wrapAsync(cleanAll))

so want to ask if there is an official doc or something to understand what are those routes and which role are playing on the board module

besides that, I didn't found in README regarding my server being hit at this route GET /queues/queues/? and the server is replying with 304 🤔

if anyone can help me will be very appreciated

As part of monitoring I'd love to see results of individual jobs. I imagine this must be possible by somehow altering what is shown in the data-field as part of the 'completed' tab.

How would I instruct bull to show custom data?

for ref:

The value returned by your process function will be stored in the jobs object and can be accessed later on, for example in a listener for the completed event

Is the object / value that is returned by the consumer available in the dashboard? I'm seeing the object that is passed to the consumer, and the options — but I don't actually see the value that is returned.

Hey, thanks for this great dashboard,
i am not sure, but after a job finishes , the progress bar doesnt go to 100% it gust go to finished tab,
but the percentege progress still 0,

also this is my code :-

myFirstQueue.add('normal', { name: 'Hello', });

myFirstQueue.process('normal', (job, done) => { for (let i=0;i<=1000;i++){ file.write('Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.'); } done(); });

how could i make the process take time , so i can see the progess in the dashboard,
would appreciate any help,
and thanks for this great work

GET /queues
returns
{"status":500,"message":"Cannot use 'in' operator to search for 'redis_version' in undefined"}

when I return new Redis.Cluster(...) in createClient while creating Bull.Queue

When creating a job with the following properties

await queue.add('job-name', { foo: 'bar' }, {
         repeat: {
           every: 1000,
           startDate: new Date(),
           tz: "America/Los_Angles",
         },
      });

Then clicking on the delayed tab in the UI I get the following error

RangeError: Invalid time value
    at bundle.js:30
    at delay (bundle.js:30)
    at to (bundle.js:22)
    at Gl (bundle.js:22)
    at gu (bundle.js:22)
    at mu (bundle.js:22)
    at ru (bundle.js:22)
    at bundle.js:22
    at t.unstable_runWithPriority (bundle.js:30)
    at La (bundle.js:22)

Hello.

First of all, thank you for making a nice and clean interface for bull. I use it in my project that syncs data between databases. I do the work using batches, so I never know the max value of things that need processing in order to calculate the percentage. So if i do job.progress(item.count) where item.count is for example 4235 items, the frontend interface will go crazy because the green progressbar will be style="width: 4235%".

What im proposing is a simple check, something along the lines of:
if (job.progress > 100) // Don't render class="progress-bar", style="width: 4235%" and the % sign

progress: ({ job }) => {
    switch (type(job.progress)) {
      case 'Object':
        return (
          <Highlight className="json">
            {JSON.stringify(job.progress, null, 2)}
          </Highlight>
        )
      case 'Number':
       if (job.progress > 100)  {
            return (
          <div className="progress-wrapper"> {job.progress}</div>
        )
        }
        return (
          <div className="progress-wrapper">
            <div
              className="progress-bar"
              style={{
                width: `${job.progress}%`,
              }}
            >
              {job.progress}
              %&nbsp;
            </div>
          </div>
        )
      default:
        return '--'
    }
  },

Thanks in advance

Integration tests, a suggestion is to use Dockest to spin up a redis instance so it would be possible to add/remove jobs in queues and test the code

Currently the Bull-board is not providing the needed info.

For example when a job fails we retry it X amount of times depending on the endpoint and this returns ‘job stalled more than allowable limit’ when it fails. Before in Kueue this would give the data object which contained the reason for failing.

Currently it's only possible to see the complete message by calling ‘jobs/failed/retrieve/0/99999999’ endpoint.

For this I would like the DATA and OPS object to be added to the failed job or an other solution would be to add a link to retrieve the complete message.

Solution one extend the array of queue(ui/components/Queue.js 63) :
failed: ['id', 'failedReason','data', 'timestamps', 'progress', 'attempts', 'retry'],

How can we integrate the UI to nestjs project where queues are alreay created and maanged by @nest/bull package?

Hello!

Thank you very much for the work done.
Your UI looks great.
But I have a problem with CSS.

This can probably be solved with:

pre {
      white-space: pre-wrap;
}

Hello
If you will try to setQueue(bullInstance) to the board and then bullInstance.close() => it will produce error "Error: Connection is closed.", please make it just removing it from dashboard

Reported by sentry, I'm not sure on which specific conditions this happens:

TypeError: Cannot read property 'id' of null File "/usr/src/app/node_modules/bull-board/routes/getDataForQeues.js", line 24, col 13, in formatJob id: job.id, ?, in Array.map File "/usr/src/app/node_modules/bull-board/routes/getDataForQeues.js", line 67, col 20, in null.<anonymous> jobs: jobs.map(formatJob), ?, in runMicrotasks File "internal/process/task_queues.js", line 93, col 5, in processTicksAndRejections ?, in async Promise.all File "/usr/src/app/node_modules/bull-board/routes/getDataForQeues.js", line 54, col 18, in async getDataForQeues const counts = await Promise.all( File "/usr/src/app/node_modules/bull-board/routes/queues.js", line 5, col 5, in async handler await getDataForQeues({

A website will allow for a more structured way of showing documentation

eslint was set up with the minimal requirements for prettier, but it still doesn't have any rules.

we could at least have eslint:recommended in the extends list and make it work with both React and Node env (routes/ and ui/ folders)

I'm using bull-board in a cluster setup and whenever it pulls for new data, the page will become completely empty probably due to different data returned from the workers.

bull-board currently uses vorpal as a dependency, which was last updated in August of 2018.

There are three high-risk security vulnerabilities in a dependency of vorpal relating to lodash which were fixed in 2017, see them using npm audit

Path for all, bull-repl > vorpal > inquirer > lodash

• https://npmjs.com/advisories/577
• https://npmjs.com/advisories/782
• https://npmjs.com/advisories/1065

It seems to me the alternatives are to use instead either:

1. https://github.com/tj/commander.js
2. https://github.com/oclif/oclif
3. https://github.com/yargs/yargs
4. https://github.com/vorpaljs-reforged/vorpal

I am not familiar with Vorpal or Oclif which would we prefer? This security issue really must be addressed and since the only way to do it is to change dependencies, as the one it's in is dead, there needs to be some discussed about what CLI bull-board is going to in-future use.

if i added more than one queue would the UI create another tabes for the new queue,
ar all queues insert in the same tabs ?