rq / flask-rq2 Goto Github PK

A Flask extension for RQ.

Home Page: https://flask-rq2.readthedocs.io/

License: MIT License

Python 100.00%

rq python tasks queue redis jobs scheduler periodic periodic-jobs

flask-rq2's Introduction

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

RQ requires Redis >= 3.0.0.

Full documentation can be found here.

Support RQ

If you find RQ useful, please consider supporting this project via Tidelift.

Getting started

First, run a Redis server, of course:

$ redis-server

To put jobs on queues, you don't have to do anything special, just define your typically lengthy or blocking function:

import requests

def count_words_at_url(url):
    """Just an example function that's called async."""
    resp = requests.get(url)
    return len(resp.text.split())

Then, create an RQ queue:

from redis import Redis
from rq import Queue

queue = Queue(connection=Redis())

And enqueue the function call:

from my_module import count_words_at_url
job = queue.enqueue(count_words_at_url, 'http://nvie.com')

Scheduling jobs are also similarly easy:

# Schedule job to run at 9:15, October 10th
job = queue.enqueue_at(datetime(2019, 10, 10, 9, 15), say_hello)

# Schedule job to run in 10 seconds
job = queue.enqueue_in(timedelta(seconds=10), say_hello)

Retrying failed jobs is also supported:

from rq import Retry

# Retry up to 3 times, failed job will be requeued immediately
queue.enqueue(say_hello, retry=Retry(max=3))

# Retry up to 3 times, with configurable intervals between retries
queue.enqueue(say_hello, retry=Retry(max=3, interval=[10, 30, 60]))

For a more complete example, refer to the docs. But this is the essence.

The worker

To start executing enqueued function calls in the background, start a worker from your project's directory:

$ rq worker --with-scheduler
*** Listening for work on default
Got count_words_at_url('http://nvie.com') from default
Job result = 818
*** Listening for work on default

That's about it.

Installation

Simply use the following command to install the latest released version:

pip install rq

If you want the cutting edge version (that may well be broken), use this:

pip install git+https://github.com/rq/rq.git@master#egg=rq

Related Projects

If you use RQ, Check out these below repos which might be useful in your rq based project.

Project history

This project has been inspired by the good parts of Celery, Resque and this snippet, and has been created as a lightweight alternative to the heaviness of Celery or other AMQP-based queueing implementations.

flask-rq2's People

Contributors

Stargazers

Watchers

flask-rq2's Issues

Cancel a running job?

How can I cancel a running job? Is it possible?

rq timeout set in application factory is not taken into account

Hello, we would like to set a timeout in a rq job.
To do so, we first used the rq.job decorator i.e.

@rq.job(timeout=1)
def wait():
    time.sleep(10)

In that case when launching the job, it timeouts correctly.
However, when using the application factory pattern, we tried the following but does not work.

def create_app(some_arg):
    app = Flask(__name__)

    from jobs.ex import rq
    rq.init_app(app)
    rq.default_timeout = 1 # that does not work

I'd like to know whether there is another way to implement the equivalent feature as above, using the application factory pattern ?

Initial Update

Hi 👊

This is my first visit to this fine repo, but it seems you have been working hard to keep all dependencies updated so far.

Once you have closed this issue, I'll create separate pull requests for every update as soon as I find one.

That's it for now!

Happy merging! 🤖

How to keep the rq worker active when there's job waiting to process

Dear Jezdez,

I have a flask web app and I am trying to use your flask-rq2 to manage the async queue.
Due to the nature of async, I've written a separate worker. (I've tried to start the worker inside my web app but without success.). The code the worker is simple.

import os
from app import rq, create_app

app = create_app(os.getenv('FLASK_CONFIG') or 'default')
with app.app_context():
    worker = rq.get_worker("high")
    worker.work(burst=True)

when I run my web app, I start the worker simply using: python worker.py.

It worked to listen to the queue and process the job inside the queue. However, when the jobs are done, it quit automatically.

Is there a way to keep this worker active all the time. or how it can start automatically when there's job enter the queue?

I'm new to the queue management with rq. any best practice advice to use your package would be very appreciated.

Thanks a lot,

RQ function that saves output to file saves same output even after function is modified.

I have a function which I run using flask-rq which takes some input and writes to a new excel file in the server. The name of the file is result_{yyyy__mm__dd}.xlsx, hence the name of the file doesn't change for a day. This resulted in lot of trouble for me. After I noticed some error in the output I modified the function but I kept getting the same output (original output files were deleted). I finally decided to change the name of file and then I got the expected output. Is this a bug or a feature? Is redis actually storing the result, if so how can I change that?

Please, update docs; form_url exists no more

Docs cite a method called form_url, which is no longer available at redis-py. https://flask-rq2.readthedocs.io/en/latest/#rq-redis-url

Ability to set timeout at enque time

Hi,

I'd like to be able to set the job timeout when queuing the job instead of when declaring it with the decorator.

Alternatively it would be good to be able to set it via a flask config variable.

NoRedisConnectionException on starting app

When running python app.py on the attached app.py. I'm running into a NoRedisConnectionException

when RQ(app) is called - the init_app method calls connect
stack.top is None and no rq_redis is returned

      @property
      def connection(self):
          ctx = stack.top
          if ctx is not None:
              if not hasattr(ctx, 'rq_redis'):
                  ctx.rq_redis = self._connect()
              return ctx.rq_redis

using python 3.4.2

Flask-RQ2 (16.1.1)
Flask (0.11)
rq (0.6.0)
rq-scheduler (0.7.0)


from flask import Flask
from flask_rq2 import RQ
from datetime import datetime, timedelta

app = Flask(__name__)

app.config['RQ_REDIS_URL'] = 'redis://localhost:6379/0'

rq = RQ(app)

@rq.job
def add(x,y):
    return x+y

#add.schedule(timedelta(seconds=60), 1,2)
add.queue(1,2)

if __name__ == '__main__':
    app.run()

Traceback (most recent call last):
  File "app.py", line 15, in <module>
    add.queue(1,2)
  File "/Users/cchowdvi/.virtualenvs/py3tfsheroku/lib/python3.4/site-packages/flask_rq2/helpers.py", line 45, in queue
    return self.rq.get_queue(self.queue_name).enqueue_call(
  File "/Users/cchowdvi/.virtualenvs/py3tfsheroku/lib/python3.4/site-packages/flask_rq2/app.py", line 324, in get_queue
    connection=self.connection,
  File "/Users/cchowdvi/.virtualenvs/py3tfsheroku/lib/python3.4/site-packages/rq/queue.py", line 59, in **init**
    self.connection = resolve_connection(connection)
  File "/Users/cchowdvi/.virtualenvs/py3tfsheroku/lib/python3.4/site-packages/rq/connections.py", line 73, in resolve_connection
    raise NoRedisConnectionException('Could not resolve a Redis connection')
rq.connections.NoRedisConnectionException: Could not resolve a Redis connection

Reenable pyup.io

Hi @nvie,

would you mind re-enabling pyup.io again or give me enough permissions to be able to to it myself?

Thanks!

AttributeError: 'module' object has no attribute 'add'

Hello, I'm getting this error on the worker when enqueue this simple job:

from flask import Flask
from flask_rq2 import RQ

app = Flask(__name__)

rq = RQ(app)

@rq.job
def add(x, y):
    return x + y

@app.route('/test')
def my_test():
    job = add.queue(1, 2)
    return 'ok'

19:14:43 default: __main__.add(1, 2) (75ab9148-4aca-44bc-a41e-cffd368509ba)
19:14:43 AttributeError: 'module' object has no attribute 'add'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python2.7/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/site-packages/rq/job.py", line 206, in func
    return import_attribute(self.func_name)
  File "/usr/local/lib/python2.7/site-packages/rq/utils.py", line 151, in import_attribute
    return getattr(module, attribute)
AttributeError: 'module' object has no attribute 'add'
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/rq/worker.py", line 700, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python2.7/site-packages/rq/job.py", line 500, in perform
    self._result = self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python2.7/site-packages/rq/job.py", line 206, in func
    return import_attribute(self.func_name)
  File "/usr/local/lib/python2.7/site-packages/rq/utils.py", line 151, in import_attribute
    return getattr(module, attribute)
AttributeError: 'module' object has no attribute 'add'
19:14:43 Moving job to u'failed' queue

I'm running everything inside this docker container:

FROM tiangolo/uwsgi-nginx-flask:flask-upload

RUN pip install rq
RUN pip install Flask-RQ2

Hope you can help me.

Flask-RQ2 is not compatible with click 8.0.0, required for Flask 2.0

Under click 8.0.0 (released May 12, 2021):

myapp/__init__.py:83: in <module>
    rq.init_app(app)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/flask_rq2/app.py:193: in init_app
    self.init_cli(app)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/flask_rq2/app.py:207: in init_cli
    from .cli import add_commands
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/flask_rq2/cli.py:14: in <module>
    from rq.cli import cli as rq_cli
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/rq/cli/__init__.py:2: in <module>
    from .cli import main
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/rq/cli/cli.py:98: in <module>
    @pass_cli_config
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/rq/cli/cli.py:77: in pass_cli_config
    func = option(func)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/click/decorators.py:247: in decorator
    _param_memo(f, OptionClass(param_decls, **option_attrs))
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/click/core.py:2467: in __init__
    super().__init__(param_decls, type=type, multiple=multiple, **attrs)
../../../virtualenv/python3.7.1/lib/python3.7/site-packages/click/core.py:2108: in __init__
    ) from None
ValueError: 'default' must be a list when 'multiple' is true.

Upgrading to redis>=3.0 results in failed job

With redis==2.10.6 all is working as expected. After upgrading to the latest redis>=3.0 jobs start to fail with AttributeError: 'int' object has no attribute 'items' exceptions.

I created a simple job:

from flask_rq2 import RQ

rq = RQ()

@rq.job
def add(x, y):
    return x + y

Running Flask RQ worker as usual: $ flask rq worker --worker-ttl=420

08:51:13 RQ worker 'rq:worker:localhost.4064' started, version 0.12.0
08:51:13 *** Listening on default...
08:51:13 Cleaning registries for queue: default
08:51:22 default: flask_demo.jobs.add(1, 2) (48624d9a-44fd-4ce5-8190-35d1d1a44a8b)
Traceback (most recent call last):
  File "/Users/user/.virtualenvs/flask_demo/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/flask/cli.py", line 894, in main
    cli.main(args=args, prog_name=name)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/flask/cli.py", line 557, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/flask/cli.py", line 412, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/flask_rq2/cli.py", line 55, in new_func
    return func(rq, ctx, *args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/flask_rq2/cli.py", line 154, in worker
    **shared_options(rq)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/cli/cli.py", line 75, in wrapper
    return ctx.invoke(func, cli_config, *args[1:], **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/cli/cli.py", line 236, in worker
    worker.work(burst=burst, logging_level=logging_level)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 493, in work
    self.execute_job(job, queue)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 662, in execute_job
    self.fork_work_horse(job, queue)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 599, in fork_work_horse
    self.main_work_horse(job, queue)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 677, in main_work_horse
    success = self.perform_job(job, queue)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 781, in perform_job
    self.prepare_job_execution(job)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/worker.py", line 706, in prepare_job_execution
    registry.add(job, timeout, pipeline=pipeline)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/rq/registry.py", line 47, in add
    return pipeline.zadd(self.key, score, job.id)
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/redis/client.py", line 2263, in zadd
    for pair in iteritems(mapping):
  File "/Users/user/.virtualenvs/flask_demo/lib/python3.7/site-packages/redis/_compat.py", line 123, in iteritems
    return iter(x.items())
AttributeError: 'int' object has no attribute 'items'
08:51:22 Moving job to 'failed' queue (work-horse terminated unexpectedly; waitpid returned 256)

Incompatibility with RQ >= 0.7.0

Since version 0.7.0 of RQ, I get the following error when starting a worker:

> ./app/manage.py rq worker
Traceback (most recent call last):
  File "./app/manage.py", line 96, in <module>
    manager.run()
  File "/home/bertrand/code/web-findings/venv/lib/python3.5/site-packages/flask_script/__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/home/bertrand/code/web-findings/venv/lib/python3.5/site-packages/flask_script/__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "/home/bertrand/code/web-findings/venv/lib/python3.5/site-packages/flask_rq2/script.py", line 42, in __call__
    return self.run(*args, **kwargs)
  File "/home/bertrand/code/web-findings/venv/lib/python3.5/site-packages/flask_rq2/script.py", line 179, in run
    queues=queues or self.rq.queues,
TypeError: <flask_rq2.script.WorkerCommand object at 0x7fac62dd5630>: worker() missing 1 required positional argument: 'connection_class'

It looks like this is due to rq/rq#741 that added the connection_class parameter to the worker command.

Note that Flask-RQ2 only requires rq>=0.6.0. I suppose this requirement should be updated until the incompatibility is fixed.

Cannot set result_ttl=0 in RQ.job

https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/app.py#L262
should be
result_ttl=DEFAULT_RESULT_TTL if result_ttl is None else result_ttl
so if result_ttl = 0 passed value is not overwritten with DEFAULT_RESULT_TTL

Add a testing option to use fakeredis

When writing unit tests it would be useful to have an option to use fakeredis instead of a real redis server. Are there any accepted methods that I am missing or would it make sense to integrate it directly into this library?

One approach is described by python-rq but I am not sure how to best integrate it.

For context, I am using Flask-base and would like to test the registration aspect, which creates a queue event but fails since there is no running redis server.

Launching flask-rq2 using supervisord

For whatever reason, launching flask-rq2 from supervisord seems to be giving me incredible difficulty which I have never encountered using vanilla rq or even running a Flask app with RQ2 in docker. And of course, it seems that supervisord makes debugging incredibly difficult.

Currently, I have a fairly straight-forward Flask app which is running on an Ubuntu server. Everything is inside a virtual environment, and I am launching as follows:

rq.conf

[program:rq]
directory=/home/<user>/ceams
command=sh boot-workers.sh
process_name=$(program_name)s-%(process_num)s
numprocs=1
autostart=true
autorestart=true
stopsignal=Term

boot-workers.sh

#!/bin/bash
set -eu

. venv/bin/activate
flask rq worker

Whenever I log in and launch boot-workers manually, everything works like a charm. With supervisord, it launches for a few seconds before exiting (with status 1) with the following error:

KeyError: <flask.cli.ScriptInfo object at 0x7f2c8ca77828>
Traceback (most recent call last):
  File "/home/ceams_admin/ceams/venv/bin/flask", line 11, in <module>
    sys.exit(main())
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 894, in main
    cli.main(args=args, prog_name=name)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 557, in main
    return super(FlaskGroup, self).main(*args, **kwargs)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/click/core.py", line 1132, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/click/core.py", line 1171, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 515, in get_command
    rv = info.load_app().cli.get_command(ctx, name)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 377, in load_app
    raise_if_not_found=False)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 254, in locate_app
    return find_best_app(script_info, module)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 76, in find_best_app
    app = call_factory(script_info, app_factory)
  File "/home/ceams_admin/ceams/venv/lib/python3.6/site-packages/flask/cli.py", line 114, in call_factory
    return app_factory(script_info)
  File "/home/ceams_admin/ceams/app/__init__.py", line 29, in create_app
    app.config.from_object(config[config_name])

It appears the virtual environment isn't being activated or something like that, though I'm not sure why. Has anyone successfully launched rq2 from supervisord? If so, can someone provide some debugging advice and/or an example? Thanks!

RQ v0.11 changed worker_ttl

The new version of RQ (v0.11) changed the way the default worker_ttl is accepted:

rq/rq@3133d94#diff-1aac421e8c53e895ee40f3b7dc63144fR158

So using Flask-RQ2's CLI breaks. The following works around this issue (by explicitly setting the default), until it is updated:

flask rq worker --worker-ttl 420

This fix would be to not pass the None to RQ, or to teach Flask-RQ2 the default in RQ.

List running workers

Is there a way to get a list of all currently running workers?

how to empty cron jobs

I found that after changing the cron task name, the service was restarted. The old cron task could not be deleted and remains in redis. How can I handle this situation?

Timeout in cron

Hi there,

The module rq-scheduler permit to set a timeout in cronjobs, but this flask plugin not allows

You can view cron function definition in https://github.com/ui/rq-scheduler/blob/master/rq_scheduler/scheduler.py#L188
def cron(self, cron_string, func, args=None, kwargs=None, repeat=None, queue_name=None, id=None, timeout=None, description=None):

But in flask plugin the call is this
def cron(self, pattern, name, *args, **kwargs):

I need to set timeout to cronjobs according rq_scheduler module.

Kind regards

environment variable RQ_SCHEDULER_INTERVAL is not used

https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/cli.py#L167
sets default=60.0, so get_scheduler always receives value from click.

Environment variable RQ_SCHEDULER_INTERVAL saved here https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/app.py#L127 to self.scheduler_interval is never used here cause
https://github.com/jezdez/Flask-RQ2/blob/master/src/flask_rq2/app.py#L290 interval variable is always not None.

Not all the worker launched if use supervisor to launch multiple workers

Hello there,

I'm host my app in docker and launch the rq worker with supervisor.

I create a bash script to help launch the worker, the core code is:

#!/usr/bin/env bash

worker_name=$1
queues=$2
launch_time=`date +"%s"`

flask rq worker -n ${worker_name}.${launch_time} ${queues}
exit $?

I tried launch 12 workers by supervisor. But only part of the workers launched and can be find in rq-dashboard. But all the process of each worker launched by script were up.

Are you have any idea about this issue?

And I found another weird issue. The queues of worker become blank in rq-dashboard, and the worker not process any job after that when the working running for a long time.

I had to restart the worker to take it back. When I review the log of supervisord and rq worker, nothing found.

Are you know what caused this issue?

Job running without Flask app context

Hi, I'm trying to save to my db in one of my jobs, but I am getting the out of context error: RuntimeError: No application found. Either work inside a view function or push an application context.

Here's a simple example that tests for the app context.

$ export FLASK_APP=app.py
$ flask run

# app.py

from flask import Flask, has_app_context
from flask_rq2 import RQ

app = Flask(__name__)
app.config['RQ_REDIS_URL'] = 'redis://127.0.0.1:6379/0'

rq = RQ(app)


@rq.job
def add(x, y):
    print(has_app_context()) # False
    return x + y


@app.route('/')
def index():
    add.queue(1, 3)
    return 'hello world!'

I can get around this by using ScriptInfo().load_app(), or with a delayed app instance import, but that's not ideal.

from flask.cli import ScriptInfo

@rq.job
def add(x, y):
    with ScriptInfo().load_app().app_context():
        print(has_app_context()) # True
    return x + y

@rq.job
def add(x, y):
    from module_where_app_is_created import app
    with app.app_context():
        print(has_app_context()) # True
    return x + y

Add getting started section to docs

This could use the code or part of it in https://github.com/jezdez/rq2test.

Setting interval for rqscheduler doesn't work

Hi,

I have in issue setting interval for rqscheduler:
TypeError: unsupported operand type(s) for -: 'str' and 'float'.
Since setting this parameter with rqscheduler usually works fine, I guessed it was flask_rq2 that instantiates a scheduler passing it a string.

So I changed in cli.py line 192:
scheduler = rq.get_scheduler(interval=interval, queue=queue)
to
scheduler = rq.get_scheduler(interval=int(interval), queue=queue)
And now it works.

Is it a bug in my setup or a bug in Flask_RQ2 ?
If it's the latest, do you want a PR ?

Upcoming release of RQ 1.0

Hey there, just want to give a heads up about the next major RQ release, v1.0. I plan on releasing this new version in the second half of Feb (although this may change due to work commitments).

This release will have a few backward incompatible changes, the biggest one is the introduction of FailedJobRegistry that replaces FailedQueue. I have documented the new registry here.

The branch for the upcoming release is here: https://github.com/rq/rq/tree/v1.0
You may also find the changelog helpful: https://github.com/rq/rq/blob/v1.0/CHANGES.md

Let me know if there's any questions. Thanks!

Queuing Python Static Methods.

Hello,

I am looking to queue some of the Python @staticmethods. For some reason, I am getting following error when using Python RQ/ Flask-RQ2. Please help.

@staticmethod
@rq.job
def add_request(request):
    db.session.add(request)
    db.session.commit()

@staticmethod
@rq.job
def monitor_request(id):
    job = Job.fetch(id)
    accepted_statues = ['finished','failed']
    while job.status not in accepted_statues:
        time.sleep(1)
    request = Requests.query.filter_by(jobid=job.id).first()
    request.status = job.status
    db.session.add(request)
    db.session.commit()

15:30:06 AttributeError: 'module' object has no attribute 'add_request'
Traceback (most recent call last):
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/worker.py", line 789, in perform_job
rv = job.perform()
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 573, in perform
self._result = self._execute()
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 579, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 206, in func
return import_attribute(self.func_name)
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/utils.py", line 153, in import_attribute
return getattr(module, attribute)
AttributeError: 'module' object has no attribute 'add_request'
Traceback (most recent call last):
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/worker.py", line 789, in perform_job
rv = job.perform()
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 573, in perform
self._result = self._execute()
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 579, in _execute
return self.func(*self.args, **self.kwargs)
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/job.py", line 206, in func
return import_attribute(self.func_name)
File "/home/PROD.CAN/rsojshah/netaut/lib/python2.7/site-packages/rq/utils.py", line 153, in import_attribute
return getattr(module, attribute)
AttributeError: 'module' object has no attribute 'add_request'
15:30:06 Moving job to u'failed' queue

Advantages over pure RQ

I think the documentation would benefit from an explanation why we need Flask-RQ2 if we could use pure RQ as well. I have a hard time spotting the differences.

Handling of `RQ_ASYNC` is confusing when using the application factory pattern

When using the application factory pattern, the RQ_ASYNC configuration is only respected if it has not already been set for the task queue being initialized.

Concrete example to show the behaviour:

rq = RQ()  # rq._is_async is set to None
app = Flask(__name__)
app.config['RQ_ASYNC'] = False
rq.init_app(app)  # rq._is_async is set to False
app.config['RQ_ASYNC'] = True
rq.init_app(app)  # rq._is_async is unchanged

I'm not sure if this is a bug or not - from a look at the code, my guess is that the intention is that RQ_ASYNC does not override constructing the queue with the is_async argument set manually, which seems like a sensible stance. However, it does seem a bit strange - altering any of the other RQ_X options will take effect on a re-init.

What is the expected behavior here?

Setting RQ_ASYNC = False doesn't affect scheduling

Hey,

in my tests I use RQ_ASYNC = False which works fine for .queue(), but seems to do nothing for .schedule().

Thank you in advance!

Redis sentinel mode connection support.

Looks like for now Flask-RQ2 can't support redis sentinel connection. It just use the connection_class.form_url for client creation.

def connection(self):
if not self._ready_to_connect:
raise RuntimeError('Flask-RQ2 is not ready yet to connect to '
'Redis. Was it initialized with a Flask app?')
if self._connection is None:
self._connection = self._connect()
return self._connection

def _connect(self):
    connection_class = import_attribute(self.connection_class)
    return connection_class.from_url(self.redis_url)

[feature request] Unique job per job_id

Hello; it would be nice to be able to only add a job to the queue given another job with the same identifier (or arguments?) is there.

python3.7 when run rq2 in flask with error of "syntax"

[version]
python3.7.0
Flask-RQ2 18.0
Flask 1.0.2

[error]
def init(self, app=None, default_timeout=None, async=None):
^
SyntaxError: invalid syntax

Define rq-scheduler version in the dependency

There is no defined rq-scheduler version in the dependency list. As such, it fetches the latest version.

As it stands, the latest version of rq-scheduler (0.8.1) is not compatible withFlask-RQ2

Here is the error:

Traceback (most recent call last):
  File "manage.py", line 60, in <module>
    manager.run()
  File "/Users/joaocosta/.virtualenvs/chat-api/lib/python2.7/site-packages/flask_script/__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/Users/joaocosta/.virtualenvs/chat-api/lib/python2.7/site-packages/flask_script/__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "/Users/joaocosta/.virtualenvs/chat-api/lib/python2.7/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "manage.py", line 23, in scheduler
    scheduler = rq.get_scheduler()
  File "/Users/joaocosta/.virtualenvs/chat-api/lib/python2.7/site-packages/flask_rq2/app.py", line 288, in get_scheduler
    raise RuntimeError('Cannot import rq-scheduler. Is it installed?')
RuntimeError: Cannot import rq-scheduler. Is it installed?

I don't know what is the latest version of rq-scheduler that works but 0.7.0 is compatible.

Use @job decorator for class methods

Hey,

Thanks for great module!

I have some problems with using @job decorator with class methods. Check code below:

from flask import current_app
from flask_rq2 import RQ
rq = RQ(current_app)


class Test(object):
    @rq.job
    @classmethod
    def hello(cls):
        print('Hello world')


# add new task in queue
Test.hello.queue()

In worker logs I got:

AttributeError: 'module' object has no attribute 'hello'

Any ideas? I will be grateful for any tips.

how to do sth after job finished?

when the job finished or failed, i want hook a function that can write status to somewhere ,for example, mysql table. etc.
i want to use job.status to record the status to a table. how to do this

is this project still active?

Hello, RQ is a well maintained project and I would like to use it my flask app but seeing last release is from 2 years ago, master ci is failing and there are lots of unanswered issues and prs, flask-rq2 project seems to deprecated.

So my question is, is it better to create by own rq wrapper in my flask app instead of using this project?

unable to start rq worker on windows?

Hi
When I run python manage.py rq worker
Then throw error message: AttributeError: 'module' object has no attribute 'fork'
i's not supported on windows platform?
Thank you

E:\workspace2\H-Service>python manage.py rq worker
D:\Anaconda2\lib\site-packages\flask_cache\jinja2ext.py:33: ExtDeprecationWarnin
g: Importing flask.ext.cache is deprecated, use flask_cache instead.
  from flask.ext.cache import make_template_fragment_key
21:16:39 RQ worker u'rq:worker:NOTEBOOK66.7220' started, version 0.6.0
21:16:39 Cleaning registries for queue: default
21:16:39
21:16:39 *** Listening on ?[32mdefault?[39;49;00m...
21:17:19 ?[32mdefault?[39;49;00m: ?[34mapp.jobs.send_message('a', 'b')?[39;49;00m (324bec07-ed74-4d65-9504-71621ce97159)
Traceback (most recent call last):
  File "manage.py", line 13, in <module>
    manager.run()
  File "D:\Anaconda2\lib\site-packages\flask_script\__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "D:\Anaconda2\lib\site-packages\flask_script\__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "D:\Anaconda2\lib\site-packages\flask_rq2\script.py", line 42, in __call__
    return self.run(*args, **kwargs)
  File "D:\Anaconda2\lib\site-packages\flask_rq2\script.py", line 180, in run
    queues=queues or self.rq.queues,
  File "D:\Anaconda2\lib\site-packages\rq\cli\cli.py", line 204, in worker
    w.work(burst=burst)
  File "D:\Anaconda2\lib\site-packages\rq\worker.py", line 450, in work
    self.execute_job(job, queue)
  File "D:\Anaconda2\lib\site-packages\rq\worker.py", line 511, in execute_job
    child_pid = os.fork()
AttributeError: 'module' object has no attribute 'fork'

manage.py:

import os
from flask_rq2.script import RQManager
from flask_script import Manager

from app import create_app, rq

app = create_app(os.getenv('FLASK_CONFIG') or 'default')
manager = Manager(app)
manager.add_command('rq', RQManager(rq))

if __name__ == '__main__':
    manager.run()

Scheduler --burst option not supported in CLI

Since 0.7.0 rq-scheduler has nice option to work in burst mode, similar to rq worker yet this option is not supported by CLI interface in Flask-RQ2.

Is rq.schedule() not working?

Is rq.schedule(...) not implemented or not working? I try as in doc, but nothing happen. But rq.queue(...) working fine.

@rq.job
def add(x, y):
    return x + y

# queue job at a certain datetime (UTC!) from other function.
add.schedule(datetime(2016, 12, 31, 23, 59, 59), 1, 2)

using app = Flask(name)

When using app = Flask(__name__) (as stated in doc), your management script won't work dying on cryptic import like:

(tvocr) starenka /work/tvocr master at a7d02b8? % manage.py rq worker default
Traceback (most recent call last):
  File "/data/.envs/tvocr/bin/manage.py", line 6, in <module>
    exec(compile(open(__file__).read(), __file__, 'exec'))
  File "/work/tvocr/tvocr/api/manage.py", line 13, in <module>
    manager.run()
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/flask_script/__init__.py", line 417, in run
    result = self.handle(argv[0], argv[1:])
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/flask_script/__init__.py", line 386, in handle
    res = handle(*args, **config)
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/flask_rq2/script.py", line 42, in __call__
    return self.run(*args, **kwargs)
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/flask_rq2/script.py", line 180, in run
    queues=queues or self.rq.queues,
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/rq/cli/cli.py", line 175, in worker
    worker_class = import_attribute(worker_class)
  File "/data/.envs/tvocr/local/lib/python2.7/site-packages/rq/utils.py", line 150, in import_attribute
    module = importlib.import_module(module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named backend_tvocr.api.app

using app = Flask('somename') would work fine.

Not sure if I missunderstand flask or rq2 or both, but bare with me and maybe fix the docs (or the code) ;)

[help] How to get job by job_id in Flask-RQ2

I added a job, but I found that I have no way to get it through job_id. What should I do?

tasks.run.queue("test", 1234, a=1, b=2, job_id="test-job")

Support passing arguments to worker constructor

Though rq.worker.Worker constructor accepts many arguments (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L159
), it seems that Flask-RQ2 does not support passing arguments to worker constructor. Do you guys want to support this feature?

Background

rq.worker.Worker uses hostname and pid for its name by default (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L239) but we run workers in container and hostname/pid are all the same. As a result, workers failed to start due to the duplicated name (https://github.com/rq/rq/blob/eaf598d73ce234791ee22ef82f7f095aece9f2a6/rq/worker.py#L275).

Thanks!

Flask-RQ2 depends on setuptools but setuptools is not listed as a requirement

Steps to reproduce

Create a clean venv. python3 -m venv env.
Install Flask-RQ2 from source using pip install -e .
Trying to run/import Flask-RQ2 causes the following error:

Traceback (most recent call last):
  File "__init__.py", line 11, in <module>
    from pkg_resources import get_distribution, DistributionNotFound
ModuleNotFoundError: No module named 'pkg_resources'

I would fix this myself but I am absolutely not an expert with setuptools and I don't know where exactly to put this requirement in setup.py. Thanks!

rq version conflict in the dependency

When I install Flask-RQ2 from PYPI, I got rq version conflict.

$ pipenv install flask-rq2
Creating a virtualenv for this project…
Pipfile: /home/chi/Projects/just4test/Pipfile
Using /home/chi/.pyenv/versions/3.7.1/bin/python3.7 (3.7.1) to create virtualenv…
⠹ Creating virtual environment...Already using interpreter /home/chi/.pyenv/versions/3.7.1/bin/python3.7
Using base prefix '/home/chi/.pyenv/versions/3.7.1'
New python executable in /home/chi/Projects/just4test/.venv/bin/python3.7
Also creating executable in /home/chi/Projects/just4test/.venv/bin/python
Installing setuptools, pip, wheel...
done.

✔ Successfully created virtual environment! 
Virtualenv location: /home/chi/Projects/just4test/.venv
Creating a Pipfile for this project…
Installing flask-rq2…
Adding flask-rq2 to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✘ Locking Failed! 
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/resolver.py", line 69, in resolve
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches rq<0.13.0,>=0.12.0,>=0.13
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.13, 0.4.0, 0.4.0, 0.4.1, 0.4.1, 0.4.2, 0.4.2, 0.4.3, 0.4.3, 0.4.4, 0.4.4, 0.4.5, 0.4.5, 0.4.6, 0.4.6, 0.5.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.5.3, 0.5.3, 0.5.4, 0.5.4, 0.5.5, 0.5.5, 0.5.6, 0.5.6, 0.6.0, 0.6.0, 0.7.0, 0.7.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0, 0.8.1, 0.8.1, 0.8.2, 0.8.2, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 0.10.0, 0.10.0, 0.11.0, 0.11.0, 0.12.0, 0.12.0, 0.13.0, 0.13.0
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches rq<0.13.0,>=0.12.0,>=0.13
Tried: 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.13, 0.4.0, 0.4.0, 0.4.1, 0.4.1, 0.4.2, 0.4.2, 0.4.3, 0.4.3, 0.4.4, 0.4.4, 0.4.5, 0.4.5, 0.4.6, 0.4.6, 0.5.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.5.3, 0.5.3, 0.5.4, 0.5.4, 0.5.5, 0.5.5, 0.5.6, 0.5.6, 0.6.0, 0.6.0, 0.7.0, 0.7.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0, 0.8.1, 0.8.1, 0.8.2, 0.8.2, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 0.10.0, 0.10.0, 0.11.0, 0.11.0, 0.12.0, 0.12.0, 0.13.0, 0.13.0
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 726, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/home/chi/.pyenv/versions/3.7.1/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches rq<0.13.0,>=0.12.0,>=0.13
[pipenv.exceptions.ResolutionFailure]:       Tried: 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.13, 0.4.0, 0.4.0, 0.4.1, 0.4.1, 0.4.2, 0.4.2, 0.4.3, 0.4.3, 0.4.4, 0.4.4, 0.4.5, 0.4.5, 0.4.6, 0.4.6, 0.5.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.5.3, 0.5.3, 0.5.4, 0.5.4, 0.5.5, 0.5.5, 0.5.6, 0.5.6, 0.6.0, 0.6.0, 0.7.0, 0.7.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0, 0.8.1, 0.8.1, 0.8.2, 0.8.2, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 0.10.0, 0.10.0, 0.11.0, 0.11.0, 0.12.0, 0.12.0, 0.13.0, 0.13.0
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches rq<0.13.0,>=0.12.0,>=0.13
Tried: 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.13, 0.4.0, 0.4.0, 0.4.1, 0.4.1, 0.4.2, 0.4.2, 0.4.3, 0.4.3, 0.4.4, 0.4.4, 0.4.5, 0.4.5, 0.4.6, 0.4.6, 0.5.0, 0.5.0, 0.5.1, 0.5.1, 0.5.2, 0.5.2, 0.5.3, 0.5.3, 0.5.4, 0.5.4, 0.5.5, 0.5.5, 0.5.6, 0.5.6, 0.6.0, 0.6.0, 0.7.0, 0.7.0, 0.7.1, 0.7.1, 0.8.0, 0.8.0, 0.8.1, 0.8.1, 0.8.2, 0.8.2, 0.9.0, 0.9.0, 0.9.1, 0.9.1, 0.9.2, 0.9.2, 0.10.0, 0.10.0, 0.11.0, 0.11.0, 0.12.0, 0.12.0, 0.13.0, 0.13.0
There are incompatible versions in the resolved dependencies.

As you can see, pipenv can't find a version matches rq<0.13.0,>=0.12.0,>=0.13.

I checked the dependencies, found that:

Flask-RQ2 18.2 on PYPI need:

rq>=0.12.0,<0.13.0
rq-scheduler>=0.8.3

and the version of rq-scheduler in PYPI is 0.9.0, need:

rq>=0.13

so that's the problem.

I think, could we freeze rq-scheduler version in dependencies, or just remove the rq version limit? because rq already fixed the redis>=3.0.0 issue.

Can not set the job queue name dynamic when using @rq.job() decorator

Hi there,

I'm using Flask-RQ2 to deal with some jobs need executed one by one.

I want to set the queue name dynamic because we need split jobs in different queue for different consumer.

But I checked in the source code, not found a way to reach it.

Maybe we can add a parameter for job decorator which can execute a function and set the queue name by the return value.

Is there any other way to reach my requirement?

Not able to start worker

Flask is 0.12.2, I have customized launcher, basically normal FlaskGroup with couple additional commands

If I try to start worker using rq worker I'm getting following error:

$ python manage.py rq worker
Traceback (most recent call last):
  File "manage.py", line 35, in <module>
    cli()
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/flask/cli.py", line 380, in main
    return AppGroup.main(self, *args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/flask/cli.py", line 257, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/flask_rq2/cli.py", line 40, in new_func
    return func(rq, ctx, *args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/flask_rq2/cli.py", line 138, in worker
    queues=queues or rq.queues,
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/rq/cli/cli.py", line 175, in worker
    worker_class = import_attribute(worker_class)
  File "/home/jazg/v/zkw/local/lib/python2.7/site-packages/rq/utils.py", line 150, in import_attribute
    module = importlib.import_module(module_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named backend_zkw.app

Application zkw otherwise runs fine and I can enqueue jobs.

Error: RQ object has no attribute 'scheduler_cls'

I'm running RQ2 on my docker instance and when I schedule a job with cron, I get the following error:

AttributeError: 'RQ' object has no attribute 'scheduler_cls'

Any ideas?

ImportError: No module named backend_dist.manage when using Flask-Script

Hi Everyone,

Thanks for contributing this project. It is extremely useful.

I'm struggling to get it working with Flask-Script however. Running

python manage.py rq

lists all the commands correctly.

However, as soon as I attempt to start a worker with:

python manage.py rq worker

I am hit with:

Traceback (most recent call last):
  File "manage.py", line 23, in <module>
    manager.run()
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/flask_script/__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/flask_script/__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/flask_rq2/script.py", line 42, in __call__
    return self.run(*args, **kwargs)
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/flask_rq2/script.py", line 179, in run
    queues=queues or self.rq.queues,
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/rq/cli/cli.py", line 175, in worker
    worker_class = import_attribute(worker_class)
  File "/Users/name/Documents/Development/venv/lib/python2.7/site-packages/rq/utils.py", line 151, in import_attribute
    module = importlib.import_module(module_name)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named backend_dist.manage

Looking through flask_rq2.app.py I noticed a module_path is being set on line 189.

self.module_path = 'flask_rq2.backend_%s' % app.name

I do have an application in dist.manage but I suspect flask_rq2 may be looking for it in the wrong place. My directory structure is:

manage.py
dist
- manage
-- __init__.py
-- *.py

Any help much appreciated.

(I know this probably isn't a bug, so this may not be the best way of asking for help, but thanks anyway!)

How to run multiple workers?

Can you run multiple workers listening on the same queue?