richardhenry / hotqueue Goto Github PK

HotQueue is a Python library that allows you to use Redis as a message queue within your Python programs.

Home Page: https://richardhenry.github.com/hotqueue/

License: MIT License

Python 100.00%

hotqueue's Introduction

HotQueue

IMPORTANT NOTE: This project has not been actively maintained in a long time! You probably want to consider something like the Disque Redis module instead: https://github.com/antirez/disque

HotQueue is a Python library that allows you to use Redis as a message queue within your Python programs.

The main advantage of this model is that there is no queue server to run, other than Redis. This is particularly ideal if you're already using Redis as a datastore elsewhere. To install it, run:

pip install -U hotqueue

The best place to get started is the documentation.

The source code is available on GitHub.

To get help with HotQueue, use the HotQueue Users mailing list.

Contributing

The source is available on GitHub. To contribute to the project, fork it on GitHub and send a pull request, all contributions and suggestions are welcome.

hotqueue's People

Contributors

Stargazers

Watchers

hotqueue's Issues

Possibly unmaintained; consider yarqueue

This project may no longer be maintained. For anyone coming to this page in the future, consider the package yarqueue (Yet Another Redis queue). It's inspired by hotqueue, but fixes some issues and adds some extra convenience:

Fixed compatibility with multiprocessing.Queue (exceptions, init arguments etc.)
First-in first-out, last-in first-out, and double-ended queues
Joinable variants of each of the above
Uses a new redislite instance if no redis client was given
Additional convenience methods
Tested against 3.6+

Use the pickle library if the cPickle library is not available

Misc fixes to make it possible to connect to redis via unix socket

At the moment you can connect HotQueue to redis server via unix socket by passing unix_socket_path to it constructor, however
it's generating exception if you are trying to get repr of the HotQueue object.

Can you fix that?

Could you please tag 0.2.8 version

It is quite misleading that it is not present...

strange "max number of clients reached"

In /etc/redis.conf the maxclients is 100000.
"netstat -na | grep 6379 | wc -l" shows there are 1131 connections.
The client script reports exception: max number of clients reached

How to solve this problem?

Error when installing: TypeError: chown() missing 1 required positional argument: 'numeric_owner'

Hi guys,

When I install this lib, i got the following error on windows 7:

$ pip install -U hotqueue
Collecting hotqueue
  Using cached hotqueue-0.2.7.tar.gz
    Complete output from command python setup.py egg_info:
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
    Extracting in C:\Users\user1\AppData\Local\Temp\tmpts_g6vwx
    Traceback (most recent call last):
      File "c:\users\user1\appdata\local\temp\pip-build-7dkaissd\hotqueue\distribute_setup.py", line 143, in use_setuptools
        raise ImportError
    ImportError

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\user1\AppData\Local\Temp\pip-build-7dkaissd\hotqueue\setup.py", line 7, in <module>
        from distribute_setup import use_setuptools; use_setuptools()
      File "c:\users\user1\appdata\local\temp\pip-build-7dkaissd\hotqueue\distribute_setup.py", line 145, in use_setuptools
        return _do_download(version, download_base, to_dir, download_delay)
      File "c:\users\user1\appdata\local\temp\pip-build-7dkaissd\hotqueue\distribute_setup.py", line 125, in _do_download
        _build_egg(egg, tarball, to_dir)
      File "c:\users\user1\appdata\local\temp\pip-build-7dkaissd\hotqueue\distribute_setup.py", line 99, in _build_egg
        _extractall(tar)
      File "c:\users\user1\appdata\local\temp\pip-build-7dkaissd\hotqueue\distribute_setup.py", line 467, in _extractall
        self.chown(tarinfo, dirpath)
    TypeError: chown() missing 1 required positional argument: 'numeric_owner'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user1\AppData\Local\Temp\pip-build-7dkaissd\hotqueue\

Anybody can help me?

Thanks,
Kris

LIFO queue

Is redis based LIFO queue safe to use?

It seems all I have to change to adapt hotqueue is

msg = self.__redis.lpop(self.key)
to
msg = self.__redis.rpop(self.key)

and

msg = self.__redis.blpop(self.key, timeout=timeout)
to
msg = self.__redis.brpop(self.key, timeout=timeout)

Redis client raises SocketError if conn timeout reached

Redis client will inevitable raise SocketError when connection timeout has been reached.

That situation breaks the consume() generator.

It would be cool for hotqueue to catch such exception and re-connect to Redis again.

If time permits, I can provide a patch.

No documentation for redis authentication

There is no support for connecting to redis that requires authentication
(at least not documented)

from hotqueue import HotQueue
queue = HotQueue("myqueue", host="localhost", port=6379, db=0)
queue.put(1)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/hotqueue.py", line 120, in put
self.__redis.rpush(self.key, _msgs)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 1265, in rpush
return self.execute_command('RPUSH', name, *values)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 565, in execute_command
return self.parse_response(connection, command_name, *_options)
File "/usr/local/lib/python2.7/dist-packages/redis/client.py", line 577, in parse_response
response = connection.read_response()
File "/usr/local/lib/python2.7/dist-packages/redis/connection.py", line 574, in read_response
raise response
redis.exceptions.ResponseError: NOAUTH Authentication required.

老伙计，你这个观察者咋没了，不维护了？

老伙计，你这个观察者 hotwatch 咋没了，不维护了？

Library fails to install

Currently, hotqueue appears to fail to install:

$ env/bin/pip install hotqueue
Collecting hotqueue
  Using cached hotqueue-0.2.7.tar.gz
    Complete output from command python setup.py egg_info:
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-RJ3enJ/hotqueue/setup.py", line 7, in <module>
        from distribute_setup import use_setuptools; use_setuptools()
      File "/tmp/pip-build-RJ3enJ/hotqueue/distribute_setup.py", line 145, in use_setuptools
        return _do_download(version, download_base, to_dir, download_delay)
      File "/tmp/pip-build-RJ3enJ/hotqueue/distribute_setup.py", line 124, in _do_download
        to_dir, download_delay)
      File "/tmp/pip-build-RJ3enJ/hotqueue/distribute_setup.py", line 193, in download_setuptools
        src = urlopen(url)
      File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
        return opener.open(url, data, timeout)
      File "/usr/lib/python2.7/urllib2.py", line 435, in open
        response = meth(req, response)
      File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
        'http', request, response, code, msg, hdrs)
      File "/usr/lib/python2.7/urllib2.py", line 473, in error
        return self._call_chain(*args)
      File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
        result = func(*args)
      File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
        raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
    urllib2.HTTPError: HTTP Error 403: SSL is required

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-RJ3enJ/hotqueue/

I think this is due to a recent change that PyPI made that disallows HTTP(no S) access. The fix here appears to be trivial: I think this line just needs to be https:// (and a new version would need to be released to PyPI).

Edit: actually, the current code tries to import setuptools first, and only if that fails to import does it attempt to access PyPI over HTTP. So, strictly speaking, for my use case (installing w/ pip), since setuptools is always present, all that is needed is just to release master as it is now as a new version. I think it would still be good to change that URL, on the off chance someone runs that code path. (As currently, if that code path runs, it would appear to fail.)

~~I'll send a PR for changing the URL.~~ Actually, it appears that @ryanstutes has already sent a PR, #20, that would fix this.

richardhenry / hotqueue Goto Github PK

hotqueue's Introduction

HotQueue

Contributing

hotqueue's People

Contributors

Stargazers

Watchers

Forkers

hotqueue's Issues

Recommend Projects

Recommend Topics

Recommend Org