just-work / django-celery-rpc Goto Github PK

pip install django-celery-rpc[server]
pip install django-celery-rpc[client]

Server has extra dependency - Django itself. Setup.py should be configured to allow installation for both variants

Что то вроде такого

#34 (comment)

1. Tox-based CI is now broken, because tox installs latest setuptools version, and it dropped support for use2to3 argument.
2. use2to3 is used in anyjson package, last release was in 10 years ago
3. anyjson is transient dependency for celery through kombu
4. Celery-4.4 does not have this problems

So, to fix CI, we must either fix setuptools version for tox, or drop Celery-3.1 support.

... but must not.

Methods update, getset, delete must raise exception when single mode is used (working with only one instance which are sent as dict).

Sync expiration time and client timeout to cancel missed tasks

Docs:

http://docs.celeryproject.org/en/latest/userguide/calling.html?highlight=expire#expiration

http://docs.celeryproject.org/en/latest/reference/celery.app.task.html?highlight=expire#celery.app.task.Task.expires

https://github.com/just-work/django-celery-rpc/runs/738207356?check_suite_focus=true

Run python setup.py sdist bdist_wheel
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'bdist_wheel'
##[error]Process completed with exit code 1.

Support for add/remove m2m relations

What do you think about making perform_changes atomic?
It's possible to add flag for enbale/disable atomicity in server settings or in task kwargs or both.

Try to add queue .high_priority and check the order of task consuming by worker.

RequestError: ('Something goes wrong while sending request', EncodeError(RuntimeError("Django Q-objects does not supported by 'x-json' codec. For running with Q-objects use celery_rpc>0.16 on both sides and set 'x-rpc-json' as task serializer for client",),))

it seems impossible to do default value with empty Q object in filter sinature.

• add info messages on client and server side

in current release version set to "version='0.1'".

2014-10-20_09:31:53.74706 ContentDisallowed: Refusing to deserialize untrusted content of type x-rpc-json (application/x-celery-rpc-json)

У меня тут баг, ты сменил контент тайп но не добавил его в registry (см. http://stackoverflow.com/questions/6628016/how-to-disallow-pickle-serialization-in-celery)

reported by @voron3x

In create_celery_app it seems to be overriding CELERY_QUEUES with:

    app.conf.update(
        CELERY_HIGH_PRIORITY_QUEUE=high_q,
        CELERY_HIGH_PRIORITY_ROUTING_KEY=high_rk,
        CELERY_QUEUES=(Queue(q, routing_key=rk),
                       Queue(high_q, routing_key=high_rk)))

Wouldn't it be better if it could be overridden in CELERY_RPC_CONFIG? If so please let me know so I can submit a pull request

• Add to build matrix
• Ensure everything works with recent python versions

from here:

MyModel._meta.get_all_related_objects() becomes:

[
    f for f in MyModel._meta.get_fields()
    if (f.one_to_many or f.one_to_one)
    and f.auto_created and not f.concrete
]

I.e., we have City-Country-Continent db schema. I need to obtain continents for some cities. To do this, now I have these options:

• query City with depth=2
• write custom serializer class on server side

depth is not a bad option, but if you need more than one join, all these relations may become a problem.

So, the proposal is to add "select_related" option to filter() arguments.

• it adds qs.select_related(*fields) to serializers queryset
• it automatically created fields, named like select_related values

For example:

>>> client.filter('City', kwargs={
  'filters': {},
  'select_related': ['country__continent']
})

[{
  'id': 1,
  'name': 'Moscow',
  'country__continent': 'Eurasia'
}]

Problems found:

• Django select_related() adds related objects as obj.country.continent
• Thus, you need to write complex serializer method to obtain continent serialized data
• if you setattr this method to GenericSerializerClass object, it doesn't become 'bound method', so some __new__ magic is needed
• also, ModelSerializer need to be constructed for Continent model, and model class must be resolved before instantiation of GenericSerializerClass

Lot's of stuff to think about.

test and adapt with DRF3

There are needs to run several task as one batch (may be as atomic action).

Requested by @bourivouh

А как можно решить ситуацию, когда на сервер возможны два вида ошибок first.module.Error и second.module.Error

Need something similar to:

p = cl.pipe()
p.filter('model', kwargs={'filters': {'active': True}})
p.method('deacivate', qs=True, args, kwargs)

if qs set then call queryset method, else on each model.

And:

p = cl.pipe()
p.create('model', data)
p.method('start_play', args, kwargs)

• Add support for Django Q-object
  • Almost done by @voron3x (see #26)
• Add exclude