Cache invalidation problems with asynchronous tasks about django-cachalot HOT 8 CLOSED

Dat title
On 12 Feb 2016 15:53, "rahulmuraliqllabscom" [email protected]
wrote:

—
Reply to this email directly or view it on GitHub
#46.

from django-cachalot.

Strange, there should be no problem, I specifically designed django-cachalot with asynchronous tasks in mind. I use cachalot along with RQ on several projects without a single issue.

Are you sure you’re using the same cache storage on the server and celery? If so, did you read the limits of Locmem?

By the way, I suggest you try another solution than celery. I lost a full-time week in 2014 trying to figure out why it was working only partially on my projects (often lost tasks for no reason using only Redis). I still don’t get it, but switching to RQ was an instant-working revelation. Besides this unreliability, I find celery heavy, slow, and poorly coded (try to write tests for an asynchronous celery task without CELERY_ALWAYS_EAGER and welcome to a world of pain).

from django-cachalot.

Sir,
Firstly sorry for the late reply I was out of town for the weekend. I am thankful that you noticed my issue, I am a newbie to the techworld this is my first full scale project. Ours is a small firm and I am the only one working on Django. I don't have anyone around for help, when it came to the requirement that I needed some task to be done asynchronously first solution I found was to use celery, so I went ahead with it and things were working fine for me till it came down to this issue. I read High performance Django by the team from Lincolon Loop In that they suggest using jhonycache for queryset cache, but because of my blunder and little knowledge I had already seleced Django1.8 and python3 both of which are not compatible with johnycache, there they have mentioned that johnycache also has the same issue and could be overcomed by adding

import johnny.cache
johnny.cache.enable()

into the projects init.py file

Other than this issue Cachalot was working fine I reviewed database logs and only new quires were directed to database others were served from cache. I also had some issues in saving of a model a filed was simply going null eventhough I saved values to it. In the above comment you have asked

Are you sure you’re using the same cache storage on the server and celery?

I have not set any separate cache for celery. I use memcached for cache storage and have set it as default cache in django settings file. I am using the package django-celery.
When I did load testing with a heavy dataset I found that I badly need queryset caching for some queries

Please help me sir I am stuck on this
Also can you suggest me some good advanced django tutorials.

from django-cachalot.

We have also been seeing some invalidation issues recently, which @devoto13 suggested may have something to do with PYTHONHASHSEED.

We had PYTHONHASHSEED set to randomas recommended by Django's Deployment Checklist. This could lead to issues since Cachalot now uses the built-in hash function instead of SHA-1. Django and Celery would get different hash seeds, so their cache keys may no longer match.

I'm currently testing out static PYTHONHASHSEED values in our development environment to see if this solves the issue.

from django-cachalot.

thank you for the concern you are showing

from django-cachalot.

@JockeTF You’re right. I didn’t face the issue as I always have this environment variable set to 1. I made a mistake, I’ll fix it in a bugfix release I’ll issue in the following days. Many thanks for identifying this cause!

from django-cachalot.

@rahulmuraliqllabscom: Are you using Python 3.3, 3.4 or 3.5? If so, please update to the new django-cachalot 1.2.1 and this issue can be closed.

from django-cachalot.

Thank you all for the help, now things are working fine also I was able to come across the python3 random hashseed property.

from django-cachalot.