Allowing profiling outside of the request/response cycle about django-silk HOT 10 OPEN

Silk is an awesome tool, it would be great to use it for profiling celery or other async tasks that happen outside the request/response cycle.

from django-silk.

We have a use case where this feature would be extremely useful. Our application has a lot of long-running asynchronous "tasks" using RQ. Currently, django-silk can't help us because none of the hard work is being done within the scope of a request. Each individual task is very intensive but there are not that many of them, relatively speaking, so it's not a "high volume" problem.

A lot of applications use Celery, also. Same principle. I would love to be able to use Silk to debug our task performance.

from django-silk.

Do not want to necro something, is there a way to achieve this somehow?

from django-silk.

FYI I found this GitHub gist that you can use to run django-silk against arbitrary functions not wrapped in APIs. This was a good-enough workaround for me to debug some performance challenges, although it would be fantastic to see something supported by django-silk in the future.

from django-silk.

How to that? what about profiling django-channels request with silk? @avelis @smaccona

from django-silk.

@auvipy Not sure if django-channels is within the purview of this library. Not against it though. The issue becomes volume. Since channels is meant for websockets it will definitely be high-volume.

from django-silk.

OK. we can re think about django channels at a later stage.

from django-silk.

+1

from django-silk.

any prospective API wor workflow do you have in mind how should be the DevUX for the API of that profiler to profile celery/ etc with or outside of django req/resp cycle?

from django-silk.

Hi,
Why not give an option in the plugin so that Celery/Rabbit MQ can be used to log the data in Database asynchronously. We face timeouts for requests which have large amounts of data.
Is this a suggested strategy? if yes then my team can work on the feature and push it

from django-silk.