About the celery task queue in this project about choose-your-own-adventure-presentations HOT 10 CLOSED

Hey @ripitrust, the Celery task queue is just a simple example - if you want to save the in-memory Redis data to the persistent storage PostgreSQL database then this would show you how. Unfortunately, it's a bit of a contrived example, not a complicated use case for Celery. I added it because I wanted to make sure deployments would include a task queue, which is pretty standard in most deployments. I found the first few times I stumbled through adding a task queues to my deployments it was a real pain so having it would be valuable for readers.

Daemonization comes in via Supervisor. Supervisor runs both Celerybeat and Celery task queue. The bash scripts are just for development purposes when you're not using Supervisor.

from choose-your-own-adventure-presentations.

@makaimc
Thank you so much for your reply
So in this project, the celery task (persist_votes) is run by the celery beat, right ?

from choose-your-own-adventure-presentations.

No problem!

Celery is a little confusing because when you run periodic tasks you need two processes. One is Celerybeat, which is the scheduler. The other part is Celery, which actually executes the task. It was kind of the hard conceptual idea for me to get over first. I kept thinking that the Celery process did both things, schedule the next execution and actually do the execution, but they are separate parts. That's why both are necessary in this case with a periodic task.

In a really simple case, where you just fire off tasks in the background, you could just run Celery without Celerybeat. Celery would execute the tasks as soon as possible without adhering to a pre-defined schedule like once an hour.

from choose-your-own-adventure-presentations.

@makaimc
Great, I start to get the idea.
So let's say I have my flask app dispatching background task to a new thread everytime it comes in, is it similar to send tasks to celery queue in the simple case?

from choose-your-own-adventure-presentations.

yep, in your case you'd have the simpler case. just fire off the tasks to Celery and you'll let it handle them as soon as it can, no need to have a scheduler like Celerybeat running. makes set up a bit easier

from choose-your-own-adventure-presentations.

@makaimc and if the task is periodic, then I can't run it in thread.

from choose-your-own-adventure-presentations.

if you want it to run at a designated interval, like once an hour, then Celerybeat will add a new task to Celery each hour to run

from choose-your-own-adventure-presentations.

@makaimc
Great, thank you so so much for your explanation
Btw I have to say your full stack deployment book helped me hugely. really great work

from choose-your-own-adventure-presentations.

awesome! thank you! I'm really excited to get this new update out which will smooth out some of the confusion readers have had with particular sentences and sections. hopefully will ship it as soon as I get back to the States (I'm in Italy on vacation at the moment).

good luck!

from choose-your-own-adventure-presentations.

@makaimc

nice to hear that, already feel excited about the new update

enjoy your vacation

from choose-your-own-adventure-presentations.