Comments (6)
Hi @waqas-ali-pk
It has been a while that I have worked with this project. If there is still a cronjob configured then probably via Ansible. Let me know if that was helpful.
from klee-web.
Hi @andronat Thanks for sharing this!! I also have another scenario, sometimes celery worker stop without showing any error message, how that case be handled. I really appreciate your help on this.
Hm, well in general I was hopping to find time to upgrade to latest Celery as it seems to be more robust (e.g. heartbeats). But I never managed. PRs are always welcomed 😃. So definitely two things I can think of: 1) you could consider put the time to upgrade to latest Celery, 2) you could set a standard time point that you just blindly restart workers. Not all of them together, maybe with a rolling strategy.
How we can restart only that specific worker if we do not use docker?
Well that depends on how you run the celery project on your infrastructure. Docker in general is the easy way out and I highly recommend it.
from klee-web.
@Denis-Gavrielov That's great idea indeed, to run cronjob after every hour and check status of workers. Are you restarting that specific worker/container, or restarting everything?
Can you share cronjob details/code, how you have implemented it for your case. I want to implement same solution, if you share stuff, it will be of great helpful.
from klee-web.
Hey @waqas-ali-pk I think this is what you are looking for. Honestly there another cronjob that I've been using recently but recently I'm finding very difficult to find time and make a proper PR.
Nevertheless, I just paste what I'm using in a quick a dirty way here:
- name: "Restart {{ kleeweb_worker_container }} if celery heartbeat missed"
cron:
name: "Restart {{ kleeweb_worker_container }} if celery heartbeat missed"
minute: "30"
job: 'if [ $(sudo docker logs --tail=1 {{ kleeweb_worker_container }} | grep "missed heartbeat from celery" | wc -l) -eq 1 ]; then $(sudo docker restart {{ kleeweb_worker_container }}); fi'
user: klee
when: not ci
- name: "Kill all remaining klee containers every day"
cron:
name: "Kill all remaining klee containers every day"
minute: "0"
hour: "0"
job: "sudo docker ps --filter ancestor=klee/klee -q | xargs sudo docker kill"
user: klee
when: not ci
- name: "Restart {{ kleeweb_worker_container }} every day"
cron:
name: "Restart {{ kleeweb_worker_container }} every day"
minute: "0"
hour: "0"
job: "sudo docker restart {{ kleeweb_worker_container }}"
user: klee
when: not ci
from klee-web.
Hi @andronat Thanks for sharing this!! I also have another scenario, sometimes celery worker stop without showing any error message, how that case be handled. I really appreciate your help on this.
How we can restart only that specific worker if we do not use docker?
from klee-web.
@andronat Thanks!! This is helpful.
from klee-web.
Related Issues (20)
- Display bug in the /manage interface
- klee-output S3 bucket HOT 6
- Move circleci to 2.0
- Freeze all dependences HOT 1
- angstwad.docker_ubuntu does not install docker-py
- Rework git_repo group_var
- Pass `receivers_email` as an ENV variable
- Add C++ support HOT 1
- GeoLite2 databases are no longer served publicly
- KLEE Web Error Running clang on vanilla example HOT 2
- Public web-hosted instance (http://klee.doc.ic.ac.uk) returns "502 Bad Gateway" HOT 1
- Can't pass multiple options? HOT 2
- Enable downloading raw KLEE results in development mode
- Update Node.js into latest version
- Use REST practices for /job/notify HOT 3
- Fix end-to-end Klee Web tests on CircleCI
- Fix the Klee Web coverage report. HOT 1
- Fix /manage
- Hidden Examples and Tutorials when logged in
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from klee-web.