Comments (4)
You're pretty much close to the solution. I can think of solving these two use cases in two ways
-
wherever I've dependent task I can use post processing handler that will enqueue the next task. Using this we can model A -> B
-
wherever I've multiple tasks to complete i can use middleware along with thread pool, but the challenge here is retry on failure. Let's take an example, A has three sub tasks A1, A2 and A3. While running these tasks in parallel, any one of them could fail, on which task failure I should retry ?
There's a way to attach multiple listeners on the same message like in this case we can add three handlers and I'll designated one of them as primary. But now we've to handle race condition for example primary task could complete before other two tasks that's undesirable behaviour. So modelling based upon subtask is tricky at this moment.
from rqueue.
What is your case ? Airflow dag has such features. Currently RQueue does not support such features. But sequentially model can be supported
from rqueue.
Similar to the second scenario I mentioned, my asynchronous task, Job B, relies on the successful completion of all parallel tasks of Job A (A1, A2, A3) before it can execute. I am curious to know if it's possible to implement my idea using middleware in rqueue.
from rqueue.
I can break down the Job A tasks into A1, A2, A3 at the business logic level, but I'm unsure whether this type of workflow with task dependencies (Job A -> Job B) can be implemented within rqueue.
from rqueue.
Related Issues (20)
- Serialization support java time module HOT 1
- Deleted messages are kept in memory HOT 5
- Add static resource mapping handler when rqueue.web.url.prefix is set HOT 5
- Weighted priorisation with maximum concurrency HOT 2
- Question: Re-starting processing task when system restarts HOT 7
- Do you plan to use spring-cloud-sleuth to propagate the traceId? HOT 5
- Dashboard shows empty even thought Rqueue is working fine HOT 13
- Question: Integration scenario with multiple queues HOT 4
- Support of jackson-databind 2.16.0 HOT 3
- Login/pass for queue dashboard HOT 4
- Lack of available threads HOT 2
- Service-Level Concurrency Control HOT 1
- IllegalArgumentException when accessing queue details HOT 1
- springboot 3.2 can not work HOT 5
- springboot 2.7 can not work HOT 9
- Typo in dashboard text HOT 2
- RQueue attempting to bootstrap despite being "disabled" HOT 1
- Best practice of initializing a queue (producer only) HOT 1
- RqueueListener sometimes do not consume message. (Intermittently) HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rqueue.