Comments (11)
Thanks. Makes sense.
I think the the handler.emit
could queue up any messages send before the task is started, making it optional to check whether the status is running before sending something.
from workerpool.
I've created a PR with a relatively simple change to let the parent send messages to the worker instances parallel to the current execution.
from workerpool.
I guess it was the second part of this PR that was never implemented? :( #210
from workerpool.
Thanks for your suggestion Michael. Yes there was a start with something like this in #210, but we still have to think through a good API for this.
Help would be welcome!
from workerpool.
Hm. Just spitballing here... how about a new callback, similar to on() events. Something like
let executionId: number | undefined;
pool.exec('eventExample', [], {
onExecutionStart: (id) => {
executionId = id;
},
on: function (payload) {
//...
},
});
...
if (executionId) {
pool.getWorkerInstance(executionId).emit('Take this!', {data: {...}});
}
from workerpool.
Yes that could work, though it involves an indirect solution via an identifier.
How about extending the returned Promise-like instance:
const handler = pool.exec('fibonacci', [10], {
on: function (payload) { ... }
}
// the returned promise-like `handler` actually is an extended version of Promise,
// having methods .then(callback), .catch(callback), .cancel(), .timeout(delay), and .emit(payload)
promise.emit('Take this!')
const result = await handler
To prevent confusion, we should maybe rename Promise
to ExecutionHandler
or something like that.
from workerpool.
That'd be much nicer of course!
Also, it'd be nice if the messages would be queued up as well, in case the worker has not started yet (and alternatively, the promise should also return the current state of the task (queued, running, finished,...) so messages don't get lost)
Adding this would make the pool so much more useful for controlling long running tasks!
from workerpool.
Yes, it will be nice to be able to have two way messaging between the main process and the worker.
I'm not sure if it is possible to queue messages upfront: upfront you do not know at which worker a task will be assigned, only as soon as the task started you can send something to the worker where the task is running I think?
Maybe you can share your use-case to better understand what a good API would be to cater for it?
from workerpool.
It'd be enough if the promise returned the state so that only in case it is running, a message could be sent.
if (handler.status() === 'running') handler.emit('Take this!');
The use case here is to run a long running task, which in itself is a workflow of multiple steps. These could take a while, so on the one hand we need to be able to timeout and cancel the task (which ideally requires some extra steps for a clean shutdown, other than just killing the task), and on the other hand we'd like to trigger actions on the running task as well (like adding a new step while it is running).
It's all rather convoluted, tbh., but it requires a channel back into the worker, in any case :)
from workerpool.
Sounds like a plan! Let me know if I can help out somehow.
from workerpool.
I'm not planning on implementing this myself, so if it is important to you or someone else have to jump in.
from workerpool.
Related Issues (20)
- Fix the code coverage script HOT 1
- App bundle fails to start worker, with message : "Worker is terminated" HOT 1
- Type of error is not kept when error goes from inside to outside the workerpool HOT 1
- Uncaught SyntaxError: Unexpected token '<' (at worker.ts:1:1) HOT 1
- Concurrent async tasks on a single worker HOT 1
- MaxListenersExceededWarning with worker_threads in node.js HOT 5
- Unsafe-eval violation HOT 2
- Main node process crashes with OOM error HOT 3
- How to use with typescript HOT 2
- Error: TimeoutError: Promise timed out after 600000 milliseconds HOT 4
- Update type definitions with the new 'Transfer' options HOT 3
- Send message to persistent worker's stdin HOT 7
- Cannot use import statement outside a module (in Vite) HOT 15
- Possible bugs in the code. HOT 3
- child_process.spawn() or child_process.exec() HOT 1
- TypeError: process.send is not a function HOT 4
- (question) How to override the .exec function ? HOT 2
- How to use TypeORM or Prisma inside worker ? HOT 7
- how to use FileReaderSync in workerpool HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from workerpool.