flipkart-incubator / hydra_job Goto Github PK

There are no metrics generated or no logs produced that helps measure the overall performance (through put/ response times/ wait times) of the workflow or find the bottleneck jobs or identify the hot spots.

Measure system parameters such as CPU and memory per Job (and per Node level)

Right now the job dependency is very simple DAG where each job depends exactly on one job.
Need capability to represent arbitrary job dependency DAG.

Need more visibility into how jobs are getting processed. For example: which job is executing right now on which thread, which path the job took (and why), how many times a job was retried (and why).

Status of the workflow (hot spots, failure rate from a node)

Through put of a thread, status of a thread (sleeping, waiting for a job, stack trace).

AnyNode was a workaround to avoid generics and maintain run time type safety when creating workflow dynamically from configurations. It is proving to be a maintenance night mare (for example: Refactoring node abstractions require Any* version for each of the new actors)

• Require control dynamically add/remove jobs, skip/re-route
• Require micro level and dynamic configuration such as to tune number of threads, enable/disable debug logging,
• Can interrupt a thread
• Pause the workflow