galeria-kaufhof / ha-jobs Goto Github PK

It appears to me that the lock touching actor KeepJobLockedActor is touching the lock immediately after creation leading to a lock touch immediately after lock acquiring.

See https://github.com/Galeria-Kaufhof/ha-jobs/blob/master/ha-jobs-core/src/main/scala/de/kaufhof/hajobs/KeepJobLockedActor.scala#L23
and https://github.com/Galeria-Kaufhof/ha-jobs/blob/master/ha-jobs-core/src/main/scala/de/kaufhof/hajobs/JobExecutor.scala#L88

Is that intentional or am I not getting something?

Hi,

In EVAL we had a UI that could manage the jobs (start / stop them).. I'm not sure we used this job lib or worked directly with the streams, but if the code still exists, maybe it can be added here.

just stumbled upon this repo when looking for something similar at my new gig.

Under the "Constraint" section, I read this: There's no need for work distribution / if a single instance can execute a job.

Please elaborate what this constraint means. :)
A (future) use-case we are looking for for a "job framework" includes work distribution to numerous "workers" or processors both via a push or a pull mechanism.

Push mechanism is something like via Spark delegation.
Pull mechanism is something like perhaps multiple client Applications taking on(locking) Jobs from the Job repo.

Debugging jobs only via the rest api will suck pretty soon => Let's build a simple ui

Multiple service Instances logged
actor name [_LOCK] is not unique!
One of the Instances at least got in trouble because of C*-Timeouts.
After restarting that instance the errors from the other instances disappeared, too.
akka.actor.InvalidActorNameException: actor name [AttributeModifications_LOCK] is not unique! at akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:130) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at akka.actor.dungeon.Children$class.reserveChild(Children.scala:76) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at akka.actor.ActorCell.reserveChild(ActorCell.scala:369) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at akka.actor.dungeon.Children$class.makeChild(Children.scala:201) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at akka.actor.dungeon.Children$class.actorOf(Children.scala:37) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at akka.actor.ActorCell.actorOf(ActorCell.scala:369) ~[com.typesafe.akka.akka-actor_2.11-2.3.11.jar:na] at de.kaufhof.hajobs.JobExecutor$$anonfun$de$kaufhof$hajobs$JobExecutor$$run$1.apply(JobExecutor.scala:88) ~[de.kaufhof.ha-jobs_2.11-1.0.2.jar:1.0.2] at de.kaufhof.hajobs.JobExecutor$$anonfun$de$kaufhof$hajobs$JobExecutor$$run$1.apply(JobExecutor.scala:80) ~[de.kaufhof.ha-jobs_2.11-1.0.2.jar:1.0.2]

glad to find this project, it think it will be better if cross building for scala 2.12

configured: 3 retrigger attempts after a failed job
in db an endless array of attempts is shown (once per minute)

Hi,

Any guideline or formula how to determine an optimal Cassandra Keyspace Class & ReplicationFactor ?
For example, we have a 6 node Cassandra dev cluster. Production cluster may have more nodes.

Thanks

This is important when the JobSupervisor/JobUpdater checks if there are jobs running without a matching lock. If the LockRepository.getAll does not return a lock because it could not resolved (was not supplied by LockTypes), a running job gets canceled and nothing in the logs tells you why this happened.

Sending a closure over self to a child to be notified upon 'touch-failure' is nice

val lockKeeper = context.actorOf(
        KeepJobLockedActor.props(lockRepo, job.jobType, jobId, job.lockTimeout, () => self ! LostLock(jobContext)),
        job.jobType.name + "_LOCK")

(ha-jobs-core/src/main/scala/de/kaufhof/hajobs/JobExecutor.scala)

I would not have thought that it is ok to do that, but now it seems pretty clear. Thanks!

It would be great if there were dependencies between jobs, so that a Job 'B' can be started after Job 'A' was finished.