RecoveryPolicy is misnamed - it is not related to Recoverable [BATCH-27] about spring-batch HOT 4 CLOSED

Lucas Ward commented

There are three separate concerns that are being controlled by these policies:

• RestartPolicy: should the same job instance always be used for given RuntimeInformation? - Job Level
• StartPolicy: should a step be allowed to start? - Step Level. This policy is absolutely meaningless if restart = false.
• RecoveryPolicy: should restart data be persisted for a given step? This is equally as meaningless if restart = false.

Since both start and recovery policy rely on RestartPolicy being true, there could be just one RestartPolicy. However, things get a bit hairy when you want to control the start counts for different steps independently of the job as a whole. For example, if the job has a restart limit of 10 (a start limit on a non-restartable job is pointless) and has two steps, the first has an infinite start limit (functionally whatever the restart limit on the job is) and the second step has a start limit of one. This means that for some reason, step number two can only be run once, any failures will require some stop of manual intervention before trying to run again, or a new job to be created, so that all data is 'blown out'. Using only one limit in a RestartPolicy means that the whole job would have to have a start limit of 1, which, if step1 failed, wouldn't be correct.

One solution would be to separate the 'start limit' from the restart policy. This would mean that the JobConfiguration and StepConfiguration objects would all have integer startLimit's similar to StepConfiguration's commitInterval. The only real downside to this is that if a job is restartable, all start limit's would be effectively useless. (although I'm not sure I mind this)

Another Solution would be to continue and have policies and name them differently, although, I'm not sure what the best name would be.

A final wrinkle to this issue is, if limits are moved out of the policies and directly into configuration, is there really a need for policies? Is there any reason why RestartPolicy would ever contain more than a boolean true/false, which is acted upon by the execution environment? The same could be said for RecoveryPolicy, which is essentially :shouldRestartDataBePersisted=true/false. Again, acted upon by the execution environment.

from spring-batch.

Dave Syer commented

I'm in favour of keeping it simple at this stage. I.e. remove the policies and replace with booleans in the Job/StepConfiguration. That would solve the problem of naming the policies at least. And the shouldRestart flag could have the same name for Steps and Jobs (actually I'm not sure the policy couldn't but that's irrelevant if we ditch it).

from spring-batch.

Lucas Ward commented

Modified Step and JobConfiguration to no longer use policies, but instead have all values wired directly in:

JobConfiguration->setRestartable(boolean)
JobConfiguration->setStartLimit(int)

StepConfiguration->setStartLimit(int)
StepConfiguration->setSaveRestartData(boolean)
StepConfiguration->setAllowStartIfComplete(boolean)

The logic for determining if a step should start is now contained within the JobExecutor, which I think I'm okay with. In the unlikely event that a project team really needs to have some other type of logic to determine if a step should be started, they can create a custom JobExecutor. This meshes well with separating the configuration domain from the execution domain, since now the individual developer will no longer need to change their wiring.

from spring-batch.

Dave Syer commented

Assume closed as resolved and released

from spring-batch.