Job marked in error state is actually running about sisyphus HOT 2 OPEN

I think seeing an error.run.1 file on disk takes precedence over seeing the task running in slurm (or sge for that matter).

Now how did we get here? Was this maybe another case of job submitted to queue while another instance of the same job was still running? I imagine if the first instance errored while the second one was still pending then we would arrive at this situation.

Or maybe was the second instance submitted after the first instance crashed but before the manager found the error file?

from sisyphus.

I can confirm that the manager reports ans error as soon as the error file is found and also agree that the most likely scenario is that two jobs were submitted and one crashed.

That the manager restarted the job before it found the error file is unlikely. I would not rule it out, but there are steps to prevent this. Otherwise this would probably frequently happen to finished jobs as well.

from sisyphus.