allefeld / htcondor-matlab Goto Github PK

make the latter secondary in output

record ClusterId for each job separately

in preparation for #9

runs into problems because uniform environment is no longer guaranteed:
Matlab version → location of binary, Matlab path...

clean up cluster directory
either all beyond 30 days age (asking), or named cluster
prevent clusters with running jobs from being cleaned

→ how to check whether job is still running?
solution depends on #13

before condor_submit

No way to flush stdout and stderr within Matlab.

Happens even if Condor task is terminated, log shows that, but error messages are not yet written to task###_err. But isn't stderr already closed and therefore flushed when Matlab terminates?

Maybe it would help to not use Condor's out= and err=, but to redirect stdout and stderr within Matlab? Then it might be possible to fclose and thereby flush them?

Can we be sure that results are written when the task is terminated?

condorCreateCluster takes a cluster description as optional second argument. if not specified, it is generated from name of calling function à la initDiag. it is stored in a plain text file in the job directory, and shown in condorMonitorCluster and condorListClusters (see #6)
date and time of job creation are appended to description

which executes jobs sequentially on the local machine

which means make a GUI

so that it checks which jobs failed and restarts only those, in condorSubmitJob

depends on breaking 1:1 correspondence between clusterHandle & ClusterId, see #12

prevent re-running job that already have results, and those that are still running
depends on #13, condor_job_status:
in queue (JobStatus ~= 3, 4) -> cannot be restarted
ExitCode == 0 -> should not be restarted
latter will probably be a subset of JobStatus = 4

giving summary information about an htcondor-matlab cluster: number of jobs, of those still running, finished, and exited with results / with error

as a basis for #9 and #6, and also of use for condorMonitorJob

probably only decently doable via GUI

use copyfile, supports copying into existing folder structure and creating one.

problem: complains about not being able to preserve permissions
use workaround from http://de.mathworks.com/matlabcentral/answers/20166-copyfile-permissions

print a list of all clusters present in the cluster directory, including number of jobs, of those still running, finished, and exited with error, and possibly cluster description

depends on #13

in input script / condorAddJob

wrapper around condor_rm

condorRemove(clusterHandle, jobNumber) applies to the specified job(s) (→ condorInspect),
condorRemove(clusterHandle) applies to held jobs,
condorRemove(clusterHandle, '-all') to all running jobs

problem: "remove" already has another meaning in condorClusters fixed with babf273

call from condor_Clean?

-forcex?