Why job_mem and job_cpu about dask-jobqueue HOT 3 CLOSED

So after some research, I take it your talking about Slurm implementation, and that's me that introduced these terms.
Several things here IIRC:

• At the beginning, I believe Slurm implementation was a little wrong, and only used processes as the number of cores booked in a Slurm job, leaving aside the thread_per_worker arg.
• job_mem and job_cpu in Slurm are kind of equivalent to PBS resource_spec, and we had a discussion with @jhamman in this issue: #12. From what I've seen, sometimes @jhamman does some over subsription, e.g. booking every CPU from a node on the cluster, but launching more total threads on it: PBSCluster(processes=24, threads=2, resource_spec='select=1;ncpus=24;mem=120GB')

For my part I've never done this, and having job_cpu = processes * threads is currently fine for me. But I can imagine that some people may want to have some more control about it. Hope this helps.

from dask-jobqueue.

I propose to close this and keep these two arguments, are they may be of use to Slurm users as resource_spec is of use to PBS ones.

from dask-jobqueue.

Closing, it's possible we discuss this again if we go on with #133

from dask-jobqueue.