Remote job submission script issue: remote mpiexec doesn't support '-ppn' flag

[ianhill]

Hello all,

I've been trying to run QuantumATK simulations on a remote SLURM machine, however the versions/implementations of 'mpiexec' and 'mpirun' on this cluster do not support the '-ppn' option. As a result I am unable to run any simulations on the remote machine as QuantumATK does not allow for manual tweaking of the submission script or disabling of this feature. I feel like I must be missing something obvious, but the online MPI documentation also doesn't mention a '-ppn' flag, is this from older versions of 'mpirun'?

My question is if there is any way I can circumvent or resolve this issue. It would be really helpful to utilize my remote machine for larger simulations, thanks in advance for anyone who has insight on this topic. The relevant error message is printed below, and I have the R-2020.09-SP1.f51397f65b build of QuantumATK.


------------------------------------------------------
Allocation (N):    -n 32 -ppn 32
------------------------------------------------------
MPI nodes (NCPU):  1
Cores (NCORES):    32
Threads per process: 1
------------------------------------------------------
Node list
xxxxx
Core list
xxxxx
------------------------------------------------------
SLURM: sbatch is running on xxxxxxxxxxxxxxxxxxxxxxx
SLURM: executing partition is xxxxxxxxxx
SLURM: working directory is /scratch/ianhill/xxxxxx/xxxxxxxxx
SLURM: account name is xxxxxxxxxx
SLURM: job name is xxxxxxxxxxxx
SLURM: job identifier is xxxxxxxxxx
------------------------------------------------------
mpirun: Error: unknown option "-ppn"
Type 'mpirun --help' for usage.

[Umberto Martinez]

Hello ianhill,

two comments here:

• any specific reason for not using mpiexec which is shipped with QuantumATK? can you test that? you can find it in QuantumATK_installation_folder/libexec/mpiexec.hydra
• sure, you can still tweak job managers plugins to fit your requirements. see in QuantumATK_installation_folder\lib\site-packages\AddOns\JobManager\...
  these can be useful https://docs.quantumatk.com/tutorials/addons/addons.html and https://docs.quantumatk.com/tutorials/builder_manual/builder_addon/builder_addon.html

[ianhill]

Hello Umberto,

Thanks for the response. Addressing your comments:

1. I had already tried mpiexec, it gives the same '-ppn' flag issue.

2. This seems like a good option, thanks! I'm assuming updating the JobManager AddOn within QuantumATK would overwrite any modifications, correct? Also for folks who aren't comfortable with Python programming this solution may be prohibitively challenging. In my case it's fine, but may be worth considering having a less 'involved' solution path for users without the required background.

Thanks again,
Ian

[Anders Blom]

Sounds like you still are not using the mpiexec that comes with QuantumATK, because that for sure will support -ppn

I agree recoding the Job Manager plugin is for experts but so far we have not really seen cases where it's absolutely necessary, and if the situation is general enough we will of course fix it in an upcoming release.

Please keep the feedback coming, so we can continue to improve the product!