Cannot assign more than 5 cores for AKMC

[pshinyeong]

Hello,

I am trying to run AKMC with cluster but it seems that I can't assign more than 6 cores. When using low calculation parameter with lcao 1 node 7 cores, qatk_quantum licensed number of users already reached appears.. When using forcefield to run AKMC 1 node 7 cores, qatk_forcefield licensed number of users already reached appears too. 

Considering that AKMC is quite heavy calculation, I don't understand why the licenses stated above can only allow one core per license. It seems that I have 5 qatk_quantum and qatk_forcefield license each.

Is there a way around this?

Thank you

[Anders Blom]

With 5 licenses you can only parallelize over up 5 concurrent saddle searches but each on of these can run on multiple cores, using MPI parallelization. You do however have to set processes_per_saddle_search=N correctly in the ParallelParameters object attached to the calculator (cf. https://docs.quantumatk.com/manual/Types/AdaptiveKineticMonteCarlo/AdaptiveKineticMonteCarlo.html#parallel-saddle-searches), so that 5*N = M = the total number of MPI processes you spawn (mpiexec -n M).

[pshinyeong]

Hello,

I’m revisiting this topic because I encountered the same license error while running AKMC with a trained MTP. Specifically, when I use the forcefield calculator and assign more than 7 cores, I get a license error that says, “licensed number of users already reached.”

You previously mentioned that I could set processes_per_saddle_search in the ParallelParameters object to distribute the cores per saddle search. However, it seems that this option isn’t available when I’m using the forcefield calculator (TremoloXCalculator())

Is there another way to manage core allocation for the forcefield calculator under the same license restrictions? I want to make full use of the available cores to speed up this calculation.

Thanks!

[Anders Blom]

You can not run more saddle searches concurrently than you have force field licenses. Now, normally each search can be parallelized with threads over many cores, however it seems this has not been implemented for force field calculators... I still have to double-check this with some people who are on vacation this week, and if there is a work-around, but for now just run on 6 cores (I don't expect 7 to work, but 6 should because it will have 1 dispatcher and 5 workers).

[Anders Blom]

Sorry, my previous post is not entirely correct.

The number of saddle searches is controlled by how many MPI processes you assign, divided by processes_per_saddle_search. But this is only relevant if you want each saddle search to be MPI parallel, and normally we don't need or want that for force fields.

Instead, what you want is to run 6 (see above) MPI processes, i.e. mpiexec -n 6 but ensure that the job has access to all the cores on the machine, so each saddle search can thread over the cores. This is a matter of reserving the relevant number of cores in the job scheduler, if you have one, and you might need to set the environment variable OMP_NUM_THREADS suitably for this, you can experiment (and let me know how it works out, for reference).

[pshinyeong]

Yes, threading solved the issue. Thank you!