How many nodes

[ams_nanolab]

In http://quantumwise.com/documents/tutorials/latest/ParallelGuide/index.html/chap.quick.html#sect1.quick.nodes
I found:
--------------
For regular calculations, which are not critical in memory usage, assign one MPI process per socket (essentially a socket is a CPU, although this term is outdated).

For large calculations, which would use more than half of the RAM of each node in serial, assign one MPI process per node (that is, per physical machine).

For very small calculations, in particular with many k-points, assign one MPI process per core.
--------------

I have a 2 Intel xeon processors in my system (each have 12 cores), I have 16 GB (2x8GB) of RAM. For running optimization of large suprecells with DFT, what is the safe number of nodes I can use.

I have license for 1 master and 3 slave. So technically I can run 4 nodes?? or per CPU core (i.e. 24 nodes?)

[Anders Blom]

Please don't rely on the old parallel guide, it's only valid for versions like 12.x or maybe 13.x. We're working on new guidelines for running ATK 2015 in parallel, since there have been a huge amount of improvements and modifications.

Specifically, in ATK 2015 (and 2014 too) you can easily use more than one MPI process per socket/node/CPU, however it's extremely important that you set
MKL_NUM_THREADS=1 if you do that.

So, you don't have 24 nodes, you have 1 node (machine) with 24 cores. That can run up to 24 MPI processes, which perhaps is a bit much on a single node due to memory duplication in MPI, but in principle you could run 24 if you have enough RAM. Then again, since you only have 3 slave licenses, you can at most run 4 MPI processes, so there is nothing to think about: you should definitely use 4 MPIs, i.e. mpiexec -n 4.

Note that "nodes" and "slaves" have nothing to do with each other. "slaves" = how many MPI processes you can parallelize one job over. "nodes" = how many machines you have to distribute those processes across.

[ams_nanolab]

Thanks for the answer, do I need to set MKL_NUM_THREADS=1 before running mpiexec -n 4 , or I can just do it straight way.

[Anders Blom]

The absolutely most general point is that when running

export MKL_NUM_THREADS=X
mpiexec -n Y

the product X*Y must be equal to the number of physical cores on your machine. If the best performance is obtained by primarily using the cores as MPI nodes, or by using threading, depends on the problem and needs to be tested. For DFT - typically it's best to use many MPIs, unless you have few k-points. For classical potentials, all benefit is from threading. No script behaves the same in terms for parallel scaling.

[ams_nanolab]

Okay, thanks for the tip 

[ams_nanolab]

export MKL_NUM_THREADS=X
mpiexec -n Y

whats the "safe" number of threads?? I don't want to damage my machine 

[Anders Blom]

Quite simply, X*Y should be less than or equal to the number of physical cores on the machine.

[ams_nanolab]

Thanks for the help