License limit

[krabidix]

Hello,
how exactly license consumption is counted for QuantumATK jobs. From user experience, the license usage does not scale linearly with -n in the MPI command.
For instance, I’ve submitted jobs using --nodes=1 to --nodes=4 (sometimes 6 too) and corresponding values like -n 40, -n 80, and even -n 160, and they’ve often run without any license-related errors.
This makes it difficult to determine in advance how many licenses a job will actually consume. But sometimes I got the instructions from the IT admin that I used more than 100 Licenses (with the same setting of submission script). Below is the submission script we use for submitting jobs in our local cluster.
#!/bin/bash
# Job name
#SBATCH --job-name hse_sc

#SBATCH --nodes=1                       # Number of nodes
#SBATCH --ntasks-per-node=40              # How many tasks on each node
##SBATCH --cpus-per-task=20
#SBATCH --time=72:00:00
#SBATCH --exclusive
#Export all environment variables
#SBATCH --export=ALL


# Load the QuantumATK module

# Load the QuantumATK module
module use /n/work00/software/modulefiles
module load puck_quantumatk/2023.12-sp1


# Set the executables
export ATK_EXE=/n/work00/software/quantumatk/V-2023.12-SP1/bin/atkpython
export MPI_EXE=/n/work00/software/quantumatk/V-2023.12-SP1/mpi/bin/mpiexec.hydra
export SNPSLMD_LICENSE_FILE=27027@acme

#export MKL_DYNAMIC=TRUE


# Threading
#export OMP_NUM_THREADS=2             
export MKL_DYNAMIC=TRUE



${MPI_EXE} -n 12 ${ATK_EXE} optical_spectrum.py > optical_doped.log

[Anders Blom]

The licenses are more related to what's in the Python script than the settings, at least in terms of the main qatk_quantum licenses. The -n keyword for MPI basically only determines how many DP licenses will be used for parallelization, but if you have a complex task in the script like a ChargedPointDefect calculation, which runs several DFT calculations, these can be run simultaneously, or sequentially, depending on how many nodes you allocate, but also how many processes you assign per task.

So in a sense it's very simple, but the complexity comes in the interplay between what the script contains and the hardware you allocate, and cannot be stated generically, except for the fact that every DFT calculation needs one qatk_quantum license (similar for forcefield etc) and can be parallelized by qatk_dp licenses.

[krabidix]

Thanks for the reply.
Is there anyway to get at least a rough estimate for the number of license usage for a given script? Sometimes even for simple Band structure calculations of large system (lets say ~100-200 atoms) with DFT-PW in PAW and HSE06, 100 Licenses are insufficient?  Please consider the documentation of this issue, even toy examples can help/guide a lot. Currently, nothing exist for the License limit on QuantumATK documentation.

[Anders Blom]

There is no need for an estimate, you can tell exactly in a simple way. The kind of calculation you mention should use exactly 1 license of the qatk_quantum type, because only 1 calculation is running, and if you run it mpiexec -n it will additionally check out n-1 qatk_dp licenses. That's it.

[Anders Blom]

You can add the environment variable FLEXLM_DIAGNOSTICS=3 to a run, this will show licenses being checking out and in. That way you can track the license usage for some simple calculations, and you will see the logic.