Installing QuantumATK-U-2022.12-Linux64.bin

[hagelberg]

I'm reaching out with a question related to QuantumATK-U-2022.12-Linux64.bin.  I installed this version on the CADES-Machines at Oak Ridge National Laboratories (ORNL), but I fail when trying to run batch jobs with it.

At the same time, I am successful running the earlier version, QuantumATK-T-2022.03-Linux64.bin, on CADES. Here is an example for a working  QuantumATK-T-2022.03-Linux64.bin submission script:

#!/bin/bash
#SBATCH --nodes 3
#SBATCH --ntasks-per-node 16
#SBATCH --cpus-per-task 1
#SBATCH --exclusive
#SBATCH --mem=50g
#SBATCH -p batch
#SBATCH -A cnms
#SBATCH -t 48:00:00

module load PE-intel
export SNPSLMD_LICENSE_FILE="27020@151.141.244.36"

/home/fhagelberg/QuantumATK/QuantumATK-T-2022.03/libexec/mpiexec.hydra -n 3 /home/fhagelberg/QuantumATK/QuantumATK-T-2022.03//bin/atkpython bias_run_AFM_0.4_gate_polarized-b.py > bias_run_AFM_0.4_gate_polarized-b.log
#END


Here is a submission script I used to test QuantumATK-U-2022.12-Linux64.bin:

#!/bin/bash
#SBATCH --nodes 1
#SBATCH --ntasks-per-node 2
#SBATCH --cpus-per-task 1
#SBATCH --exclusive
#SBATCH --mem=50g
#SBATCH -p batch
#SBATCH -A cnms
#SBATCH -t 4:00:00

module load PE-intel
export SNPSLMD_LICENSE_FILE="27020@151.141.244.36"

/home/fhagelberg/QuantumATK-U/QuantumATK-U-2022.12/mpi/bin/mpiexec.hydra -n 1 /home/fhagelberg/QuantumATK-U/QuantumATK-U-2022.12/atkpython/bin/atkpython MoSe2_H_AFM.py > MoSe2_H_AFM-b.log
#END

This script fails, with the following error message:

/home/fhagelberg/QuantumATK-U/QuantumATK-U-2022.12/atkpython/bin/atkpython: line 2: /slowfs/qatkdev2/bamboo/de02vlbamboo20/bamboo-agent-home/xml-data/build-dir/quantumsource/application/build/atkpython/bin/python: No such file or directory

I also want to mention that, installing  QuantumATK-U-2022.12-Linux64.bin on CADES, I received a success message, but also a note that said "Cannot install wrappers".

Would you have some advice on what to change in my QuantumATK-U-2022.12-Linux64.bin script?

I add that the technical staff of CADES could not help me with this. I sent this question two times to Synopsys Support  - for which I qualify - but did not receive any answer -  I'm hoping for better luck with the QuantumATK forum.

[Anders Blom]

I will ensure someone with deeper knowledge on Linux takes a look. The warning/error you got seems to indicate it was unable to generate softlinks and/or wrapper scripts, which is actually what "atkpython/bin/atkpython" is. Maybe you will need to set those up by hand.

[filipr]

Hmm yes, the installer failed to install the launcher scripts.

Can you try to install it again (just remove/uninstall the current installation first), but when you do:

• Run installer with debug output, i.e.: ./QuantumATK-U-2022.12-Linux64.bin --debuglevel 4
• Specify installation directory as absolute path

If it fails again, there should be a log file installbuilder_installer_*.log (with * the highest number) from the installer in your /tmp directory - please send us that file, so that we can see what went wrong and why.

Also, since you have to install it again, I recommend getting the new service pack version. The installation process hasn't changed, though.

[Ulrik G. Vej-Hansen]

Hi Frank, your previous inquiries had not reached our dashboard, but I could find them in the system when I knew what I was looking for. I'm sorry about that. We'll investigate why they got lost in transit.

[hagelberg]

It seems that the installbuilder file was not attached to my previous post. According to the response, there was a size problem. This is strange since the file was within the specified size limits (and had one of the allowed extensions (log)). Anyway, I try again - this time, I submit the file in three portions.

[hagelberg]

Thanks for your instructions - I deinstalled the earlier version of  QuantumATK-U-2022.12  and followed your description when installing QuantumATK-U-2022.12-SP1.

Here is my test script:

#!/bin/bash
#SBATCH --nodes 1
#SBATCH --ntasks-per-node 8
#SBATCH --cpus-per-task 1
#SBATCH --exclusive
#SBATCH --mem=50g
#SBATCH -p batch
#SBATCH -A cnms
#SBATCH -t 4:00:00

module load PE-intel
export SNPSLMD_LICENSE_FILE="27020@151.141.244.36"

/home/fhagelberg/QuantumATK/QuantumATK-U-2022.12-SP1/mpi/bin/mpiexec.hydra -n 1 /home/fhagelberg/QuantumATK/QuantumATK-U-2022.12-SP1/atkpython/bin/atkpython Fgraphene-half-spinit-lower-edge.py > Fgraphene-half-spinit-lower-edge.log
#END

Once more, the job fails with an error message:

Traceback (most recent call last):
  File "/home/fhagelberg/QuantumATK/QuantumATK-U-2022.12-SP1/atkpython/bin/atkpython", line 8, in <module>
    sys.exit(__run_atkpython())
  File "zipdir/ATKExecutables/atkwrappers/__init__.py", line 837, in __run_atkpython
  File "<frozen zipimport>", line 281, in load_module
  File "zipdir/NL/__init__.py", line 13, in <module>
  File "zipdir/NL/ATKLicenseHelpers.py", line 193, in atkpythonLicenseCheck
  File "zipdir/NL/ATKLicenseHelpers.py", line 167, in atkpythonLicenseCheck
  File "<frozen zipimport>", line 281, in load_module
  File "zipdir/NLEngine.py", line 21, in <module>
ImportError: /software/tools/compilers/intel/compilers_and_libraries_2016.1.150/linux/mpi/intel64/lib/libmpifort.so.12: undefined symbol: MPI_UNWEIGHTED

I submitted the installbuilder file in a separate post. Please let me know about any other relevant information I could supply.
 
Frank

[filipr]

Hi Frank,

It appears the installation was successful this time. We did have an issue with using relative paths for installation directories in the past, but it should have been fixed in version 2022.12, so it's still unknown what the original issue was. Most likely something system specific.

Anyway, you new problem is that the program finds the libraries of an existing Intel MPI installation from 2016 (/software/tools/compilers/intel/compilers_and_libraries_2016.1.150/linux/mpi/intel64/lib/libmpifort.so.12). Maybe that's what you load with module load PE-intel? Try to remove that from your submission script. Also be sure that your PATH and LD_LIBRARY_PATH environment variables do not contain directories to any MPI libraries, and that no environment variables called I_MPI_ROOT and MKLROOT exist. Typically these are set in e.g. your .bashrc by sourcing environment scripts shipped by Intel software like source /path/to/intel/mpi/mpivars.sh, so be sure none of that is done. You don't want your shell environment to contain any traces of MPI, whether it is OpenMPI, MPICH or Intel-MPI.

[hagelberg]

Removing the line 'module load PE-intel' from the script settled the issue.
Thanks so much,

Frank