M3GNET

[AsifShah]

Dear Admin,

How to access M3GNET in new QATK release V2023.12
I tried using forcefield block but it does not show M3GNET there?

[AsifShah]

Dear QATK team,

Please respond to this question.

Thanks
regards

[AsifShah]

Dear QATK team,

Could you please answer this? I am not able to find M3GNET in ATK latest release?

[Julian Schneider]

Thank you for your question. Using M3GNet requires the installation of pytorch into QuantumATK first. There are two possibilities how to do that in QuantumATK. Option 1: Installing pytorch in a python virtual environment (venv, see also https://docs.python.org/3/library/venv.html ): This is the safe way as it keeps pytorch separated from the QuantumATK installation. Create a folder with enough space for the venv (we call it $MY_ENV_DIR). Set up the venv using the python executable in the QuantumATK package:

Code

"$QATK_DIR/atkpython/bin/python" -m venv --system-site-packages "$MY_ENV_DIR"

Activate the venv:

Code

source "$MY_ENV_DIR/bin/activate"

Installing pytorch 2.1

Code

python -m pip install torch==2.1.1 --index-url https://download.pytorch.org/whl/cu118

All simulations need to be run in the activated venv. The differences to running a simulation via atkpython is: 1. Use the python executable instead of atkpython. 2. Add the line

Code

from QuantumATK import *

at the top of the python script you want to run. Then M3GNet can be imported and set up like a normal calculator in QuantumATK:

Code

from tremolox.neighborlistcalculator.M3GNetCalculator import TremoloXM3GNetDirectPESCalculator
calculator = TremoloXM3GNetDirectPESCalculator(device='cpu')

Option 2: Installing pytorch directly into the QuantumATK installation. Although we have not encountered any compatibility issue between pytorch and the external modules in quantumatk, there is still a small risk that this can cause problems. Therefore we recommend making a separate installation of QuantumATK for this. The advantage is that the usage is more straightforward. Install pytorch using pip:

Code

"$QATK_DIR/atkpython/bin/python -m pip install torch==2.1.1 --index-url https://download.pytorch.org/whl/cu118

In this case scripts can directly be run via atkpython, as one is used to. M3GNet can be imported and set up like a normal calculator in QuantumATK:

Code

from tremolox.neighborlistcalculator.M3GNetCalculator import TremoloXM3GNetDirectPESCalculator
calculator = TremoloXM3GNetDirectPESCalculator(device='cpu')

To use GPU acceleration one can set

Code

device='cuda'

Note, these instructions are for Linux systems. For Windows the steps are identical but some of the directories have to be adapted slightly, e.g. replacing bin/ with Scripts/ In case you encounter problems with disc space during installation of pytorch, we recommend to set the environment variables XDG_CACHE_HOME (for the pip cache) and TMPDIR to directories where enough disc space is available. Hope this helps.

[AsifShah]

Dear Admin,

Thanks for responding.
I tried the second method (Attached file) and then run the program using atkpython MoS2_M3GNet.py > logfile.log

But I am getting the following error:
"Initialising the GPUs failed: CUDA driver version is insufficient for CUDA runtime version."

Kindly help in this regard.

[Julian Schneider]

Dear Asif,

this message is not necessarily pointing towards an error in the calculation, it just means that the CUDA drivers cannot be initialized.
It could still be that the calculation ran through fine, especially since your calculation ran M3GNet on CPU only.
Could you please double check if the calculations completed ok (meaning that the results files are there and no other error message is written to the log)?
If there are other error messages or a traceback in the log, could you please report them here, then we can let you know how to fix it?

[AsifShah]

Dear Julian Schneider,

Yes, I observed that calculations are running fine. Log file has all the details.

Thanks for your help.

[Julian Schneider]

Good to hear that it works!
I can understand that the warning can be misleading and we will work on improving that.

Best regards,
Julian

[AsifShah]

Dear Julian Schneider,

Thanks for your response.
The M3GNet Molecular Dynamics is running fine on single CPU core, but when I tried running it on multiple CPU cores (using mpiexec.hydra) with ~3100 atoms in a system.

I got the following error:

File "zipdir/ATKExecutables/atkwrappers/__init__.py", line 780, in __run_atkpython File "AuMoS2Au.py", line 418, in <module> Exception: Non trivial MPI calculation with more than one process is not supported as errorOnMPI is True for the Calculator. Exception: Non trivial MPI calculation with more than one process is not supported as errorOnMPI is True for the Calculator. File "build/atkpython/lib/python3.11/site-packages/tremolox/calculation/calculation.py", line 305, in __init__ File "build/atkpython/lib/python3.11/site-packages/tremolox/calculation/calculation.py", line 251, in wrapper File "build/atkpython/lib/python3.11/site-packages/tremolox/calculation/calculation.py", line 402, in _setParallelOptions Exception: Non trivial MPI calculation with more than one process is not supported as errorOnMPI is True for the Calculator. Traceback (most recent call last): File "/home/Asif/QATK_V2023_12/quantumatk/V-2023.12/atkpython/bin/atkpython", line 8, in <module> sys.exit(__run_atkpython())


How do I run it on multiple CPU cores?
Do I need to change something in code?

Thanks

[Julian Schneider]

Dear Asif, The M3GNet calculator does currently not support parallelization over multiple cores via MPI. Instead you could use OMP threading to parallelize over the cores. In the job manager you can do it as shown in the attached screenshot (assuming you are running on 8 cores). From the terminal, one would do it like this:

Code

# Define the number of threads to use
export OMP_NUM_THREADS=8
export MKL_NUM_THREADS=8
mpiexec -n 1 atkpython my_script.py

[AsifShah]

Dear  Julian Schneider,

Thanks for your great quick support.

I have a doubt, so when you say we can use OMP threading, does it mean the simulation will run on single core with multithreading or multi-cores with multi-threading?

Would it effectively increase the speed similar to when using MPI?

Also, In running an MD using M3GNET for bilayer MoS2 ,I have observed M3GNET does not account vdW forces. The two layers separated very far away during MD run. I tried fixing Center of Mass but it didn't help. Have you faced any such problem?

As a remedy, I was thinking to use LJ between two layers but I worry it should not introduce any artificial or additional forces in system. Please provide your comments.

UPDATE: I tested simulations of a structure with ~3000 atoms using:

OMP_NUM_THREADS=46 and MKL_NUM_THREADS=46 (Simulation time for 150 ps = 3 min)
OMP_NUM_THREADS=1 and MKL_NUM_THREADS=1 (Simulation time for 150 ps = 3 min)

but it shows no speedup.

Thanks a lot once again.
Regards

[AsifShah]

Dear  Julian Schneider,

Can you please answer this?

[Julian Schneider]

With OMP-threading I mean multi-core threading, so using as many cores as you have threads. Then you should see an increase in performace. I have tested it and I could see a decrease in simulation time with increasing number of cores / OMP-threads. The actual speedup may depend on the size of the system.

[Julian Schneider]

To include vdW interactions, it is posssible to combine the M3GNet calculator with a D3-dispersion correction (in analogy to what one would in a DFT calculation for such a system). The calculator would look like this:

Code

calculator_m3gnet = TremoloXM3GNetDirectPESCalculator(device='cpu') 
calculator_d3 = TremoloXCalculator(parameters=DispersionD3Z(xc='PBE'))
calculator = CombinedCalculator([calculator_m3gnet,calculator_d3])

[AsifShah]

Dear  Julian Schneider,
Thanks for your response.
Yes I got the speedup.
Earlier I was mistakenly using mpiexec.hydra instead of mpiexec.

Also, Thanks for the vdW code