1
General Questions and Answers / Quantifying Memory Requirements for a Parallelized Job
« on: June 16, 2022, 19:09 »
I would like to quantify the actual memory requirements for a parallelized job on a cluster since some jobs have run out of memory in the past due to insufficient RAM from each node. How can we calculate the memory requirements for an entire job calculation (how do we quantifiably determine it from the *.log file where it provides memory per k-pt, per dense matrix dimension, and per real-space grid)? A snippet of the memory requirement from a previous *.log file is shown below:
+------------------------------------------------------------------------------+
| K-point grid: 4 x 4 x 4 |
| Number of irreducible k-points: 32 |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| Real space grid sampling is (209, 209, 209) in a, b, and c directions. |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| Memory requirements for the calculation |
+------------------------------------------------------------------------------+
| Dense matrices: 1.52 GB per matrix [Matrix dimensions 9984 x 9984] |
| Total memory required per k-point: 4.56 GB |
| |
| Storage of real-space orbitals: Enabled |
| Storage requires 306 MB |
| |
| Total memory required per real-space grid: 139 MB |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| SCF History |
+------------------------------------------------------------------------------+
| Memory required to store SCF history: 10.02 GB |
| Number of history steps: 20 |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| K-point grid: 4 x 4 x 4 |
| Number of irreducible k-points: 32 |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| Real space grid sampling is (209, 209, 209) in a, b, and c directions. |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| Memory requirements for the calculation |
+------------------------------------------------------------------------------+
| Dense matrices: 1.52 GB per matrix [Matrix dimensions 9984 x 9984] |
| Total memory required per k-point: 4.56 GB |
| |
| Storage of real-space orbitals: Enabled |
| Storage requires 306 MB |
| |
| Total memory required per real-space grid: 139 MB |
+------------------------------------------------------------------------------+
+------------------------------------------------------------------------------+
| SCF History |
+------------------------------------------------------------------------------+
| Memory required to store SCF history: 10.02 GB |
| Number of history steps: 20 |
+------------------------------------------------------------------------------+