ATK parallel with more than 100% CPU

[liximao]

Dear All,
Hello!
I finish the installation of atk 12.8 with a floating license.
The parallel test with test_mpi is normal when submit it with mpich2. It show 3 slave and 1 master when 4 cpu is occupied.
But there is still a problem there: it will  occupy more than 100% cpu.
You can see that as  list below that there are four atkpython processes , but it will occupy all the cpu of this node when I submit 1 atk job with four  cpu.
 
 [root@hn test]# /share/appr/mpich2-1.5/bin/mpiexec -n 4 atkpython zigzag.py > 3.out&
[root@hn test]# top
top - 17:31:15 up 32 days,  7:03,  3 users,  load average: 0.88, 3.29, 3.59
Tasks: 617 total,   5 running, 612 sleeping,   0 stopped,   0 zombie
Cpu0  : 89.4%us,  9.3%sy,  0.0%ni,  1.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  : 89.8%us,  9.3%sy,  0.0%ni,  0.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  : 84.9%us,  8.4%sy,  0.0%ni,  6.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  : 79.2%us, 10.6%sy,  0.0%ni, 10.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  : 84.2%us, 10.3%sy,  0.0%ni,  5.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  : 83.5%us, 10.7%sy,  0.0%ni,  5.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  : 86.9%us,  8.5%sy,  0.0%ni,  4.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  : 76.9%us, 10.3%sy,  0.0%ni, 12.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  : 85.5%us, 11.5%sy,  0.0%ni,  3.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  : 85.5%us, 11.5%sy,  0.0%ni,  3.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 : 89.8%us,  9.3%sy,  0.0%ni,  0.8%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 : 86.1%us, 10.9%sy,  0.0%ni,  2.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 : 89.4%us, 10.6%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 : 83.1%us, 11.4%sy,  0.0%ni,  5.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 : 85.0%us, 10.7%sy,  0.0%ni,  4.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 : 81.4%us, 10.1%sy,  0.0%ni,  8.4%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65944340k total, 62385596k used,  3558744k free,   423196k buffers
Swap: 131071992k total,        0k used, 131071992k free, 56208136k cached
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
22376 root      20   0 1565m 224m  20m R 582.4 0.3   0:31.53 atkpython_exec
22375 root      20   0 1564m 212m  20m R 482.8  0.3   0:27.39 atkpython_exec
22374 root      20   0 1567m 197m  21m R 231.7  0.3   0:16.42 atkpython_exec
22377 root      20   0 1565m 198m  20m R 213.6  0.3   0:19.59 atkpython_exec
18990 liuwei    20   0 1020m 4944 1784 S  0.8  0.0   0:38.83 lmx-serv
 4466 root      20   0  266m  19m 4536 S  0.4  0.0   1:20.49 gnome-screensav
22364 root      20   0 25516 1436 1176 S  0.4  0.0   0:00.01 mpiexec
22378 root      20   0 15484 1688  964 R  0.4  0.0   0:00.26 top
    1 root      20   0 19396 1564 1256 S  0.0  0.0   0:01.97 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.01 kthreadd

Please give me some suggestions, thanks a lot!
 

[Nordland]

I think it is because it is threading using all the cores?

[liximao]

Thanks！
Then, how to resolve it?

[Anders Blom]

Resolve what? The more CPU usage the better performance.

However, it is true that you can suffer in performance if you overload the memory bus etc. But this is more a result of putting too many MPI processes on the same machine. The ideal parallel scenario for the current version of ATK is still few MPIs/node, but using many nodes.

[liximao]

First of all, thanks for your reply!
I understand what you mean.
But here, this node has 16 core, but only less than 8 core for each job was permitted in the lab. And we need  submit other vasp jobs on it. 
The internet manager and other students are disagree with that I use all the 16 core resource.
Can you understand it?
I think maybe I can do some setting to make the atk job occupy only 4 core. Can I do it?

[Anders Blom]

You can experiment with the OMP_NUM_THREADS environment variable.