Accuracy vs. Nb of processors (2)

[pmignon]

Hello,
Some days ago I reported a problem concerning a difference in the results I obtained with respect to the number of processors I use. Starting with identical files (INCAR POSCAR POTCAR KPOINTS) using 16 or 32 processors did not give the same results: difference in Energy, final relaxed geometry and forces.
It was suggested to use the same number of NBANDS for 16 or 32 processors. That I tried ...
However even if I use the same NBANDS for 16 or 32 processors, I still do not get the same energies and forces ... and geometry

Any suggestions ?
Thank you
Pierre.
Here follows the difference between the two OUTCAR files :
2,4c2,4
< executed on Marenostrum date 2008.06.05 20:42:56
< running on 16 nodes
< distr: one band on 1 nodes, 16 groups
---
> executed on Marenostrum date 2008.06.04 11:50:56
> running on 32 nodes
> distr: one band on 1 nodes, 32 groups
670,671c670,671
< NSW = 30 number of steps for IOM
< NBLOCK = 1; KBLOCK = 30 inner block; outer block
---
> NSW = 550 number of steps for IOM
> NBLOCK = 1; KBLOCK = 550 inner block; outer block
713c713
< EBREAK = 0.65E-08 absolut break condition
---
> EBREAK = 0.70E-08 absolut break condition
1111c1111
< For predicting wavefunctions 312.50 MBYTES are necessary
---
> For predicting wavefunctions 312.34 MBYTES are necessary
1156c1156
< eigenvalues EBANDS = 1456.91958248
---
> eigenvalues EBANDS = 1456.91958247
1484,1485c1484,1485
< eigenvalue-minimisations : 2131
< total energy-change (2. order) :-0.1349615E-03 (-0.3694815E-03)
---
> eigenvalue-minimisations : 2130
> total energy-change (2. order) :-0.1349567E-03 (-0.3694767E-03)
1487c1487
< augmentation part 58.1510854 magnetization
---
> augmentation part 58.1510853 magnetization
1502c1502
< eigenvalues EBANDS = -5227.69313969
---
> eigenvalues EBANDS = -5227.69313968
1523c1523
< augmentation part 49.9245184 magnetization
---
> augmentation part 49.9245185 magnetization
1538,1540c1538,1540
< -1/2 Hartree DENC = -13695.26359095
< -V(xc)+E(xc) XCENC = 1870.84222473
< PAW double counting = 22035.36718066 -21759.80527778
---
> -1/2 Hartree DENC = -13695.26356442
> -V(xc)+E(xc) XCENC = 1870.84222286
> PAW double counting = 22035.36717896 -21759.80527842
1542c1542
< eigenvalues EBANDS = -3999.89760864
---
> eigenvalues EBANDS = -3999.89763102
1545c1545
< free energy TOTEN = -911.20341110 eV
---
> free energy TOTEN = -911.20341118 eV
1547c1547
< energy without entropy = -911.20341110 energy(sigma->0) = -911.20341110
---
> energy without entropy = -911.20341118 energy(sigma->0) = -911.20341118

[forsdan]

energy without entropy = -911.20341110 energy(sigma->0) = -911.20341110

and

energy without entropy = -911.20341118 energy(sigma->0) = -911.20341118

looks the same to me. I hope you're not referring to the difference in the 8:th digit? In your previous posting you had a difference of roughly 0.13 eV between the final geometries and that's a noticable difference while this current difference is nothing.

I can't say anything about the forces and geometry since you didn't post them, but if you get identical energies the geometry must be identical as well.

Cheers,
/Dan Fors



<span class='smallblacktext'>[ Edited Fri Jun 06 2008, 11:34PM ]</span>

[pmignon]

Dear,

You are right this is a small difference in energy in the 8:th digit. But this is the first electronic optimization. Since my systems need around 150-250 ionic optimization steps ... Finally the difference in Energy is quite noticeable.
at 150 steps I have:
16 proc:
150 FORCES: Max/Atom : 0.125149 RMS : 0.029298
150 free energy TOTEN = -912.063741 eV
32 proc:
150 FORCES: Max/Atom : 0.101954 RMS : 0.020793
150 free energy TOTEN = -912.053593 eV

Delta(E)~ 0.01 eV

and 16porc:
176 FORCES: Max/Atom : 0.212646 RMS : 0.056338
176 free energy TOTEN = -912.114840 eV

32proc:
176 FORCES: Max/Atom : 0.116493 RMS : 0.033692
176 free energy TOTEN = -912.077718 eV

Delta(E)~ 0.04 eV

And forces are still not small enough.
Around 250 steps the difference will be larger .... and the optimized geometries somehow different ...

I am just surprised by this difference ... don't you ?
Pierre.

[forsdan]

Ok, that puts things in another perspective. Two thoughts are that either the forces are not accurate enough or the calculations are not fully converged.

1. What EDIFF and EDIFFG settings do you have? Are they small enough? If EDIFF is too rough you might get slightly different forces at each ionic step which in 200 steps adds up to two slighly different geometries. If EDIFFG is too rough the relaxation algorithm might stop at a point near the equilibrium but not in it, i.e the calculations are not fullt converged. If you continue from the CONTCAR-files with increased accuracy will the two calculations converge to the same value?

2. Is every ionic step converged? I.e. do you reach EDIFF or does the self-consistency loop stop after NELM steps at any stage for either calculation? If this happens the you can obtain forces that are not reliable and thus end up in a geometry which doesn't correspond to the true equilibrium.

3. Which relaxation algorithm do you use? In the case of IBRION = 1, you might have to consider the NELMIN settings. See the manual: http://cms.mpi.univie.ac.at/vasp/vasp/node112.html

4. Have you tried to switch relaxation algorithm and see if the problem persist?

The above considerations all revolve around the fact that at some stage you get a difference in forces due to too low accuracy. If none of the above explains it I'm out of ideas.


Best regards,
/Dan Fors



<span class='smallblacktext'>[ Edited Tue Jun 10 2008, 02:07AM ]</span>

[pmignon]

Thanks for your reply Dan,

1- my Ediff and ediffg parameters are:
EDIFF=1E-5
EDIFFG=-0.05
For ionic steps it is not so accurate but for my systems it seems difficult to get higher accuracy without spending too much time ..

2- electronic relaxation is always relaxed. After NELMIN steps (min nb of electronic steps) the energy is small enough :
at step 30:
29 F= -.91191136E+03 E0= -.91191136E+03 d E =-.186944E-01
N E dE d eps ncg rms rms(c)
RMM: 1 -0.911925495154E+03 -0.91193E+03 -0.26130E+00 1536 0.266E+00 0.279E-01
RMM: 2 -0.911922317566E+03 0.31776E-02 -0.22368E-02 1564 0.206E-01 0.179E-01
RMM: 3 -0.911921588351E+03 0.72921E-03 -0.22004E-03 1729 0.763E-02 0.528E-02
RMM: 4 -0.911921570989E+03 0.17362E-04 -0.25568E-04 1693 0.213E-02 0.170E-02
RMM: 5 -0.911921567179E+03 0.38097E-05 -0.15782E-05 1005 0.635E-03 0.761E-03
RMM: 6 -0.911921565700E+03 0.14795E-05 -0.90843E-06 970 0.462E-03 0.224E-03
RMM: 7 -0.911921565375E+03 0.32460E-06 -0.36812E-06 960 0.318E-03
30 F= -.91192157E+03 E0= -.91192157E+03 d E =-.102055E-01
N E dE d eps ncg rms rms(c)
RMM: 1 -0.911923030023E+03 -0.91192E+03 -0.40173E-01 1536 0.104E+00 0.109E-01
RMM: 2 -0.911922536838E+03 0.49318E-03 -0.34654E-03 1571 0.812E-02 0.702E-02
RMM: 3 -0.911922422364E+03 0.11447E-03 -0.34913E-04 1717 0.305E-02 0.200E-02
RMM: 4 -0.911922419436E+03 0.29273E-05 -0.33933E-05 1102 0.816E-03 0.659E-03
RMM: 5 -0.911922419309E+03 0.12774E-06 -0.53446E-06 970 0.416E-03 0.291E-03
RMM: 6 -0.911922419261E+03 0.47279E-07 -0.31718E-06 943 0.312E-03 0.861E-04
RMM: 7 -0.911922419280E+03 -0.18874E-07 -0.20884E-06 755 0.219E-03
31 F= -.91192242E+03 E0= -.91192242E+03 d E =-.110594E-01

3- I use:
IBRION=2
NELMIN=7

4-
I did not try another relaxation algorithm yet .. do you suggest anything ?


Thanx
Pierre.

[forsdan]

If we assume you are quite close to the equilibrium structure in the final step: decrease EDIFFG to say -0.01 and continue from the CONTCAR files. First try to keep IBRION = 2 otherwise (if it doesn't work) switch to IBRION = 1, which should work well close to equilibrium.

If you still don't get the same results, try do run the calculation on a different resource if you have the possibility, in order to see if it's connected to your compilation.

Best regards,
/Dan Fors

<span class='smallblacktext'>[ Edited Thu Jun 12 2008, 09:29PM ]</span>