SERIOUS PROBLEM: ADDGRID = T and PGF 7.0 compiler option
Posted: Thu Aug 07, 2008 11:06 am
Dear all,
I encountered a strange behaviour of the vasp code (4.6.34)
when using the ADDGRID flag together with a certain compiler option (PGF 7.0).
The host is an AMD opteron (64-bit).
In order to run very big jobs I need the compiler flag:
-i8 -mcmodel=medium .
The -mcmodel=medium allows you to use objects/arrays larger than 2GB.
Everything runs smoothly unless I use the ADDGRID = T flag in the INCAR file.
Then the job outputs complete nonsense.
Choosing ADDGRID = F makes everything fine again.
Choosing no optimization ( -O0 ) doesnt change this behaviour.
If I compile without the -mcmodel=medium flag both settings for ADDGRID work fine!
The output looks like:
ADDGRID = F:
vasp.4.6.34 5Dec07 complex
POSCAR found : 1 types and 2 ions
LDA part: xc-table for Ceperly-Alder, Vosko type interpolation para-ferro
POSCAR, INCAR and KPOINTS ok, starting setup
WARNING: wrap around errors must be expected
FFT: planning ... 1
reading WAVECAR
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.566219835219E+02 0.56622E+02 -0.60036E+03 792 0.103E+03
DAV: 2 -0.148459627134E+02 -0.71468E+02 -0.65149E+02 768 0.205E+02
DAV: 3 -0.161562024595E+02 -0.13102E+01 -0.12494E+01 960 0.363E+01
DAV: 4 -0.161597296207E+02 -0.35272E-02 -0.35270E-02 912 0.177E+00
DAV: 5 -0.161597483885E+02 -0.18768E-04 -0.18767E-04 984 0.122E-01
DAV: 6 -0.161597485479E+02 -0.15944E-06 -0.16017E-06 944 0.102E-02
DAV: 7 -0.161597485495E+02 -0.15578E-08 -0.15219E-08 768 0.106E-03
DAV: 8 -0.161597485489E+02 0.56585E-09 -0.16212E-09 672 0.442E-04 0.116E+01
RMM: 9 -0.139560421337E+02 0.22037E+01 -0.34412E+00 836 0.233E+01 0.321E+00
RMM: 10 -0.138211469680E+02 0.13490E+00 -0.37992E-01 852 0.638E+00 0.821E-01
RMM: 11 -0.138122908470E+02 0.88561E-02 -0.45425E-03 830 0.793E-01 0.745E-01
RMM: 12 -0.138001633899E+02 0.12127E-01 -0.62751E-03 789 0.897E-01 0.164E+00
...
RMM: 38 -0.137568305380E+02 -0.16419E-04 -0.24206E-06 808 0.194E-02 0.105E-02
RMM: 39 -0.137568313038E+02 -0.76581E-06 -0.79284E-07 712 0.105E-02 0.743E-03
RMM: 40 -0.137568318812E+02 -0.57735E-06 -0.51189E-07 682 0.858E-03
1 F= -.13756832E+02 E0= -.13754678E+02 d E =-.646203E-02 mag= 0.5037
ADDGRID = T:
vasp.4.6.34 5Dec07 complex
POSCAR found : 1 types and 2 ions
LDA part: xc-table for Ceperly-Alder, Vosko type interpolation para-ferro
POSCAR, INCAR and KPOINTS ok, starting setup
WARNING: wrap around errors must be expected
FFT: planning ... 1
reading WAVECAR
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.173997949937E+03 0.17400E+03 -0.46589E+03 824 0.724E+02
DAV: 2 0.656287721373E+02 -0.10837E+03 -0.10158E+03 1008 0.318E+02
DAV: 3 0.585456223763E+02 -0.70831E+01 -0.62727E+01 1320 0.790E+01
DAV: 4 0.584578618986E+02 -0.87760E-01 -0.87592E-01 1032 0.770E+00
DAV: 5 0.584568284139E+02 -0.10335E-02 -0.10331E-02 1248 0.695E-01
DAV: 6 0.584568212435E+02 -0.71704E-05 -0.71721E-05 1064 0.569E-02
DAV: 7 0.584568210736E+02 -0.16984E-06 -0.16777E-06 1088 0.702E-03
DAV: 8 0.584568210710E+02 -0.26819E-08 -0.58271E-08 792 0.178E-03 0.613E+01
RMM: 9 0.905197725044E+02 0.32063E+02 -0.11165E+02 1122 0.703E+01 0.442E+01
RMM: 10 0.175218455747E+02 -0.72998E+02 -0.41184E+02 1068 0.150E+02 0.240E+01
RMM: 11 0.171279617661E+02 -0.39388E+00 -0.20322E+01 1175 0.405E+01 0.224E+01
RMM: 12 0.840816898175E+02 0.66954E+02 -0.94559E+01 1119 0.779E+01 0.225E+01
RMM: 13 0.490553176400E+02 -0.35026E+02 -0.46387E+01 973 0.619E+01 0.736E+00
RMM: 14 0.344819916057E+02 -0.14573E+02 -0.87431E+00 1118 0.250E+01 0.103E+00
RMM: 15 0.341700925945E+02 -0.31190E+00 -0.48299E-01 1000 0.547E+00 0.565E-01
RMM: 16 0.355208891978E+02 0.13508E+01 -0.46073E-02 1051 0.194E+00 0.590E-01
RMM: 17 0.337351553095E+02 -0.17857E+01 -0.53947E-02 1195 0.189E+00 0.132E-01
RMM: 18 0.335229335906E+02 -0.21222E+00 -0.39278E-03 1204 0.582E-01 0.222E-01
RMM: 19 0.341399808517E+02 0.61705E+00 -0.46336E-03 1138 0.603E-01 0.209E-02
RMM: 20 0.342522390523E+02 0.11226E+00 -0.44832E-04 1200 0.199E-01 0.460E-02
RMM: 21 0.341057677034E+02 -0.14647E+00 -0.28016E-04 1109 0.146E-01 0.640E-03
RMM: 22 0.341135887858E+02 0.78211E-02 -0.71493E-06 1124 0.240E-02 0.399E-03
RMM: 23 0.341302637297E+02 0.16675E-01 -0.37529E-06 1094 0.181E-02 0.264E-03
RMM: 24 0.341245811462E+02 -0.56826E-02 -0.29491E-07 662 0.613E-03 0.123E-03
RMM: 25 0.341204461218E+02 -0.41350E-02 -0.46856E-07 780 0.625E-03 0.994E-04
....
The input files are:
INCAR:
System = Bulk-Co (hcp)
GGA = 91
ISTART = 0
ICHARG = 2
PREC = Acc
LREAL = F
ENCUT = 300
ENAUG = 900
ADDGRID = T oder F
NBANDS = 16
SIGMA = 0.10
ISMEAR = 1
ISPIN = 2
VOSKOWN = 1
ALGO = Fast
AMIX = 0.2
BMIX = 0.01
NELM = 40
NELMDL = 8
NELMIN = 10
EDIFF = 1E-7
KPOINTS:
K-Points
0
Gamma
4 4 4
0 0 0
POSCAR:
hcp:
1.000000000000000
2.4881392114111620 0.0000000000000000 0.0000000000000000
1.2440696057055820 2.1547917652342470 0.0000000000000000
0.0000000000000000 0.0000000000000000 4.0382499401203180
2
Direct
0.0 0.0 0.0
0.3333333 0.3333333 0.5
Makefile:
.SUFFIXES: .inc .f .f90 .F
# all CPP processed fortran files have the extension .f90
SUFFIX=.f90
#-----------------------------------------------------------------------
# fortran compiler and linker
#-----------------------------------------------------------------------
FC=pgf90
# fortran linker
FCL=$(FC)
CPP_ = ./preprocess <$*.F | /usr/bin/cpp -P -C -traditional >$*$(SUFFIX)
CPP = $(CPP_) -DHOST=\"LinuxPGF\" \
-Dkind8 -DNGXhalf -DCACHE_SIZE=65536 -DPGF90 -Davoidalloc \
-DRPROMU_DGEMV -DRACCMU_DGEMV
#-----------------------------------------------------------------------
FFLAGS = -Mfree -tp k8-64 -i8 -mcmodel=medium
#-----------------------------------------------------------------------
OFLAG=-O3 -fast
SMALLOPT=-O0
DEBUG = -O0
INLINE = $(OFLAG)
#-----------------------------------------------------------------------
VASPLIB=../vasp.4.lib_mcmodel
BLAS= -lpthread /home/supas163/lib/GotoBLAS_i8/libgoto.a
LAPACK= ${VASPLIB}/lapack_atlas.o
#-----------------------------------------------------------------------
LIB = -L${VASPLIB} -ldmy \
${VASPLIB}/linpack_double.o $(LAPACK) \
$(BLAS)
#-----------------------------------------------------------------------
# fft libraries:
FFT3D = fftw3d.o fft3dlib.o ../../lib/fftw-3.1.2_mcmodel/lib/libfftw3.a
#-----------------------------------------------------------------------
# general rules and compile lines
#-----------------------------------------------------------------------
...... (nothing changed following these lines)
Thanks for your reply!
I encountered a strange behaviour of the vasp code (4.6.34)
when using the ADDGRID flag together with a certain compiler option (PGF 7.0).
The host is an AMD opteron (64-bit).
In order to run very big jobs I need the compiler flag:
-i8 -mcmodel=medium .
The -mcmodel=medium allows you to use objects/arrays larger than 2GB.
Everything runs smoothly unless I use the ADDGRID = T flag in the INCAR file.
Then the job outputs complete nonsense.
Choosing ADDGRID = F makes everything fine again.
Choosing no optimization ( -O0 ) doesnt change this behaviour.
If I compile without the -mcmodel=medium flag both settings for ADDGRID work fine!
The output looks like:
ADDGRID = F:
vasp.4.6.34 5Dec07 complex
POSCAR found : 1 types and 2 ions
LDA part: xc-table for Ceperly-Alder, Vosko type interpolation para-ferro
POSCAR, INCAR and KPOINTS ok, starting setup
WARNING: wrap around errors must be expected
FFT: planning ... 1
reading WAVECAR
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.566219835219E+02 0.56622E+02 -0.60036E+03 792 0.103E+03
DAV: 2 -0.148459627134E+02 -0.71468E+02 -0.65149E+02 768 0.205E+02
DAV: 3 -0.161562024595E+02 -0.13102E+01 -0.12494E+01 960 0.363E+01
DAV: 4 -0.161597296207E+02 -0.35272E-02 -0.35270E-02 912 0.177E+00
DAV: 5 -0.161597483885E+02 -0.18768E-04 -0.18767E-04 984 0.122E-01
DAV: 6 -0.161597485479E+02 -0.15944E-06 -0.16017E-06 944 0.102E-02
DAV: 7 -0.161597485495E+02 -0.15578E-08 -0.15219E-08 768 0.106E-03
DAV: 8 -0.161597485489E+02 0.56585E-09 -0.16212E-09 672 0.442E-04 0.116E+01
RMM: 9 -0.139560421337E+02 0.22037E+01 -0.34412E+00 836 0.233E+01 0.321E+00
RMM: 10 -0.138211469680E+02 0.13490E+00 -0.37992E-01 852 0.638E+00 0.821E-01
RMM: 11 -0.138122908470E+02 0.88561E-02 -0.45425E-03 830 0.793E-01 0.745E-01
RMM: 12 -0.138001633899E+02 0.12127E-01 -0.62751E-03 789 0.897E-01 0.164E+00
...
RMM: 38 -0.137568305380E+02 -0.16419E-04 -0.24206E-06 808 0.194E-02 0.105E-02
RMM: 39 -0.137568313038E+02 -0.76581E-06 -0.79284E-07 712 0.105E-02 0.743E-03
RMM: 40 -0.137568318812E+02 -0.57735E-06 -0.51189E-07 682 0.858E-03
1 F= -.13756832E+02 E0= -.13754678E+02 d E =-.646203E-02 mag= 0.5037
ADDGRID = T:
vasp.4.6.34 5Dec07 complex
POSCAR found : 1 types and 2 ions
LDA part: xc-table for Ceperly-Alder, Vosko type interpolation para-ferro
POSCAR, INCAR and KPOINTS ok, starting setup
WARNING: wrap around errors must be expected
FFT: planning ... 1
reading WAVECAR
entering main loop
N E dE d eps ncg rms rms(c)
DAV: 1 0.173997949937E+03 0.17400E+03 -0.46589E+03 824 0.724E+02
DAV: 2 0.656287721373E+02 -0.10837E+03 -0.10158E+03 1008 0.318E+02
DAV: 3 0.585456223763E+02 -0.70831E+01 -0.62727E+01 1320 0.790E+01
DAV: 4 0.584578618986E+02 -0.87760E-01 -0.87592E-01 1032 0.770E+00
DAV: 5 0.584568284139E+02 -0.10335E-02 -0.10331E-02 1248 0.695E-01
DAV: 6 0.584568212435E+02 -0.71704E-05 -0.71721E-05 1064 0.569E-02
DAV: 7 0.584568210736E+02 -0.16984E-06 -0.16777E-06 1088 0.702E-03
DAV: 8 0.584568210710E+02 -0.26819E-08 -0.58271E-08 792 0.178E-03 0.613E+01
RMM: 9 0.905197725044E+02 0.32063E+02 -0.11165E+02 1122 0.703E+01 0.442E+01
RMM: 10 0.175218455747E+02 -0.72998E+02 -0.41184E+02 1068 0.150E+02 0.240E+01
RMM: 11 0.171279617661E+02 -0.39388E+00 -0.20322E+01 1175 0.405E+01 0.224E+01
RMM: 12 0.840816898175E+02 0.66954E+02 -0.94559E+01 1119 0.779E+01 0.225E+01
RMM: 13 0.490553176400E+02 -0.35026E+02 -0.46387E+01 973 0.619E+01 0.736E+00
RMM: 14 0.344819916057E+02 -0.14573E+02 -0.87431E+00 1118 0.250E+01 0.103E+00
RMM: 15 0.341700925945E+02 -0.31190E+00 -0.48299E-01 1000 0.547E+00 0.565E-01
RMM: 16 0.355208891978E+02 0.13508E+01 -0.46073E-02 1051 0.194E+00 0.590E-01
RMM: 17 0.337351553095E+02 -0.17857E+01 -0.53947E-02 1195 0.189E+00 0.132E-01
RMM: 18 0.335229335906E+02 -0.21222E+00 -0.39278E-03 1204 0.582E-01 0.222E-01
RMM: 19 0.341399808517E+02 0.61705E+00 -0.46336E-03 1138 0.603E-01 0.209E-02
RMM: 20 0.342522390523E+02 0.11226E+00 -0.44832E-04 1200 0.199E-01 0.460E-02
RMM: 21 0.341057677034E+02 -0.14647E+00 -0.28016E-04 1109 0.146E-01 0.640E-03
RMM: 22 0.341135887858E+02 0.78211E-02 -0.71493E-06 1124 0.240E-02 0.399E-03
RMM: 23 0.341302637297E+02 0.16675E-01 -0.37529E-06 1094 0.181E-02 0.264E-03
RMM: 24 0.341245811462E+02 -0.56826E-02 -0.29491E-07 662 0.613E-03 0.123E-03
RMM: 25 0.341204461218E+02 -0.41350E-02 -0.46856E-07 780 0.625E-03 0.994E-04
....
The input files are:
INCAR:
System = Bulk-Co (hcp)
GGA = 91
ISTART = 0
ICHARG = 2
PREC = Acc
LREAL = F
ENCUT = 300
ENAUG = 900
ADDGRID = T oder F
NBANDS = 16
SIGMA = 0.10
ISMEAR = 1
ISPIN = 2
VOSKOWN = 1
ALGO = Fast
AMIX = 0.2
BMIX = 0.01
NELM = 40
NELMDL = 8
NELMIN = 10
EDIFF = 1E-7
KPOINTS:
K-Points
0
Gamma
4 4 4
0 0 0
POSCAR:
hcp:
1.000000000000000
2.4881392114111620 0.0000000000000000 0.0000000000000000
1.2440696057055820 2.1547917652342470 0.0000000000000000
0.0000000000000000 0.0000000000000000 4.0382499401203180
2
Direct
0.0 0.0 0.0
0.3333333 0.3333333 0.5
Makefile:
.SUFFIXES: .inc .f .f90 .F
# all CPP processed fortran files have the extension .f90
SUFFIX=.f90
#-----------------------------------------------------------------------
# fortran compiler and linker
#-----------------------------------------------------------------------
FC=pgf90
# fortran linker
FCL=$(FC)
CPP_ = ./preprocess <$*.F | /usr/bin/cpp -P -C -traditional >$*$(SUFFIX)
CPP = $(CPP_) -DHOST=\"LinuxPGF\" \
-Dkind8 -DNGXhalf -DCACHE_SIZE=65536 -DPGF90 -Davoidalloc \
-DRPROMU_DGEMV -DRACCMU_DGEMV
#-----------------------------------------------------------------------
FFLAGS = -Mfree -tp k8-64 -i8 -mcmodel=medium
#-----------------------------------------------------------------------
OFLAG=-O3 -fast
SMALLOPT=-O0
DEBUG = -O0
INLINE = $(OFLAG)
#-----------------------------------------------------------------------
VASPLIB=../vasp.4.lib_mcmodel
BLAS= -lpthread /home/supas163/lib/GotoBLAS_i8/libgoto.a
LAPACK= ${VASPLIB}/lapack_atlas.o
#-----------------------------------------------------------------------
LIB = -L${VASPLIB} -ldmy \
${VASPLIB}/linpack_double.o $(LAPACK) \
$(BLAS)
#-----------------------------------------------------------------------
# fft libraries:
FFT3D = fftw3d.o fft3dlib.o ../../lib/fftw-3.1.2_mcmodel/lib/libfftw3.a
#-----------------------------------------------------------------------
# general rules and compile lines
#-----------------------------------------------------------------------
...... (nothing changed following these lines)
Thanks for your reply!