Any experiences on VASP scalability (large jobs) on 10Gb ethernet? There are some rumors (see e.g. http://www.linux-mag.com/id/7219 ) that scalability on 10GbE would be comparable to InfiniBand -- any truth to this?
If you didn't guess already, I'm building a cluster, and switch recommendations are appreciated. Probably I'll get some of the new Xeon "Nehalem" powered Xserve nodes, and want to get use all of them parallel.
While I'm at it, I might as well ask about compilers. I have good experiences from the PGI compilers. How much faster is ifortran on a Xeon?
Any comments are appreciated.
Hannes
<span class='smallblacktext'>[ Edited ]</span>
10GbE
Moderators: Global Moderator, Moderator
-
job
- Jr. Member
- Posts: 55
- Joined: Tue Aug 16, 2005 7:44 am
10GbE
Moro!
we're also in the business of getting a new cluster (slowly, and bureocratically, as ever..), and from a brief look it seems that while 10GbE has become cheaper, it's still roughly in the same price class as Infiniband. That being said, while one should take an article written by someone in the business of selling 10GbE hardware with a grain of salt, I think he's correct in the longer term. The ethernet juggernaut will squeeze the niche networks like Infiniband and Fibre Channel into increasingly smaller markets. 10GbE integrated on the motherboard could be a big step in this direction.
If you're interested in 10GbE, you might want to take a look at Open-MX: http://open-mx.gforge.inria.fr/ They claim that by getting rid of the TCP/IP overhead they can halve the MPI latency on 10GbE.
As for compilers, Intel is generally the best on Xeon. Maybe ~10 % faster than PGI. OTOH if you want to save a buck or two, gfortran is nowadays pretty close, also about 10 % slower than Intel and on par with PGI. See
http://www.polyhedron.com/pb05-linux-f90bench_p40html
http://users.physik.fu-berlin.de/~tburn ... benchmark/
In my experience anyway, VASP performance is determined more by having a well optimized BLAS, LAPACK and FFTW libraries than by the compiler. Considering this, the Intel MKL library might be a worthwhile investment.
we're also in the business of getting a new cluster (slowly, and bureocratically, as ever..), and from a brief look it seems that while 10GbE has become cheaper, it's still roughly in the same price class as Infiniband. That being said, while one should take an article written by someone in the business of selling 10GbE hardware with a grain of salt, I think he's correct in the longer term. The ethernet juggernaut will squeeze the niche networks like Infiniband and Fibre Channel into increasingly smaller markets. 10GbE integrated on the motherboard could be a big step in this direction.
If you're interested in 10GbE, you might want to take a look at Open-MX: http://open-mx.gforge.inria.fr/ They claim that by getting rid of the TCP/IP overhead they can halve the MPI latency on 10GbE.
As for compilers, Intel is generally the best on Xeon. Maybe ~10 % faster than PGI. OTOH if you want to save a buck or two, gfortran is nowadays pretty close, also about 10 % slower than Intel and on par with PGI. See
http://www.polyhedron.com/pb05-linux-f90bench_p40html
http://users.physik.fu-berlin.de/~tburn ... benchmark/
In my experience anyway, VASP performance is determined more by having a well optimized BLAS, LAPACK and FFTW libraries than by the compiler. Considering this, the Intel MKL library might be a worthwhile investment.
Last edited by job on Fri Apr 17, 2009 12:17 pm, edited 1 time in total.