Performance analysis of open-source distributed file systems for practical large-scale molecular ab initio, density functional theory, and GW + BSE calculations

International Journal of Quantum Chemistry

3 years ago

Performance analysis of open-source distributed file systems for practical large-scale molecular ab initio, density functional theory, and GW + BSE calculations

Kim K. Baldridge, Riccardo Murri, Loïc M. Roch, Tyanko Aleksiev

Application of conventional post-Hartree–Fock theory, density functional theory, and GW + BSE theory, on chemical and biological problems of growing molecular scale, has created a need for innovative designs in high-performance computing infrastructures and associated middleware technologies. The growing trend toward petascale computing creates an enormous demand for highly efficient storage solutions. Notably, in reaching limitations of the scale-up model, technological progress, and continual growth in demand has evolved into solutions that alternatively scale-out. This investigation involves an analysis of commonly used open source storage solutions for high demand ab initio calculations. Comparisons are shown for the most competitive solutions for a broad range of workloads, following disk usage of key ab initio algorithms in the widely used open source electronic structure theory software, GAMESS. Assessment is based on results of read/write speeds and ratios, scaling ability, and impact of block-size on overall performance of the distributed storage. GlusterFS is shown to generally outperform CEPH and local disk file systems, providing the most reasonable alternative to local storage, and is therefore recommended for large-scale molecular calculations. Application of conventional post-Hartree–Fock, density functional, and GW + BSE theories for chemical and biological problems of increasing molecular scale creates an enormous demand for highly efficient storage solutions. In the limitations of the scale-up model, technological progress and continual growth in demand has evolved into solutions that scale-out. Open-source distributed file systems for large-scale molecular calculations are reviewed and performances analyzed and discussed.

Publisher URL: http://onlinelibrary.wiley.com/resolve/doi

DOI: 10.1002/qua.25392