New High Performance GPGPU Code Transformation Framework Applied to Large Production Weather Prediction Code.

We introduce "Hybrid Fortran", a new approach that allows a high performance GPGPU port for structured grid Fortran codes. This technique only requires minimal changes for a CPU targeted codebase, which is a significant advancement in terms of productivity. It has been successfully applied to both dynamical core and physical processes of ASUCA, a Japanese mesoscale weather prediction model with more than 150k lines of code. By means of a minimal weather application that resembles ASUCA's code structure, Hybrid Fortran is compared to both a performance model as well as today's commonly used method, OpenACC. As a result, the Hybrid Fortran implementation is shown to deliver the same or better performance than OpenACC and its performance agrees with the model both on CPU and GPU. In a full scale production run, using an ASUCA grid with 1581 x 1301 x 58 cells and real world weather data in 2km resolution, 24 NVIDIA Tesla P100 running the Hybrid Fortran based GPU port are shown to replace more than 50 18-core Intel Xeon Broadwell E5-2695 v4 running the reference implementation - an achievement comparable to more invasive GPGPU rewrites of other weather models.