Applicant
Prof. Dr. Harald Köstler
Chair of Computer Science 10 (System Simulation)
Friedrich-Alexander-Universität Erlangen-Nürnberg
Project Overview
This project aims to extend TerraNeo, an extreme-scale mantle convection code originally developed for CPU systems, toward efficient execution on modern GPU-based supercomputers. TerraNeo uses a matrix-free finite-element design in modern C++ with Kokkos + MPI as the core parallel programming model. The matrix-free approach is especially important for GPUs because it reduces memory pressure and avoids storing very large sparse matrices, which is typically a bottleneck in large Stokes/advection–diffusion solves. The codebase also includes the solver and modeling infrastructure needed for full simulations (multigrid/Krylov methods, compressible Stokes discretization, SUPG advection–diffusion, and I/O/checkpointing), so the GPU work is integrated into a complete application rather than a standalone kernel prototype.
A central goal of the project is to make TerraNeo performance-portable, so it can run on all contemporary supercomputing architectures, including systems based on NVIDIA, AMD, and Intel GPUs.
In addition, the project uses adaptive Galerkin coarsening (AGCA), following the method introduced in Large-scale Multigrid with Adaptive Galerkin Coarsening (arXiv:2511.13109), to improve multigrid robustness for strongly varying coefficients while keeping memory use low by applying Galerkin coarse-grid operators only where they are needed.
Scaling runs on both NVIDIA and AMD GPU clusters have already been conducted, including on JUWELS Booster in Jülich (NVIDIA A100-based GPU nodes) and LUMI-G in Finland (AMD MI250X-based GPU nodes), with runs of up to 512 GPUs.
The project code is open-source and developed at https://github.com/mantleconvection/terraneo .