Applicant
Prof. Dr.-Ing. habil Stefan Becker
Institut für Prozessmaschinen und Anlagetechnik
Friedrich-Alexander-Universität Erlangen-Nürnberg
Project Summary
In this project, a library for the leading open-source CFD software, OpenFOAM, was developed that enables the program to output data in an efficient way by writing a single HDF5 archive instead of the default uncollated output structure or the collated file format. The work from Håkon Strandenes (link) was used as groundwork. His outdated library was rewritten to be compatible with the new OpenFOAM code structure (from OpenFOAM V6 upwards) and extended so that also boundary or/and patch data can be written to the HDF5 file. The resulting new library was named “IOH5Write_b_OFv6” and is available on the web-based git-repository manager GitHub (https://github.com/ipat-fau/IOH5Write/tree/IOH5Write_b_OFv6) to ensure that the code is shared with the community and can be forked to customize or further develop the library.
Performance tests for the new library were carried out with relatively large simulation test cases. The simulations run in parallel, distributed on 500 and 1000 cores on the RRZE cluster meggie with Lustre-based file system and Omnipath interconnect (\lxfs, $FASTTMP). The tests clearly show that the IOH5Write_b_OFv6 library generated HDF5 archive reduces the disk space allocation by more than 50 % compared to OpenFOAM’s collated file format output in decomposed state and about more than 40 % compared to the reconstructed data. Furthermore, the library allows it to write data in only one single HDF5 file, instead of one file for each flow quantity and each time step, which is the procedure of OpenFOAM’s latest collated file format.
Time measurements show that the collated file format of OpenFOAM is still the fastest approach if no reconstruction of the decomposed data is necessary. However, the reconstruction of the decomposed data is in some situations necessary because not all post-processing options are available for the decomposed output and because the decomposed data also allocates more disk space compared to the reconstructed data. However, time measurements show that the reconstruction process with OpenFOAM’s reconstructPar utility is a very time intensive routine on the used parallel file system.
This situation makes the IOH5Write_b_OFv6 library and its HDF5 output approach attractive for the OpenFOAM community and for the HPC operating institutions because it reduces the allocated disk space significantly and needs no additional time consuming reconstruction routine. Furthermore, the HDF5 output approach can facilitate efficient coupling between OpenFOAM and third party software that is already using HDF5 as their default I/O data format (e.g. the coupled field simulation software CFS++ or openCFS).
A few limitations exist for the IOH5Write_b_OFv6 library which are listed on the mentioned GitHub web page. The community already used and forked the current library to resolved some of the limitations. One of the OpenFOAM community members addressed the issue that the current library is limited to quadrilateral faces on the boundaries or patches and created a new fork of the IOH5Write_b_OFv6 library which extends the code to work also with polyhedral cells (see [link1, link2]). This demonstrates the benefit of sharing the work with the OpenFOAM community on a web-based git-repository manager like GitHub.