Containerisation, CI/CD and Benchmarking Solutions for HPC

Annotation

This training is designed for individuals working on containerisation, automation, and performance optimisation in HPC environments. It provides a structured approach to developing, deploying, and benchmarking HPC applications using modern DevOps methodologies. Participants will learn to create containerised environments, automate workflows with CI/CD pipelines, and evaluate application performance using benchmarking tools.

The course covers both fundamental and advanced topics, including Docker, Singularity/Apptainer, GitHub Actions, GitLab Runners, JacamarCI, and benchmarking with reframe-hpc and feelpp.benchmarking. Practical, hands-on exercises will help attendees apply their knowledge to real HPC environments, improving workflow reproducibility, efficiency, and scalability.

This training is ideal for HPC administrators, software developers, DevOps engineers, researchers, and scientists who want to optimise HPC workflows and benchmark applications at scale.

Benefits for the attendees: what will they learn

1. Containerize HPC Applications - Create, manage, and optimise container images (Docker, Singularity/Apptainer).
2. Implement CI/CD Workflows for HPC - Automate builds, tests, and deployments using GitHub Actions, GitLab Runners, and JacamarCI .
3. Design & Automate Benchmarking Pipelines - Use feelpp.benchmarking and reframe-hpc to evaluate performance, scaling, and resource usage.
4. Optimise & Scale HPC Workloads - Leverage HPC environment modules, schedulers, and best practices for high performance and reproducibility.

Target Audience

• Researchers, Scientists, and Engineers aiming to streamline HPC workflows.
• HPC Administrators or DevOps professionals seeking automation strategies on clusters.
• Software Developers looking to improve reproducibility and performance for scientific workloads.

Prerequisites

Essential :

• Bash/Python scripting
• Git (basic commands)
• YAML for CI configurations

Recommended :

• Docker/Singularity/Apptainer experience
• HPC environment usage (modules, job submission)

Tutors

Christophe Prud'homme - Professor in Applied Mathematics since September 2012 at the University of Strasbourg, France and from 2006 to 2012 in Grenoble. He founded Cemosis and is a manager and contributor to Feel++, an open-source framework for solving PDEs and complex problems. Feel++ is a building block for the Urban Building pilot. His expertise lies in Modeling, Simulation, Optimisation, and High-Performance Computing. He has been involved in numerous courses and tutorials on scientific computing, HPC, and open-source software development. He designed the CI/CD for the Urban Building pilot with automated deployment on EuroHPC systems.

Javier Cladellas is a Junior Research Software Engineer at Cemosis. He has been working as a developer at Cemosis since 2023. In particular, he has focused on developing a sensor measurement database for data validation and assimilation for the UB pilot in the European project Hidalgo2. Javier is also the main developer of the feelpp.benchmarking framework, conceived for benchmarking applications on HPC systems.

Sameer Haroon has been a Research Engineer at HLRS since 2023. He has a background in Systems Engineering and Domain-Specific Language development, and he is an expert in infrastructure-as-code (IaC) provisioning and maintenance of Web Services at HLRS.

Vincent Chabannes has been a Research Engineer at Cemosis in the Institute of Mathematical Research (IRMA) of Strasbourg since 2016. He is in charge of the computing infrastructure department and is one of the main developers of the Feel++ and Ktirio software. Vincent obtained his PhD in applied mathematics in 2013. The thesis focused on developing algorithms for the numerical simulation of blood flows, with particular emphasis on fluid-structure interaction modelling and high-performance computing.