Image
LOMC (UMR CNRS 6294) is a joint research unit associated with Le Havre Normandie University and CNRS. The laboratory specializes in the study of mechanical wave propagation, the behavior of complex media, as well as applications in port and coastal engineering, geo-environment, and non-destructive structural testing. LOMC notably develops Dorothy, a numerical simulation software for computational fluid dynamics (CFD) based on the Vortex Particle Method (VPM), developed jointly with IFREMER. This code, written in Fortran 90 and parallelized with MPI, discretizes the fluid into vorticity-carrying particles.
Logo:
Technical/scientific Challenge:
The Dorothy code relies on a particle redistribution algorithm called "prime_factor," valued for its load balancing efficiency and high-quality numerical results. However, the standard implementation suffers from a major scalability bottleneck: each process must gather all particles via MPI Allgather before computing the new distribution. This causes a significant memory spike, limiting the number of particles that can be processed to the memory capacity of a single process and preventing the code from scaling for large-scale simulations. The challenge was to redesign the algorithm to overcome this bottleneck while maintaining its numerical accuracy and load balancing properties.
Solution:
CRIANN supported LOMC in developing a fully parallel particle redistribution algorithm that eliminates the need for global particle collection. The new algorithm operates in two stages: first, each process computes the new partitioning for its own particles, synchronizing particle counts across processes at each sub-step to ensure global load balancing; second, particles are exchanged according to the new distribution using MPI AllToAll. The algorithm was prototyped in Python using mpi4py and then integrated into the Dorothy code in Fortran 90. Tests were successfully conducted on simulations of approximately 5 million particles across up to 192 cores, demonstrating the method’s efficiency and scalability.
Business impact:
The fully parallel redistribution algorithm removes the previous memory limitation, allowing the Dorothy code to handle much larger particle simulations. Redistribution times remain extremely low (less than 5 ms) even on 192 cores, while partitioning computation scales efficiently. Integration of the algorithm into the Dorothy code during the Extreme CFD Workshop 2025 enables LOMC to perform larger and more complex computational fluid dynamics simulations. This improvement strengthens the laboratory’s capacity to tackle advanced CFD problems and accelerates research outputs in fluid dynamics modeling.
Benefits:
Fully parallel particle redistribution eliminating memory bottlenecks
Efficient load balancing and high numerical accuracy maintained
Enables large-scale simulations of millions of particles
Extremely low redistribution times (<5 ms) on up to 192 cores
Integration into Dorothy code facilitates advanced CFD research and complex simulations