PARTICLES 2025

Coupling DualSPHysics+ with a deep reinforcement learning framework for active flow control

  • Zhan, Yi (Zhejiang University)
  • Luo, Min (Zhejiang University)
  • Khayyer, Abbas (Kyoto University)

Please login to view abstract download link

This work presents a two-way coupling model between the Smoothed Particle Hydrodynamics (SPH) method and Deep Reinforcement Learning (DRL) method to find an optimal control strategy for a moving object in ocean environments. The SPH-DRL coupling is implemented using the open-source SPH code DualSPHysics+ [1] (an enhanced version of DualSPHysics [2]) and the machine learning framework LibTorch (the C++ distribution of PyTorch [3]). During each training episode, DualSPHysics+ provides real-time flow field data to the DRL agent, guides the agent's actions and updates the flow field to obtain corresponding rewards. The Twin-Delayed Deep Deterministic Policy Gradient (TD3) algorithm is employed to ensure stable updates of the neural networks. The proposed coupling model is fully implemented in C++, avoiding the inefficient data exchange interfaces typically seen in conventional CFD-DRL models. Moreover, GPU parallel acceleration is applied to both SPH simulations and DRL network training, enabling the efficient simulation of large-scale three-dimensional problems. The robustness, accuracy, and computational efficiency of the proposed coupling model are validated through two benchmark cases, including sloshing suppression and the active control of a three-dimensional heaving box breakwater. The results demonstrate that the fluid fields reproduced by DualSPHysics+ are physically consistent, and the DRL agent successfully learns the optimal control strategy after multiple training iterations. Additionally, compared to traditional CPU-based models, the proposed framework achieves significantly higher computational efficiency and acceleration ratios.