Reinforcement learning with function approximation for 3-spheres swimmer

Luca Berti (1) , Zakarya El Khiyati (2) , Youssef Essousy (3) , Christophe Prud'Homme (1) , Laetitia Giraldi (2)

1 IRMA - Institut de Recherche Mathématique Avancée
2 CALISTO - Stochastic Approaches for Complex Flows and Environment
3 UM6P - Université Mohammed VI Polytechnique [Ben Guerir]

We study the swimming strategies that maximize the speed of the three-sphere swimmer using reinforcement learning methods. First of all, we ensure that for a simple model with few actions, the Q-learning method converges. However, this latter method does not fit a more complex framework (for instance the presence of boundary) where states or actions have to be continuous to obtain all directions in the swimmer's reachable set. To overcome this issue, we investigate another method from reinforcement learning which uses function approximation, and benchmark its results in absence of walls.

Mots clés: Reinforcement learning control; Micro-swimming; Three-sphere swimmer; Function approximation

Domaines: Equations aux dérivées partielles [math.AP]; Optimisation et contrôle [math.OC]

Luca Berti, Zakarya El Khiyati, Youssef Essousy, Christophe Prud'Homme, Laetitia Giraldi. Reinforcement learning with function approximation for 3-spheres swimmer. 2022. ⟨hal-03538754⟩