LAUSR

Aquatic creatures such as fish, cetaceans, and jellyfish could inspire innovative designs to improve the ways that manmade systems operate in and interact with aquatic environments [1,2]. Jellyfish in nature use jet propulsion to move through the water, which have been proven to be one of the most energetically efficient swimmers on the planet [3]. Researchers are making an integrated effort to develop smart actuators to fabricate various robotic jellyfish, such as shape memory alloys (SMA) [4], ionic polymer metal composites (IPMC) [5], and dielectric elastomer actuator [6]. Most of existing robotic jellyfish cannot freely adjust their three-axis attitude, which has an adverse effect on free-swimming propulsion and plausible applications. We investigate how to design and control a bio-inspired motor-driven robotic jellyfish capable of three-dimensional (3D) motion. The main contributions of this work are twofold. First, a novel 3D barycenter adjustment mechanism is implemented, allowing flexible regulation of the robot’s barycenter. Second, the proposal of the reinforcement learning based attitude control method makes autonomous attitude regulation possible. In comparison with most of the other robotic jellyfish, the built robot displays a high order of structure flexibility and yaw maneuverability. Therefore, this self-propelled robotic jellyfish with 3D motion has great implications for bio-inspired design of jet propulsion system with great agility. Development of the robotic prototype. As illustrated in Figure 1, the developed robotic jellyfish, which models after Aurelia aurita (commonly termed moon jellyfish), is hemispherical in shape and consists of a bell-shaped rigid head, a cylindroid main cavity, four separate six-bar linkage mechanisms, and a soft rubber skin. Considering most existing robotic jellyfish lacking turning capability, a barycenter adjustment mechanism for flexible attitude control is introduced. Specially, the barycenter adjustment mechanism is created through both horizontally and vertically altering the relative position of two clump weights. Each clump weight is connected to a step motor by a rocking bar and a gear set. Regulating the center of the gravity (CG) of the barycenter adjustment mechanism will cause change of CG of the overall robotic system over time, allowing 3D spherical change of the barycenter. Reinforcement learning based attitude control. Reinforcement learning relies on trial-and-error mechanism, which has been investigated and utilized in many fields such as artificial intelligence and machine learning. As a type of model-free reinforcement learning, Q-learning can be used to solve the optimal value problems and the optimal policy problems in Markov decision processes (MDP) [7–9]. The goal of Q-learning is to find a policy π that maximizes the reward received by the agent over time. For a policy π : st → at, define the Q value (action-value) as