A comparison between a two feedback control loop and a reinforcement learning algorithm for compliant low-cost series elastic actuators

Abstract

Highly-compliant elastic actuators have become progressively prominent over the last years for a variety of robotic applications. With remarkable shock tolerance, elastic actuators are appropriate for robots operating in unstructured environments. In accordance with this trend, a novel elastic actuator was recently designed by our research group for Serpens, a low-cost, open-source and highly-compliant multi-purpose modular snake robot. To control the newly designed elastic actuators of Serpens, a two-feedback loops position control algorithm was proposed. The inner controller loop is implemented as a model reference adaptive controller (MRAC), while the outer control loop adopts a fuzzy proportional-integral controller (FPIC). The performance of the presented control scheme was demonstrated through simulations. However, the efficiency of the proposed controller is dependent on the initial values of the parameters of the MRAC controller as well as on the effort required for a human to manually construct fuzzy rules. An alternative solution to the problem might consist of using methods that do not assume a priori knowledge: a solution that derives its properties from a machine learning procedure. In this way, the controller would be able to automatically learn the properties of the elastic actuator to be controlled. In this work, a novel controller for the proposed elastic actuator is presented based on the use of an artificial neural network (ANN) that is trained with reinforcement learning. The newly designed control algorithm is extensively compared with the former approach. Simulation results are presented for both methods. The authors seek to achieve a fair, non-biased, risk-aware and trustworthy comparison.