Join us as first-year electrical engineering doctoral student Matthew Crespo '24 presents "Mathematical Background and Analysis on Temporal Difference Learning for Multi-Agent Reinforcement Learning."
Abstract: To decrease the communication needed for fully decentralized multi-agent reinforcement learning (MARL) policy evaluation, we study the case in which local temporal difference (TD) updates are performed by each agent before carrying out a consensus update between all of the agents. More specifically, we analyze this approach for the case of TD learning in which the average reward is maximized due to wide applicability of average reward TD learning. In this work, we show the theoretical bounds of the local TD learning algorithm function in a time-varying communication network for the convergence of agents to the mean weight vector and the convergence of this mean weight vector to the optimal value. From these bounds, we are able to confirm that the consensus error decreases with more communication rounds, but increases with longer required periods for fully connected communication networks.
Bio: Matthew Crespo is a first-year doctoral student in electrical engineering and an Artificial Intelligence Initiative Fellow at UCF. He recently graduated with his bachelor’s degree in electrical engineering at UCF. Prior to graduation, Matthew was a Caltech WAVE Fellow in summer 2022,
where he worked under Azita Emami, Ph.D. in developing dynamically adjustable contact lenses.
The following summer he worked in developing a communication system for micro-robots under Negar Reiskarimian, Ph.D. and YuFeng Chen, Ph.D. at MIT. Currently, he works under Chinwendu Enyioha, Ph.D. where his research has been focused on TD learning for MARL. His other general interests lie in control and optimization