When: Thursday, June 10, 2021, 3pm (CEST).

Where: Zoom online, check the address in the Google Calendar Event.

Topic: Compositional Reinforcement Learning from Logical Specifications.

Speaker: Suguman Bansal, NSF/CRA Computing Innovation Postdoctoral Fellow at the University of Pennsylvania.

Abstract

We study the problem of learning control policies for complex tasks given by logical specifications. Recent approaches automatically generate a reward function from a given specification and use a suitable reinforcement learning algorithm to learn a policy that maximizes the expected reward. These approaches, however, scale poorly to complex tasks that require high-level planning. In this work, we develop a compositional learning approach, called DIRL, that interleaves high-level planning and reinforcement learning. First, DIRL encodes the specification as an abstract graph; intuitively, vertices and edges of the graph correspond to regions of the state space and simpler sub-tasks, respectively. Our approach then incorporates reinforcement learning to learn neural network policies for each edge (sub-task) within a Dijkstra-style planning algorithm to compute a high-level plan in the graph. An evaluation of the proposed approach on a set of challenging control benchmarks with continuous state and action spaces demonstrates that it outperforms state-of-the-art baselines.

This is joint work with Kishor Jothimurugan, Osbert Bastani, and Rajeev Alur.

Short Bio

Suguman Bansal is an NSF/CRA Computing Innovation Postdoctoral Fellow in the Department of Computer and Information Sciences at the University of Pennsylvania. Her research interests lie at the intersection of Artificial Intelligence and Formal Methods. Specifically, she works on developing tools and techniques for the automated synthesis of computational systems that also offer formal guarantees of correctness.

She received her Ph.D. (2020) and M.S. (2016) in Computer Science from Rice University, and B.S. (with Honors) degree ( 2014) in Mathematics and Computer Science from Chennai Mathematical Institute. She is the recipient of the NSF CI Fellowship 2020, Future Faculty Fellowship 2019, EECS Rising Stars 2018, Andrew Ladd Fellowship 2016, and a Gold Medal at the ACM Student Research Competition at POPL 2016.

Material