Title: Neural Representations for Sequential Robotic Manipulation
Abstract: Autonomous robots are expected to make sequential decisions, plan their motions, and control their movements to realize their tasks. This remarkable skill set requires a new research direction where perception, decision-making, motion planning, control, and learning methods are considered jointly to provide autonomy to the agent while physically interacting with the world.
In this talk, I will mainly present one of our recent steps toward tackling this challenge. Specifically, I will focus on learning representations from visual perception data for Task and Motion Planning (TAMP) problems. TAMP addresses the problem of finding a sequence of actions and motions which minimizes the cost of a predefined objective function. To formulate such an objective function a good representation of the environment, in which the robot operates, is required. Representing the dynamic behavior of articulated objects is particularly challenging. We show that embeddings of articulated objects that enable us to render RGB images from different viewpoints can be used to predict the position of certain object specific keypoints. By splitting the embedding into an object and articulation code we can generate representations for unobserved articulations. This allows us to render images, generate semantically labeled point clouds and estimate the position of keypoints not only from different views but also for arbitrary articulations. We also show that our framework which was trained only on synthetic data is able to compute good predictions from real images. Finally, we demonstrate our approach of formulating TAMP using NSR to open a closet and place an object inside it.
To provide a broader perspective, I will also highlight some key achievements of our previous work on (i) learning explainable and effective representations directly from visual perception data, (ii) task decompositions and robust motion planning algorithms for long-horizon tasks, and (iii) (safe) learning for control of autonomous robots in the real-world.
Bio: Ozgur S. Oguz is an Assistant Professor in the Computer Engineering Department and the head of the Learning for Intelligent Robotic Agents (LiRA) lab at Bilkent University. He studied computer science at Koç University, Turkiye and University of British Columbia, Canada, and received his Ph.D. degree (summa cum laude) from Technical University of Munich, Germany. He was a Postdoctoral Researcher at TU Munich, Max Planck Institute for Intelligent Systems, and lastly at TU Berlin with the Learning and Intelligent Systems Lab. His research interests include developing autonomous systems that are able to reason about their states of knowledge, make sequential decisions to realize a goal, and simultaneously learn to improve their causal physical reasoning and manipulation skills.