Foundational Research

Engineering Research Contributions:

CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning

S Yue, G Wang, W Shao, Z Zhang, S Lin, J Ren, J Zhang

Abstract: This work aims to tackle a major challenge in offline Inverse Reinforcement Learning (IRL), namely the reward extrapolation error, where the learned reward function may fail to explain the task correctly and misguide the agent in unseen environments due to the intrinsic covariate shift. Leveraging both expert data and lower-quality diverse data, we devise a principled algorithm (namely CLARE) that solves offline IRL efficiently via integrating "conservatism" into a learned reward function and utilizing an estimated dynamics model. Our theoretical analysis provides an upper bound on the return gap between the learned policy and the expert policy, based on which we characterize the impact of covariate shift by examining subtle two-tier tradeoffs between the exploitation (on both expert and diverse data) and exploration (on the estimated dynamics model). We show that CLARE can provably alleviate the reward extrapolation error by striking the right exploitation-exploration balance therein. Extensive experiments corroborate the significant performance gains of CLARE over existing state-of-the-art algorithms on MuJoCo continuous control tasks (especially with a small offline dataset), and the learned reward is highly instructive for further learning.

Systems and methods for social-aware cooperative device-to-device communications

(US patent approved Aug 2023, Publication of US11425725B2)

Abstract: A plurality of mobile devices is provided including a first mobile device of the plurality of mobile devices that serves as a relay for communication between a second mobile device of the plurality of mobile devices and a third mobile device of the plurality of mobile devices. The first mobile device is identified as the relay based on a social trust relationship formed between the first mobile device and the second mobile device. A D2D network is formed between the first mobile device and second mobile device. A communication channel is provided to the second mobile device from a second cellular network. A portion of the communication channel is allocated to the second mobile device using the D2D network to improve cellular transmission between the second mobile device and the third mobile device.

Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing

Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo and J. Zhang

Abstract:

With the breakthroughs in deep learning, the recent years have witnessed a booming of artificial intelligence (AI) applications and services, spanning from personal assistant to recommendation systems to video/audio surveillance. More recently, with the proliferation of mobile computing and Internet of Things (IoT), billions of mobile and IoT devices are connected to the Internet, generating zillions bytes of data at the network edge. Driving by this trend, there is an urgent need to push the AI frontiers to the network edge so as to fully unleash the potential of the edge big data. To meet this demand, edge computing, an emerging paradigm that pushes computing tasks and services from the network core to the network edge, has been widely recognized as a promising solution. The resulted new interdiscipline, edge AI or edge intelligence (EI), is beginning to receive a tremendous amount of interest. However, research on EI is still in its infancy stage, and a dedicated venue for exchanging the recent advances of EI is highly desired by both the computer system and AI communities. To this end, we conduct a comprehensive survey of the recent research efforts on EI. Specifically, we first review the background and motivation for AI running at the network edge. We then provide an overview of the overarching architectures, frameworks, and emerging key technologies for deep learning model toward training/inference at the network edge. Finally, we discuss future research opportunities on EI. We believe that this survey will elicit escalating attentions, stimulate fruitful discussions, and inspire further research ideas on EI.

Public Policy Analysis: Recent Research Contributions:

California Automated Vehicle Policy Strategies (2021)

Authors: Mollie Cohen D’Agostino, Jerel Francisco, Susan A. Shaheen, and Daniel Sperling

Senate Bill (SB) 1298 granted the California Department of Motor Vehicles (DMV) a legislative mandate to develop the Autonomous Vehicle Program. In this landmark 2012 bill, the DMV is allowed to “consult with the [California Highway Patrol (CHP)], Institute of Transportation Studies at the University of California, or any other entity DMV identifies that has expertise in automotive technology, automotive safety, and autonomous system design.”2This issue paper is offered to the State of California in the spirit of this consultation privilege. This research is also a project component of the Climate Smart Transportation and Communities Consortium (C-STACC) for the Strategic Growth Council (SGC) (Task 3.4.4). A review draft of this issue paper was submitted to the California State Transportation Agency (CalSTA) in March 2021 in response to their solicitation for feedback of the draft Automated Vehicle (AV) Framework. This paper leverages the eight principles for AV policy included in the draft CalSTA framework...

Technology is Outpacing State Automated Vehicle Policy (2019)

Author: Kelly L. Fleming; Contributors: Austin Brown, Mollie D’Agostino, Tatjana Kunz, Hannah Safford, Yoon Jae Annie Lee

Existing statutes related to Automated Vehicles (AVs) tend to be preliminary in scope. This paper creates a scale for evaluating AV policy: from most permissive to most restrictive. Our findings are that AV policy among states varies considerably, but many policies are in the middle of the road, and many states have limited legislative actions to codifying definitions or establishing exploratory committees. This assessment points to the fact that states are readying to take more decisive action. Therefore, it is critical to identify some possible best practices for AV policy development as states explore the topic. Our analysis points to guidelines for developing safe, equitable and sustainable AV policy.

Mobility Data Sharing: Challenges and Policy Recommendations (2019)

Authors: Mollie D’Agostino, Paige Pellaton, and Austin Brown; Contributors: Hannah Safford, Kelly Fleming, and Cassidy Craford

Massive amounts of transportation data are generated every day. These data can support transportation planning, policy, and research— especially when it comes to emerging mobility options such as scootersharing, bikesharing, and ridehailing. However, there are not yet well-established mechanisms for sharing mobility data. New policy frameworks are needed to streamline and expand mobility data sharing while respecting privacy and proprietary concerns. Frameworks that achieve these goals must consider how best to (1) standardize, (2) share, (3) securely store, and (4) analyze and apply mobility data. This brief summarizes insights from the UC Davis issue paper “Mobility Data Sharing: Challenges and Policy Recommendations”, which addresses each of the above components....For a concise summary see the 2-page Policy Brief.