Foundational Research

Engineering Research Contributions:

CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning

S Yue, G Wang, W Shao, Z Zhang, S Lin, J Ren, J Zhang

Abstract: This work aims to tackle a major challenge in offline Inverse Reinforcement Learning (IRL), namely the reward extrapolation error, where the learned reward function may fail to explain the task correctly and misguide the agent in unseen environments due to the intrinsic covariate shift. Leveraging both expert data and lower-quality diverse data, we devise a principled algorithm (namely CLARE) that solves offline IRL efficiently via integrating "conservatism" into a learned reward function and utilizing an estimated dynamics model. Our theoretical analysis provides an upper bound on the return gap between the learned policy and the expert policy, based on which we characterize the impact of covariate shift by examining subtle two-tier tradeoffs between the exploitation (on both expert and diverse data) and exploration (on the estimated dynamics model). We show that CLARE can provably alleviate the reward extrapolation error by striking the right exploitation-exploration balance therein. Extensive experiments corroborate the significant performance gains of CLARE over existing state-of-the-art algorithms on MuJoCo continuous control tasks (especially with a small offline dataset), and the learned reward is highly instructive for further learning.

Systems and methods for social-aware cooperative device-to-device communications

(US patent approved Aug 2023, Publication of US11425725B2)

Abstract: A plurality of mobile devices is provided including a first mobile device of the plurality of mobile devices that serves as a relay for communication between a second mobile device of the plurality of mobile devices and a third mobile device of the plurality of mobile devices. The first mobile device is identified as the relay based on a social trust relationship formed between the first mobile device and the second mobile device. A D2D network is formed between the first mobile device and second mobile device. A communication channel is provided to the second mobile device from a second cellular network. A portion of the communication channel is allocated to the second mobile device using the D2D network to improve cellular transmission between the second mobile device and the third mobile device.

Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing

Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo and J. Zhang

Abstract:

With the breakthroughs in deep learning, the recent years have witnessed a booming of artificial intelligence (AI) applications and services, spanning from personal assistant to recommendation systems to video/audio surveillance. More recently, with the proliferation of mobile computing and Internet of Things (IoT), billions of mobile and IoT devices are connected to the Internet, generating zillions bytes of data at the network edge. Driving by this trend, there is an urgent need to push the AI frontiers to the network edge so as to fully unleash the potential of the edge big data. To meet this demand, edge computing, an emerging paradigm that pushes computing tasks and services from the network core to the network edge, has been widely recognized as a promising solution. The resulted new interdiscipline, edge AI or edge intelligence (EI), is beginning to receive a tremendous amount of interest. However, research on EI is still in its infancy stage, and a dedicated venue for exchanging the recent advances of EI is highly desired by both the computer system and AI communities. To this end, we conduct a comprehensive survey of the recent research efforts on EI. Specifically, we first review the background and motivation for AI running at the network edge. We then provide an overview of the overarching architectures, frameworks, and emerging key technologies for deep learning model toward training/inference at the network edge. Finally, we discuss future research opportunities on EI. We believe that this survey will elicit escalating attentions, stimulate fruitful discussions, and inspire further research ideas on EI.

Public Policy Analysis: Recent Research Contributions:

A Blueprint for Improving Automated Driving System Safety (2024)

Authors: Mollie D’Agostino, Michael Cooper, Marilia Ramos

Vehicle automation represents a new safety frontier that may necessitate a repositioning of our safety oversight systems. This white paper serves as a primer on the technical and legal landscape of automated driving system (ADS) safety. It introduces the latest AI and machine learning techniques that enable ADS functionality. The paper also explores the definitions of safety from the perspectives of standards-setting organizations, federal and state regulations, and legal disciplines. The paper identifies key policy options building on topics raised in the White House’s Blueprint for an AI Bill of Rights, outlining a Blueprint for ADS safety. The analysis concludes that potential ADS safety reforms might include either reform of the Federal Motor Vehicle Safety Standards (FMVSS), or a more holistic risk analysis “safety case” approach. The analysis also looks at caselaw on liability in robotics, as well as judicial activity on consumer and commercial privacy, recognizing that the era of AI will reshape liability frameworks, and data collection must carefully consider how to build in accountability and protect the privacy of consumers and organizations. Lastly, this analysis highlights the need for policies addressing human-machine interaction issues, focusing on guidelines for safety drivers and remote operators. In conclusion, this paper reflects on the need for collaboration among engineers, policy experts, and legal scholars to develop a comprehensive Blueprint for ADS safety and highlights opportunities for future research.

Experiences with Autonomous Vehicles in U.S. Cities (2024)

Authors: Mollie D’Agostino, Michael Cooper, Marilia Ramos, Venkataram, Prashanth S., PhD

This project convened a series of meetings and workshops to prioritize listening to multi-sector stakeholders from local government, advocacy, and industry in US cities where autonomous vehicles are operating. The objective was to listen and learn from all stakeholders, raise issues surrounding accessibility and equity, and to solicit responses. Key findings from the workshops include a consensus across the three sectors on the need for good channels of multi-stakeholder communication, and voices across all sectors agreed on the importance of disability access and serving diverse populations. Many parties, representing voices from all sectors, recognized that federal regulatory activities appear to be moving too slowly. Preventing any roadway incidents is a priority for many stakeholders, and some suggest a playbook for handling day-to-day roadway issues and common standards for first-responder interactions. Disability access is a high priority across all sectors, and there many see nearer term to accommodations for blind, hearing-impaired riders, but the timeline for providing service to people with non-folding wheelchairs is less clear. There is also ongoing debate surrounding the limits of regulatory purview, the role for cities, and how to actualize equitable expansion into rural areas. There is more work to do to advance a multi-sector dialogue around the role for local governments and community-based organizations in shepherding a safe, equitable and sustainable expansion of autonomous vehicles.

California Automated Vehicle Policy Strategies (2021)

Authors: Mollie Cohen D’Agostino, Jerel Francisco, Susan A. Shaheen, and Daniel Sperling

Senate Bill (SB) 1298 granted the California Department of Motor Vehicles (DMV) a legislative mandate to develop the Autonomous Vehicle Program. In this landmark 2012 bill, the DMV is allowed to “consult with the [California Highway Patrol (CHP)], Institute of Transportation Studies at the University of California, or any other entity DMV identifies that has expertise in automotive technology, automotive safety, and autonomous system design.”2This issue paper is offered to the State of California in the spirit of this consultation privilege. This research is also a project component of the Climate Smart Transportation and Communities Consortium (C-STACC) for the Strategic Growth Council (SGC) (Task 3.4.4). A review draft of this issue paper was submitted to the California State Transportation Agency (CalSTA) in March 2021 in response to their solicitation for feedback of the draft Automated Vehicle (AV) Framework. This paper leverages the eight principles for AV policy included in the draft CalSTA framework...

Technology is Outpacing State Automated Vehicle Policy (2019)

Author: Kelly L. Fleming; Contributors: Austin Brown, Mollie D’Agostino, Tatjana Kunz, Hannah Safford, Yoon Jae Annie Lee

Existing statutes related to Automated Vehicles (AVs) tend to be preliminary in scope. This paper creates a scale for evaluating AV policy: from most permissive to most restrictive. Our findings are that AV policy among states varies considerably, but many policies are in the middle of the road, and many states have limited legislative actions to codifying definitions or establishing exploratory committees. This assessment points to the fact that states are readying to take more decisive action. Therefore, it is critical to identify some possible best practices for AV policy development as states explore the topic. Our analysis points to guidelines for developing safe, equitable and sustainable AV policy.

Mobility Data Sharing: Challenges and Policy Recommendations (2019)

Authors: Mollie D’Agostino, Paige Pellaton, and Austin Brown; Contributors: Hannah Safford, Kelly Fleming, and Cassidy Craford

Massive amounts of transportation data are generated every day. These data can support transportation planning, policy, and research— especially when it comes to emerging mobility options such as scootersharing, bikesharing, and ridehailing. However, there are not yet well-established mechanisms for sharing mobility data. New policy frameworks are needed to streamline and expand mobility data sharing while respecting privacy and proprietary concerns. Frameworks that achieve these goals must consider how best to (1) standardize, (2) share, (3) securely store, and (4) analyze and apply mobility data. This brief summarizes insights from the UC Davis issue paper “Mobility Data Sharing: Challenges and Policy Recommendations”, which addresses each of the above components....For a concise summary see the 2-page Policy Brief.