Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online.

nuScenes: A multimodal dataset for autonomous driving

/pdf/nuscenes-a-multimodal-dataset-for-autonomous-driving-3znmpkqyuu.pdf

nuScenes: A Multimodal Dataset for Autonomous Driving

Boss is an autonomous vehicle that uses on-board sensors (global positioning system, lasers, radars, and cameras) to track other vehicles, detect static obstacles, and localize itself relative to a road model. A three-layer planning system combines mission, behavioral, and motion planning to drive in urban environments. The mission planning layer considers which street to take to achieve a mission goal. The behavioral layer determines when to change lanes and precedence at intersections and performs error recovery maneuvers. The motion planning layer selects actions to avoid obstacles while making progress toward local goals. The system was developed from the ground up to address the requirements of the DARPA Urban Challenge using a spiral system development process with a heavy emphasis on regular, regressive system testing. During the National Qualification Event and the 85-km Urban Challenge Final Event, Boss demonstrated some of its capabilities, qualifying first and winning the challenge. © 2008 Wiley Periodicals, Inc.

Autonomous Driving in Urban Environments: Boss and the Urban Challenge.

https://onlinelibrary.wiley.com/doi/pdf/10.1002/rob.20255

Autonomous driving in urban environments: Boss and the Urban Challenge

This paper looks into recent developments and research trends in collision avoidance/warning systems and automation of vehicle longitudinal/lateral control tasks. It is an attempt to provide a bigger picture of the very diverse, detailed and highly multidisciplinary research in this area. Based on diversely selected research, this paper explains the initiatives for automation in different levels of transportation system with a specific emphasis on the vehicle-level automation. Human factor studies and legal issues are analyzed as well as control algorithms. Drivers' comfort and well being, increased safety, and increased highway capacity are among the most important initiatives counted for automation. However, sometimes these are contradictory requirements. Relying on an analytical survey of the published research, we will try to provide a more clear understanding of the impact of automation/warning systems on each of the above-mentioned factors. The discussion of sensory issues requires a dedicated paper due to its broad range and is not addressed in this paper.

Research advances in intelligent collision avoidance and adaptive cruise control

This contribution provides a review of fundamental goals, development and future perspectives of driver assistance systems. Mobility is a fundamental desire of mankind. Virtually any society strives for safe and efficient mobility at low ecological and economic costs. Nevertheless, its technical implementation significantly differs among societies, depending on their culture and their degree of industrialization. A potential evolutionary roadmap for driver assistance systems is discussed. Emerging from systems based on proprioceptive sensors, such as ABS or ESC, we review the progress incented by the use of exteroceptive sensors such as radar, video, or lidar. While the ultimate goal of automated and cooperative traffic still remains a vision of the future, intermediate steps towards that aim can be realized through systems that mitigate or avoid collisions in selected driving situations. Research extends the state-of-the-art in automated driving in urban traffic and in cooperative driving, the latter addressing communication and collaboration between different vehicles, as well as cooperative vehicle operation by its driver and its machine intelligence. These steps are considered important for the interim period, until reliable unsupervised automated driving for all conceivable traffic situations becomes available. The prospective evolution of driver assistance systems will be stimulated by several technological, societal and market trends. The paper closes with a view on current research fields.

https://www.mrt.kit.edu/z/publ/download/2014/BenglerDietmayerFarberMaurerStillerWinner2014ITSM.pdf

Three Decades of Driver Assistance Systems: Review and Future Perspectives

This book takes a look at fully automated, autonomous vehicles and discusses many open questions: How can autonomous vehicles be integrated into the current transportation system with diverse users and human drivers? Where do automated vehicles fall under current legal frameworks? What risks are associated with automation and how will society respond to these risks? How will the marketplace react to automated vehicles and what changes may be necessary for companies? Experts from Germany and the United States define key societal, engineering, and mobility issues related to the automation of vehicles. They discuss the decisions programmers of automated vehicles must make to enable vehicles to perceive their environment, interact with other road users, and choose actions that may have ethical consequences. The authors further identify expectations and concerns that will form the basis for individual and societal acceptance of autonomous driving. While the safety benefits of such vehicles are tremendous, the authors demonstrate that these benefits will only be achieved if vehicles have an appropriate safety concept at the heart of their design. Realizing the potential of automated vehicles to reorganize traffic and transform mobility of people and goods requires similar care in the design of vehicles and networks. By covering all ofthese topics, the book aims to provide a current, comprehensive, and scientifically sound treatment of the emerging field of autonomous driving".

/pdf/autonomous-driving-technical-legal-and-social-aspects-488md5hik9.pdf

Autonomous Driving: Technical, Legal and Social Aspects

In the future, the functions of autonomous driving could fundamentally change all road traffic; to do so, it would have to be implemented on a large scale, in series production.

/pdf/the-release-of-autonomous-vehicles-1ndthhnjrs.pdf

The Release of Autonomous Vehicles

This paper describes focuses on adaptive cruise control (ACC) systems. It first describes the components of ACC systems, including their strengths and weaknesses. A system overview describes the basic structure and operation, control strategy, the driver-ACC interaction concept, and technical limits. Impacts on traffic flow, safety, and driver behavior are discussed. The paper concludes with a discussion on development trends for the next generation of ACC systems.

Adaptive Cruise Control System Aspects and Development Trends

This article provides a general ergonomic framework of cooperative guidance and control for vehicles with an emphasis on the cooperation between a human and a highly automated vehicle. In the twenty-first century, mobility and automation technologies are increasingly fused. In the sky, highly automated aircraft are flying with a high safety record. On the ground, a variety of driver assistance systems are being developed, and highly automated vehicles with increasingly autonomous capabilities are becoming possible. Human-centred automation has paved the way for a better cooperation between automation and humans. How can these highly automated systems be structured so that they can be easily understood, how will they cooperate with the human? The presented research was conducted using the methods of iterative build-up and refinement of framework by triangulation, i.e. by instantiating and testing the framework with at least two derived concepts and prototypes. This article sketches a general, conceptual ergonomic framework of cooperative guidance and control of highly automated vehicles, two concepts derived from the framework, prototypes and pilot data. Cooperation is exemplified in a list of aspects and related to levels of the driving task. With the concept 'Conduct-by-Wire', cooperation happens mainly on the guidance level, where the driver can delegate manoeuvres to the automation with a specialised manoeuvre interface. With H-Mode, a haptic-multimodal interaction with highly automated vehicles based on the H(orse)-Metaphor, cooperation is mainly done on guidance and control with a haptically active interface. Cooperativeness should be a key aspect for future human-automation systems. Especially for highly automated vehicles, cooperative guidance and control is a research direction with already promising concepts and prototypes that should be further explored. The application of the presented approach is every human-machine system that moves and includes high levels of assistance/automation.

Hermann Winner

Papers

Three Decades of Driver Assistance Systems: Review and Future Perspectives

Autonomous Driving: Technical, Legal and Social Aspects

The Release of Autonomous Vehicles

Adaptive Cruise Control System Aspects and Development Trends

Towards cooperative guidance and control of highly automated vehicles: H-Mode and Conduct-by-Wire.