ICCV 2021 - Workshop on "Embedded and Real-World Computer Vision in Autonomous Driving”
11th October 2021 - 13.00-18.00 CEST - Virtual/Worldwide
|13:00 – 13:15|
Introduction & Workshop Structure
Dr.-Ing. Syed Saqib Bukhari (ZF Group) & Julian Wiederer (Mercedes-Benz)
|13:15 – 14:45|
Session 1 Embedded AI – Keynote Talks
Moderator: Dr.-Ing. Syed Saqib Bukhari (ZF Group)
Keynote 1 Michael Keckeisen (ZF Group, Autonomous Mobility Systems): “ZF ProAI - High Performance Computing Power and Artificial Intelligence for Mobility”
Keynote 2 Mark Grobman (Hailo): “Quantization at Hailo – Co-Evolution of Hardware, Software and Algorithms”
Keynote 3 Mingxing Tan (Google Brain): “AutoML for Efficient Computer Vision Learning”
|14:45 – 15:00||Break|
|15:00 – 15:45|
Session 2 Embedded AI – Paper Presentations
Moderator: Dr.-Ing. Domenik Helms (OFFIS)
Paper 1 Julia Hornauer (Ulm University), Lazaros Nalpantidis (DTU - Technical University of Denmark), Vasileios Belagiannis (Universität Ulm), “Visual Domain Adaptation for Monocular Depth Estimation on Resource-Constrained Hardware”
Paper 2 Sven Mantowsky, Falk Heuer, Saqib Bukhari, Michael Keckeisen, Georg Schneider (ZF Friedrichshafen AG), “ProAI: An Efficient Embedded AI Hardware for Automotive Applications – A Benchmark Study”
Paper 3 Aditya Rajagopal, Christos-Savvas Bouganis (Imperial College London), “perf4sight: A toolflow to model CNN training performance on Edge GPUs”
|15:45 – 16:15|
Session 3 Poster Presentations
Moderator: Julian Wiederer (Mercedes-Benz)
Poster 1 Deepthi Sreenivasaiah (Merantix Labs GmbH), Johannes S Otterbach (Merantix Labs GmbH), Thomas Wollmann (Bioquant Heidelberg), “MEAL: Manifold Embedding-based Active Learning”
Poster 2 Sujan Gannamaneni, Sebastian Houben, Maram Akila (Fraunhofer IAIS), “Semantic Concept Testing in Autonomous Driving by Extraction of Object-Level Annotations from CARLA”
Poster 3 Lukas Staecker (Stellantis, Opel Automobile GmbH, DFKI), Juncong Fei (Stellantis, Opel Automobile GmbH, Karlsruhe Institute of Technology), Philipp Heidenreich ((Stellantis, Opel Automobile GmbH), Frank Bonarens ((Stellantis, Opel Automobile Gmbh), Jason Rambach (DFKI), Didier Stricker (DFKI), Christoph Stiller (Institute of Measurement and Control Systems, Karlsruhe Institute of Technology), “Deployment of Deep Neural Networks for Object Detection on Edge AI Devices with Runtime Optimization”
Poster 4 Daniel Bogdoll (FZI), Jasmin Breitenstein (Technische Universität Braunschweig), Florian Heidecker (University of Kassel), Maarten Bieshaar (University of Kassel), Bernhard Sick (University of Kassel), Tim Fingscheidt (Technische Universität Braunschweig), Marius Zöllner (FZI), “Description of Corner Cases in Automated Driving: Goals and Challenge”
Poster 5 Matthias Reuse (Valeo), Martin Simon (Valeo), Bernhard Sick (University of Kassel), “About the Ambiguity of Augmentation for 3D Object Detection in Autonomous Driving”
|16:15 – 16:30||Break|
|16:30 – 17:45|
Session 4 Real-World Computer Vision – Keynote Talks & Paper Presentations
Moderators: Matthias Rottmann (University of Wuppertal) & Tobias Kalb (Porsche)
Keynote 4 Gabriela Csurka (NAVER Labs Europe): “Visual DA in Autonomous Driving scenarios”
Paper 4 Falk Heuer, Sven Mantowsky, Saqib Bukhari, Georg Schneider (ZF Friedrichshafen AG), “MultiTask-CenterNet (MCN): Efficient and Diverse Multitask Learning using an Anchor Free Approach”
Paper 5 Florentin Poucin (Ecole des Ponts ParisTech), Andrea Kraus (Valeo), Martin Simon (Valeo), “Boosting Instance Segmentation with Synthetic Data: A study to overcome the limits of real-world data sets”
Paper 6 Maria Lyssenko (Robert Bosch GmbH), Christoph D Gladisch (Robert Bosch GmbH), Christian Heinzemann (Robert Bosch GmbH), Matthias Woehrle (Robert Bosch GmbH), Rudolph Triebel (DLR) , “Instance Segmentation in CARLA: Methodology and Analysis for Pedestrian-oriented Synthetic Data Generation in Crowded Scenes”
|17:45 – 18:00|
Closing Remarks (Organizers)
About this workshop:
Autonomous driving (AD) technology holds one of the most promising and most prominent business cases as well as one of the scientifically most challenging applications of computer vision. In recent years a significant research community has evolved around the topic which is also reflected in a great number of science-driven start-ups and major investments in the existing automotive industry.
Nevertheless, research in the field of AD has so far been mainly methodically driven – with the underlying assumption of theoretically unlimited resources. But as the technology moves closer to serial production and its use in the real-world, it becomes more critical to deploy models on safety-certified automotive hardware with real-time operating systems. Due to this resource constraint, research questions in AD need to be reformulated. It is crucial to leave the sole focus on machine learning performance metrics and discuss latency performance trade-offs on hardware as well as to develop tools and methods for better assessment of AI hardware in terms of performance and safety. On top of that, entering the real-world demands for handling rare and unknown situations which cannot be captured in a lab environment.
The aim of this ERCVAD workshop is to offer the community a place to discuss these trends in the automotive industry. By having organizers from the high-throughput automotive industry (e.g. Mercedes-Benz, Porsche, ZF), we open the discussion as traditional players in the automotive market. Within the venue of ICCV this round is widened to include world-class researchers as well as evolving start-ups in the automotive community.
Mingxing Tan (Google Brain)
“AutoML for Efficient Computer Vision Learning”
Abstract: This talk will focus on neural architecture search (NAS) for efficient computer vision learning. We will discuss the challenges and solutions in designing NAS search spaces, search algorithms, and hardware constraints. Afterwards, we will explore how to scale up and adapt neural networks through hardware-software co-design. We will conclude the talk with some representative AutoML applications on image classification, object detection, segmentation, and autonomous driving.
Bio: Mingxing Tan is a research scientist at Google Brain, mainly focusing on AutoML Research and Computer Vision. He has co-authored several popular models including EfficientNet/EfficientNetV2/EfficientDet/MobileNetV3. Prior to joining Google, he finished his Ph.D. at Peking University, China, and post-doc research at Cornell University.
Gabriela Csurka (NAVER Labs Europe)
"Visual DA in Autonomous Driving scenarios"
Abstract: Domain adaptation (DA) is one of the solutions proposed in the literature to overcome the burden of annotation, which is often critical when working with deep neural networks. The main idea is to exploit labeled data or trained models available in related source domains together with unlabeled data from the target domain. The aim of this talk will be to give an overview of visual domain adaptation, a field whose popularity in the computer vision community has increased significantly in the last few years. First, I will discuss shortly about different DA strategies to exploit deep architectures for visual recognition. Second, I will briefly present a set of deep learning-based trends in the literature to handle domain shift in visual tasks relevant in autonomous driving scenarios, such as image classification, object detection and semantic segmentation. I will conclude the talk by integrating visual DA in a larger transfer learning landscape giving some general thoughts about future perspectives in the field.
Bio: Gabriela Csurka is a Principal Scientist at NAVER LABS Europe, France. Her main research interests are in computer vision for image understanding, multi-view 3D reconstruction, visual localization, multi-modal information retrieval as well as domain adaptation and transfer learning. She has contributed to around 100 scientific communications, several on the topic of DA. She has given several invited talks and organized a tutorial on domain adaptation at ECCV’20. In 2017, she edited a book on Domain Adaptation for Computer Vision Applications.
Michael Keckeisen (ZF Autonomous Mobility Systems)
"ZF ProAI - High Performance Computing Power and Artificial Intelligence for Mobility"
Abstract: The most flexible, scalable, and powerful automotive supercomputer and the source of vehicle intelligence. ZF ProAI is a central computer suitable for all vehicle platforms, software applications and E/E-architectures. Based on ZFs open and modular approach the board can operate ZF’s own application and safety software – or that of other developers or third-party suppliers.
Bio: Michael Keckeisen is Director of the ProAI product line in the Autonomous Mobility Systems division of ZF. Since April 2000 he has been employed at ZF Friedrichshafen AG in the electronics development department (chassis, powertrain, HMI and ADAS / AD). Michael Keckeisen holds a degree in electronics engineering from the University of Ravensburg-Weingarten, Germany.
Mark Grobman (Hailo)
Quantization at Hailo – Co-Evolution of Hardware, Software and Algorithms
Abstract: Coming Soon
Bio: Mark Grobman is the ML CTO at Hailo, a startup offering a uniquely designed microprocessor for accelerating embedded AI applications on edge devices. Mark has been with Hailo since it was founded in 2017 and has overseen the ML R&D in the company. Before joining Hailo, Mark served at the Israeli Intelligence Corps Technology Unit in various interdisciplinary R&D roles. Mark holds a double B.Sc. in Physics and Electrical Engineering from the Technion and a M.Sc. in Neuroscience from the Gonda Multidisciplinary Brain Research Center at Bar-Ilan University, Israel.
Call for Papers
The workshop addresses real-world deployment and embedded AI. We are soliciting high-quality papers covering the topics listed below. Papers should follow the standard ICCV formatting instructions. Paper length should be 4 to 8 pages according to the CVPR format. Accepted papers will appear in the ICCV workshop proceedings.
In real-world deployment, this workshop will focus on the following topics:
- Safety: handling the unknown in form of domain shifts, unknown objects, road anomalies, rare events
- Dataset engineering: corner cases, frequency awareness
- Advanced training methods: active learning, semi-supervised and transfer learning, training for robustness
- Hybridization: AI models + classical physics-based models / direct methods
- Synthetic data for real world automotive AI perception: synthetic corner cases, impact on real world robustness, synthetic data for testing real world AI systems
In terms of embedded AI, the following subtopics will be covered:
- Model transformation techniques and trading of functional model performance for embedded suitability, such as pruning and quantization techniques, tensor compression and similar mathematical transformations as well as teacher-student approaches with embedded AI focus
- Determination of relevant metrics such as runtime, power or memory demand, based on hardware in the loop measurements, simulation techniques or model building techniques
- Applying electronic design automation (EDA) techniques to improve embedded AI deployment
- Tools and methods helping to assess and select embedded hardware components
- Paper Submission Deadline: 18th July 2021, Anywhere on Earth (UTC-12)
- Notification: 3rd August 2021, Anywhere on Earth (UTC-12)
- Camera-Ready Version: 17th August 2021, Anywhere on Earth (UTC-12)
We are organizing this workshop as part of the research project "KI Delta Learning" (https://ki-deltalearning.de/), funded by the German Federal Ministry for Economic Affairs and Energy (BMWi) on the basis of a decision by the German Bundestag. It is a part of the project family KI Familie of the VDA Leitinitiative Autonomous and Connected Driving. The project brings together 18 partners from industry (including OEMs, suppliers and technology providers) and the research community. Our organising committee represents this diverse consortium.
Cristóbal CURIO: Prof. Cristóbal Curio holds the chair for Cognitive Systems since 2014 in the Department of Computer Science at Reutlingen University, Germany. Before, he led the cognitive engineering research group for almost a decade at the Max Planck Institute for Biological Cybernetics in Tuebingen, Germany. His expertise includes computer vision for dynamic scene analysis and the fusion of human and machine vision. He is working on the transfer of pose recognition methods into the human-centered autonomous urban driving range. Many of his work processes are based on a Motion Capture Laboratory, which allows the production and simulation of automotive relevant sensor data. At international conferences, he organized among others in cooperation with HELLA GmbH & Co. KGaA at international IEEE IV and IEEE ITSC conferences workshops on sensor interoperability, sensor interfaces and people in autonomous driving environments.
Syed Saqib BUKHARI: Dr. Syed Saqib Bukhari received his PhD in computer science (artificial intelligence) from the University of Kaiserslautern, Germany, in 2012. Upon completion of his doctorate degree he worked in industry for two years in the field of computer vision, image processing and pattern recognition. In 2015 he joined the German Research Center for Artificial Intelligence (DFKI) as a senior researcher. Since September 2019 he has been working on autonomous driving in the ZF Artificial Intelligence Lab in Saarbrücken, Germany. He co-authored more than 80 international publications in international peer-reviewed conferences, journals, and book-chapters on the topic of artificial intelligence. He received two best papers awards and served as technical program committee member for well-known conferences in computer vision, image processing and machine learning.
Frank HAFNER: Frank Hafner has been working for several years as perception engineer at the intersection of machine learning research and embedded deployment at ZF. Previously, he had research stays at TU Delft and ETS Montreal.
Domenik HELMS: Dr. Domenik Helms is a theoretical physicist working on microelectronics and technical computer science. He has 20 years of experience in embedded system design.
Tobias KALB: Tobias Kalb is a PhD Student at Porsche Engineering in corporation with the Karlsruhe Institute of Technology (KIT) in Germany. His research interests are continual learning methods for deep architectures for automotive applications.
Matthias ROTTMANN: Matthias Rottmann is a stat scientist at the University of Wuppertal, Germany, working on the interface of applied mathematics and machine learning. He is involved in several projects with application to automated driving that deal with safety & security, handling unlearned situations and advanced learning methods. He is guiding a group of 10+ researchers and has more than 10 years of experience in data-driven method development.
Georg SCHNEIDER: Dr. Georg Schneider received his diploma in Physics at the TU Darmstadt, Germany, in 1999. After a research stay at the University of California San Diego UCSD, USA, and at the University of Stuttgart, Germany, he joined the Honda Research Institute Europe in Offenbach, Germany, where he worked on his PhD theses with the topic: “Evolutionary optimization of a biologically inspired visual object recognition system”. Then he joined Audi in Ingolstadt, Germany, where we worked in the advanced development and in the serial development of the front looking ADAS camera. In 2008 he joined ZF (at this time TRW) in Koblenz, Germany, where he developed three generations of ADAS camera functions. Since September 2019 he is responsible for the ZF Artificial Intelligence Lab in Saarbrücken, Germany.
Julian WIEDERER: Julian Wiederer is a PhD candidate affiliated with University Ulm, Germany, and the Mercedes-Benz environment perception team. He applies novel deep learning and computer vision algorithms to solve current challenges in autonomous driving. His current focus is human behavior and interaction modelling in open world scenarios.
The submitted papers must be an unpublished work. Papers will undergo full double-blind peer review by program committee members. There is neither a rebuttal nor a second review cycle. The papers will be published in conjunction with ICCV 2021 proceedings.
- Please submit your work using the CMT website: https://cmt3.research.microsoft.com/ERCVAD2021/
- For formatting instructions and LaTeX templates please refer to the ICCV submission guidelines: http://iccv2021.thecvf.com/node/4#call-for-papers
Dr. Syed Saqib Bukhari
Headerbild: ZF Friedrichshafen AG
Bild Real word employment: ZF Friedrichshafen AG
Bild Embedded AI: ZF Friedrichshafen AG
Portraits: Cristóbal Curio (Curio); Syed Saqib Bukhari (Bukhari); Frank Hafner (Hafner); Domenik Helms (Helms); Tobias Kalb (Kalb); Matthias Rottmann (Rottmann); Georg Scheider (Schneider); Julian Wiederer (Wiederer)