Mid-term Presentation

Work Streams

deep dives in four parallel sessions

Data Acquisition

In order to be able to investigate a large number of deltas from different domains, special data is required that is tailored to the requirements of the project. New data sets with different sensors will be developed for this project, which will be recorded, generated and labelled from various traffic domains and different environmental conditions.

Talks:

  • Recording Vehicle and Data Management, Roshan Muthaiya, CMORE. 
    The large amounts of collected and generated data need a platform to facilitate data management and processing. End users need insight into the content to search and navigate through the data workflow to further process the data. The next generation C.DATA solution is a modular application that runs in a browser and allows the user to manage data as well as related workflows like fully automated annotation and anonymization workflow for the recorded video streams.

  • Synchronized Simulation and Reality for Controllable Domain Shift, Michael Essich, Reutlingen University.
    Reutlingen University will give a brief introduction to the relevance of synthetic data and provide insights into their motion capture based data generation. This includes data acquisition from real world sensors and synchronization to a motion capture system for automated ground truth generation for the task of human pose estimation. Paired synthetic and real data can be generated by synchronizing the simulation to the real world data.

  • Simulating Deltas using CARLA, Thies de Graaff, Offis.
    Simulations significantly ease the data elicitation process since the desired conditions can be easily set up. In addition to that, labels are provided for free and do not need to be annotated by hand. Overall, this results in an acceleration of the algorithm research but synthetic data may also increase the performance of models on real data. In this talk, we present how the partners in KI Delta Learning are using the open driving simulator CARLA in order to generate synthetic data for the different deltas in the project.

  • Delta Labeling - Towards the KIDL Dataset, Jörg P. Schäfer, DLR.
    Many datasets have been created in the area of autonomous driving, but none of them satisfy the demands in the area of domain adaptation. Creating such a dataset is a challenging task. It requires a certain number of labeled samples as well as diversity regarding data types and tasks. This talk names relevant aspects and discusses two technical problems. On the one hand, only a subset will be labeled manually. The question at hand is how to select these samples. On the other hand, the rest of the dataset may be labeled (semi-) automatically.

 

Transfer Learning

The aim of the project is to develop methods that allow AI systems to deal with changing conditions efficiently - without necessary changes in the architecture and without requiring a completely new training. The approach of continuous learning is applied: Here, the learned knowledge is not forgotten when transferring from one domain to another. Furthermore, methods are developed that can handle several domains at the same time.

Talks:

  • Survey on Domain Adaption for Visual Perception in Automated Driving, M. Schwonberg, CARIAD; J. Niemeyer, DLR; Jan-Aike Termöhlen, TU Braunschweig.
    Domain Adaptation methods offer a way to deal with several domain shifts like different sensors, weather conditions, locations or the synthetic to real shift. We will give an overview over visual unsupervised domain adaptation methods for automotive applications. A taxonomy of the existing methods is defined and a brief introduction into the methods will be done. The highlight is a quantitative meta analysis of the research status and preliminary conclusions for further research. 

  • Continual Unsupervised Domain Adaptation for Semantic Segmentation by Online Frequency Domain Style Transfer, Jan-Aike Termöhlen, TU Braunschweig.
    We present a way to perform an online style transfer for continual domain adaptation which improves performance on unseen target domains using a given perception model. The approach is based on an image style transfer in the frequency domain and requires neither an adjustment of the given model parameters to the target domain, nor does it require any considerable amount of memory for storing its frequency domain, considering the hardware limitations in an autonomous vehicle.

  • Improving robustness against common corruptions with frequency biased models, Tonmoy Saikia, Uni Freiburg.
    We introduce a mixture of two expert models specializing in high and low-frequency robustness, respectively. Moreover, we propose a new regularization scheme that minimizes the total variation (TV) of convolution feature-maps to increase high-frequency robustness. The approach improves on corrupted images without degrading in-distribution performance. We demonstrate this on ImageNet-C and also for real-world corruptions on an automotive dataset, both for object classification and object detection.

  • Learning Visual Models using a Knowledge Graph as a Trainer, Sebastian Monka, Lavdim Halilaj, Bosch.
    We evaluate knowledge graph neural networks on different datasets. The results show that a visual model trained with a knowledge graph as a trainer outperforms a model trained with cross-entropy in all experiments, in particular when the domain gap increases. Besides a better performance and a stronger robustness to domain shifts, these networks can simultaneously adapt to multiple datasets and classes without heavily suffering from catastrophic forgetting.

 

Didactics

For automotive applications, the supervised training of neural networks has several disadvantages that can be categorized into two core problems: enormous consumption of time and resources for both training and data labeling. It makes sense from both an economic and ecological point of view to optimize the training process.

In this session one will have the opportunity to get an insight into our workstream through the following presentations:

  • Active Deep Learning - A challenging Journey, Jörg P. Schäfer, DLR.
    Deep Learning requires a huge amount of training data but labeled data is expensive. Unlabeled data is available en masse. How can we reduce the labeling costs by choosing the best (sub-)set for labeling?

  • Unsupervised Domain Adaption in Semantic Segmentation, Joshua Niemeijer, DLR.
    How to combine Semantic Self-Supervision and Self-Training for Domain Adaptation in Semantic Segmentation? A two-staged, unsupervised domain adaptation process for semantic segmentation models by combining a self-training and self-supervision strategy.

  • Learnable Duplicate Removal using Graph Neural Network for Single Stage Object Detector, Shuxiao Ding, Mercedes-Benz.
    A novel learning duplicate removal network which can be easily embedded into Single Stage Detection models and trained end-to-end. Consequently, our work removes the last hand engineered component to form an entirely learned object detection pipeline within realtime constraints.

Furthermore, you will get the chance to listen to an external contribution from our partner project “KI-Absicherung”:

  • Schulik Thomas, ZF and Frederik Blank, Bosch will talk on Ontology- and scenario-based testing, which is illustrated using NCAP-like scenarios and a test methodology.

 

Automotive Suitability

Two specific areas will be examined in more detail for the proper application of AI modules, where current AI procedures show weaknesses but are crucial for the success of highly automated vehicles: the robustness in an open world with countless combinations of scenarios and the challenges of operating AI on embedded systems with limited hardware resources while taking into account boundary requirements.

Talks:

  • Is In-Domain Data Required to Learn Multi-View Depth Estimation? Philipp Schröppel, University of Freiburg. 
    Depth estimation is an important component of autonomous driving and should function robustly in case of domain shifts or previously unseen objects. We evaluate existing methods regarding cross-domain performance and present a method that is trained only on randomized synthetic data and functions better in this setting, as it rather relies on the motion parallax between multiple images, than on domain-specific single-view depth priors. 

  • Anomaly Detection in Multi-Agent Trajectories for Automated Driving, Julian Wiederer, Mercedes Benz.
    Autonomous cars will sooner or later incur novel or insecure situations, where risk minimizing strategies have to be applied to solve these situations. This work shows methodologies to automatically detect abnormal interactions between road users in order to trigger such defensive strategies.

  • Prediction of the inference time of neural networks, Domenik Helms, OFFIS. 
    In order to design embedded AI systems, the hardware requirements and limitations should be considered from the very first step. This work thus presents a methodology to determine the to-be-expected hardware execution times from an analysis of the general AI topology. Before even training an AI, the hardware compitability can thus be made sure.

  • Hardware aware CNN compression for Semantic Segmentation, Manoj Rohit Vemparala, BMW.
    Several optimization techniques exist to adapt the topology of an existing, pre-trained AI onto the requirements of the hardware. Most such work does optimize for general, available parameters such as number of operations or number of parameters. In this work, we present how to actually regard relevant hardware metrics and how to succesfully combine several optimization methodologies into a hardware aware optimization.