Subprojects

Structure of the project

Subproject 1:
Data Acquisition

In order to be able to investigate a large number of deltas from different domains, special data is required that is tailored to the requirements of the project. New data sets with different sensors will be developed for this project, which will be recorded, generated and labelled from various traffic domains and different environmental conditions.

Data management and processing

Due to the special data requirements, it is necessary that partial data sets for specific delta scenarios, for example for certain weather conditions or locations, can be flexibly extracted from the overall data set and combined with each other. A corresponding tooling is being developed to enable this.

Real data recording

A carefully selected training data set is crucial for the meaningful project results. With the recording of real data from different domains, an enormous amount of data is recorded, which is necessary for valid research results in the context of KI Delta Learning.

Generation of synthetic data

Synthetic data will be used to complement the real data set. Situations that are difficult to record in real life are to be simulated as well as general data that simulates the real recording setup. Thus, real and synthetic data from two domains can be compared efficiently.

Labeling of data

Data for AI models is often only valuable if matching labels are available. In addition to the research of automatic labeling in the project, manual annotation techniques are used for labeling. The suitability of different approaches for delta learning is investigated.

Due to this holistic data set, methods will be developed to bridge deltas in the scaling of AI: Transfer learning,didactics and automotive suitability.

Subproject 2:
Transfer Learning

The aim of the project is to develop methods that allow AI systems to deal with changing conditions efficiently - without necessary changes in the architecture and without requiring a completely new training. The approach of continuous learning is applied: Here, the learned knowledge is not forgotten when transferring from one domain to another. Furthermore, methods are developed that can handle several domains at the same time.

Continuous learning

As a basis for the consideration of concrete Delta Learning-related use cases a broad collection of methods of continuous learning is developed and investigated. These primarily address situations where comparatively little new data is added to a still relevant database. Traditional approaches lead to "catastrophic forgetting": the networks unlearn how to handle the original, already well mastered database. This problem is addressed by investigating countermeasures, including the study of intermediate representations and continuous training.

Use of synthetic data

Synthetic data from simulation environments promise a favourable acquisition of large amounts of annotated training examples and an explicit control over the scenarios depicted in them. On the one hand, this enables the investigation of scenarios that rarely occur in the real world or that endanger human life. On the other hand, the accurate control over data generation allows the precise, isolated examination of individual deltas. Despite the progress of modern rendering engines, they are not photorealistic enough to transfer learned models from the simulation, the source domain, to the real world, the target domain. For this reason, methods to overcome the delta between synthetic and real data are analysed.

Cross sensor adaptation

In the field of cross sensor adaptation, sensor-specific cases of domain adaptation, i.e. deltas with respect to their different sensor configurations, are investigated. In addition to changes in sensor type and sensor technology, parameters such as resolution, opening angle and installation position must be taken into account for the deltas to be bridged. Methods that have been identified as suitable for closing the deltas are meta-learning methods, methods that use invariant features, GAN methods, processes for physical transformation and knowledge graph methods.

Location and time domain

Changes in the location domain as well as long-term changes in the time domain currently pose a great challenge for AI modules trained on other data. Examples are changes of the seasons or country-specific appearances, such as different road markings or signage. Often AI modules have to be re-trained for unknown or different looking locations. The project explores the efficient handling of these changes and develops methods for delta learning and knowledge transfer.

Environmental adaptation

In addition to long-term temporal changes, short-term changes in the environment, for example due to weather conditions or daytime, will also be investigated. Here the adaptation of AI modules to different dynamic changes of the same environment is examined without having recorded all changes for each type of environment.

Subproject 3:
Didactics

In the application area of automotive environment perception, it is necessary to provide a large number of annotated and high-resolution training data points to meet the high demands on the performance of the algorithms in a highly complex environment.  For automotive applications, the supervised training of neural networks has several disadvantages that can be categorized into two core problems: enormous consumption of time and resources for both training and data labeling. In summary, it makes sense from both an economic and ecological point of view to optimise the training process, i.e. to apply targeted didactics.

Partially supervised and unsupervised learning

Partially supervised and unsupervised learning refers to a category of machine learning in which no explicit annotations of data are required for specific tasks. These are important learning strategies for saving resources. In this project, unsupervised learning is investigated using a variety of approaches.

Training organisation

The successful training of neural networks is dependent on various factors, such as clean data, efficient model architectures and optimised hyperparameters. In this project the focus is on augmentation in training organisation, hierarchical data models and detection of anomalies to improve and optimise training organisation.

Active learning

Active learning is a collective term for algorithms that (partly) determine themselves for which training examples they need annotations. An example of this is the special consideration of currently incorrectly classified data points, or the identification of those situations that do not correspond to the previous training data distribution. This both reduces the labeling effort and shortens the training time.

Knowledge transfer

The term knowledge transfer describes the transfer of knowledge embedded in one algorithm to a second algorithm. Often a teacher instructing a student is used as an analogy. Knowledge transfer is therefore not a specific procedure, but a methodology involving two concepts: the representation of knowledge and the transfer of knowledge between algorithms.

Subproject 4:
Automotive Suitability

Two specific areas will be examined in more detail for the proper application of AI modules, where current AI procedures show weaknesses but are crucial for the success of highly automated vehicles: the robustness in an open world with countless combinations of scenarios and the challenges of operating AI on embedded systems with limited hardware resources while taking into account boundary requirements.

Robustness of the AI systems even in unknown situations and in case of faults

The number of situations which a vehicle has to master is extremely large due to the complex environment. Furthermore, noise in the sensor data is only one example for a variety of possible faults. A system that can handle such influences is called robust. The first approach to improve automotive suitability deals with the quantification of the robustness of AI methods. It is planned to apply the methods and metrics developed from this in various fields of application in order to specifically increase the robustness of the systems in open-world scenarios. 

Compliance with boundary conditions under certain hardware requirements

The second approach deals with the challenges of transferring trained AI models from laboratory hardware to the highly resource-limited hardware embedded in vehicles. In order to act safely, embedded AI modules must be able to respond in real time despite the limited hardware resources. However, at the same time they must also comply with additional boundary conditions, such as limitedenergy consumption. Corresponding criteria will be worked out in order to guide the development of procedures for the optimisation of AI components with regard to these criteria.

Subproject 5:
Evaluation

In order to ensure a uniform and consistent evaluation across all activities, certain activities are carried out in a centralised manner. Depending on the task, such as active learning or changes in the location domain and data type, binding metrics and test data are defined for the project increments in the project network. For example, the project partners in active learning agree on certain data sets and metrics. At the end of the project increment, the individual approaches are compared and new data or metrics are determined based on the challenges identified. In addition, after each phase research is conducted to determine whether particularly innovative new developments outside the project should be included in the work of KI Delta Learning, such as new algorithms. Finally, the status prior to the project can be effectively compared with the results at the end of the project and, hence, the boost to innovation of a development can be assessed.