Semantische Suche

In this thesis, we propose FeLOF, addressing this challenging setting of outlier detection in feature-evolving and high-dimensional data. Our algorithms extends the well-known Local Outlier Factor algorithm to the feature-evolving stream setting. We employ a variation of StreamHash random hashing projections to create a lower-dimensional feature space embedding, thereby mitigating the effects of the curse of dimensionality. To address non-stationary data distributions, we employ a sliding window approach. FeLOF utilizes efficient data structures to speed up search queries and data updates.

Extensive experiments show that our algorithm achieves state-of-the-art outlier detection performance in the static, row stream and feature-evolving stream settings on real-world benchmark data. Additionally, we provide an evaluation of our StreamHash adaptation, demonstrating its ability to cope with sparsely populated high-dimensional data.

In this thesis, we explore another approach for estimating the expected loss of an ML classifier. The aim is to enhance test data with additional expert knowledge. Inspired by recent feature attribution techniques, such as LIME or Saliency Maps, the idea is that experts annotate inputs not only with desired classes, but also with desired feature attributions. We then explore different methods to derive a large conventional test data set based on few such feature attribution annotations. We empirically evaluate the loss estimates of our approach against ground-truth estimates on large existing test data sets, with a focus on the tradeoff between the number of expert annotations and the achieved estimation accuracy.

Portfolio-basierte Methoden zum Lösen von SAT-Instanzen nutzen die komplementäre Stärke von einer Menge von verschiedenen SAT-Algorithmen aus. Hierbei wird aus einer gegebenen Menge von Algorithmen, dem sogenannten Portfolio, mittels eines Vorhersagemodells derjenige Algorithmus ausgewählt, der die bestmögliche Performance für die betrachtete SAT-Instanz verspricht.

In dieser Arbeit interessieren wir uns besonders für erklärbare Portfolios, sprich für Portfolios, für die man nachvollziehen kann, wie die Vorhersage zu ihrem Ergebnis kommt. Gute Erklärbarkeit resultiert einerseits aus einer geringen Größe des Portfolios, andererseits aus einer reduzierten Komplexität des Vorhersagemodells.

Im Vordergrund der Arbeit liegt das effiziente Finden von kleinen Portfolios aus einer größeren Menge von Algorithmen, sowie den Einfluss der Portfoliogröße auf die Performance des Portfolios.

In this thesis we present an evaluation framework for Automated Feature Generation methods, that is integrated into the scikit-learn framework for Python. We integrate nine Automated Feature Generation methods into this framework. We further evaluate the methods on 91 datasets for classification problems. The datasets in our evaluation have up to 58 features and 12,958 observations. As Machine Learning models we investigate five models including state of the art models like XGBoost.

In this thesis, we propose GAK-SVDD. We combine the well-known SVDD algorithm to detect outliers in an unsupervised fashion with Global Alignment Kernels (GAK), bypassing the feature-extraction step. We evaluate GAK-SVDD's performance on 28 standard benchmark data sets and show that it is on par with its closest competitors. Comparing GAK with a DTW-based kernel, GAK improves the median Balanced Accuracy by 4%. Additionally, we extend our method to the active learning setting and examine the combination of GAK and domain-independent attributes.

In this thesis, we provide a novel approach to enable verification of process models with data values, so-called data-value-aware process models. A distinctive of our approach is to support modification of data values while preserving the verification results. We show the functionality of our approach by conducting the verification of a real-world application: the German 4G spectrum auction model.

However, the already existing methods for time series monitoring tend to ignore the background information such as relationships between components or process structure that is available for almost any complex system. Such background information gives a context to the time series data, and can potentially improve the performance of time series monitoring tasks.

In this bachelor thesis, we show how to incorporate structured background information to improve three different time series monitoring tasks. We perform the experiments on the data from the cloud computing center, where we extract background information from system traces. Additionally, we investigate different representations and quality of background information and conclude that its usefulness is independent from a concrete time series monitoring task.

Es werden LSTMs, GRUs, CTRNNs und Elman Netze untersucht. Die Netze werden dabei untersucht sich einen Punkt zu merken und anschließend nach dem Punkt mit einem virtuellen Roboterarm zu greifen.

Bei LSTM, GRU und Elman Netzen wird auch untersucht wie die Netze die Aufgabe lösen, wenn jedes Neuron nur auf den eigenen Speicher zugreifen kann.

Dabei hat sich herausgestellt, dass LSTMs und GRUs deutlich besser bei den Experimenten bewertet werden als CTRNNs und Elman Netze. Außerdem werden die Rechenzeit und der Zusammenhang zwischen der Anzahl der zu trainierenden Parameter und der Ergebnisse der Experimente verglichen.

With this thesis, we propose PeReDeS, an approach that integrates into the development cycle of modern software projects, and explicitly models an automated performance regression detection system that provides feedback quickly and reduces manual effort for setup and load test analysis. PeReDeS is embedded into pipelines for continuous integration, manages the load test execution and lifecycle, processes load test results and makes feedback available to the authoring developer via reports on the coding platform. We further propose a method for detecting deviations in performance on load test results, based on Welch's t-test. The method is adapted to suit the context of performance regression detection, and is integrated into the PeReDeS detection pipeline. We further implemented our approach and evaluated it with an user study and a data-driven study to evaluate the usability and accuracy of our method.

This thesis evaluates the quality of estimated change sequences in the context of model consistency preservation. To derive such sequences, matching elements across the compared models need to be identified and their differences need to be computed. We evaluate a sequence derivation strategy that matches elements based on their unique identifier and one that establishes a similarity metric between elements based on the elements’ features. As an evaluation baseline, different test suites are created. Each test consists of an initial and changed version of both a UML class diagram and consistent Java source code. Using the different strategies, we derive and propagate change sequences based on the state-based difference of the UML view and evaluate the outcome in both domains. The results show that the identity-based matching strategy is able to derive the correct change sequence in almost all (97 %) of the considered cases. For the similarity-based matching strategy we identify two reoccurring error patterns across different test suites. To address these patterns, we provide an extended similarity-based matching strategy that is able to reduce the occurrence frequency of the error patterns while introducing almost no performance overhead.

In dynamical systems, like gas turbines, transition phases occur after a change in the input control signal. The domain knowledge about the steepness of these transitions can potentially help with the modeling of such systems using the data science approaches. There already exist TGDS approaches that use the information about the limits of the values. However it is currently not clear how to incorporate the information about the steepness of the transitions with them.

In this thesis, we develop three different TGDS approaches to include these transition constraints in recurrent neural networks (RNNs) to improve the modeling of input-output behavior of dynamical systems. We evaluate the approaches on synthetic and real time series data by varying data availability and different degrees of steepness. We conclude that the TGDS approaches are especially helpful for flat transitions and provide a guideline on how to use the available transition constraints in real world problems. Finally, we discuss the required degree of domain knowledge and intellectual implementation effort of each approach.

While some approaches use additional information about the current state of the environment, bandit algorithms tend to ignore domain knowledge that can’t be extracted from data. It is not clear how to incorporate domain knowledge into bandit algorithms and how much improvement this yields.

In this masters thesis we propose two ways to augment bandit algorithms with domain knowledge: a push approach, which influences the distribution of arms to deal with non-stationarity as well as a group approach, which propagates feedback between similar arms. We conduct synthetic and real world experiments to examine the usefulness of our approaches. Additionally we evaluate the effect of incomplete and incorrect domain knowledge. We show that the group approach helps to reduce exploration time, especially for small number of iterations and plays, and that the push approach outperforms contextual and non-contextual baselines for large context spaces.

In unserer Arbeit stellen wir einen Ansatz vor, der aus einer gegebenen repräsentativen Benchmark eine kleinere Teilmenge wählt, die als repräsentative Benchmark dienen soll. Wir definieren dabei, dass eine Benchmark repräsentativ ist, wenn der Graph der Laufzeiten ein festgelegtes Ähnlichkeitsmaß gegenüber der ursprünglichen Benchmark überschreitet. Wir haben hierbei einen BeamSearch-Algorithmus erforscht. Am Ende stellen wir allerdings fest, dass eine zufällige Auswahl besser ist und eine zufällige Auswahl von 10 % der Instanzen ausreicht, um eine repräsentative Benchmark zu liefern.

We extend an existing data flow diagram approach with our concept of trust. Our approach of adding knowledge to a software architecture model and providing a way to analyze model instances for access control violations shall enable software architects to increase the quality of models and further verify access control requirements under uncertainty. We evaluate the applicability based on the availability, the accuracy and the scalability regarding the execution time.

For many explainers this proposed reasoning gives us more information about the inner workings of the model or even about the training data. Since data privacy is becoming an important issue the question arises whether explainers can leak private data. It is unclear what private data can be obtained from different kinds of explanation. In this thesis I adapt three privacy attacks in machine learning to the field of XAI: model extraction, membership inference and training data extraction. The different kinds of explainers are sorted into these categories argumentatively and I present specific use cases how an attacker can obtain private data from an explanation. I demonstrate membership inference and training data extraction for two specific explainers in experiments. Thus, privacy can be breached with the help of explainers.

To close the gap between the development and having up-to-date performance models, the Continuous Integration of Performance Models (CIPM) approach has been proposed. It incorporates automatically executed activities into a Continuous Integration pipeline and is realized with Vitruvius combining Java and the PCM. As a consequence, changes from a commit are extracted to incrementally update the models in the VSUM. To estimate the resource demand in the PCM, the CIPM approach adaptively instruments and monitors the source code.

In previous work, parts of the CIPM pipeline were prototypically implemented and partly evaluated with artificial projects. A pipeline combining the incremental model update and the adaptive instrumentation is absent. Therefore, this thesis presents the combined pipeline adapting and extending the existing implementations. The evaluation is performed with the TeaStore and indicates the correct model update and instrumentation. Nevertheless, there is a gap towards the calibration pipeline.

The goal of this Master’s thesis is to develop and evaluate a framework to efficiently and effectively detect “when” and “where” concept drift occurs in high-dimensional data streams. We introduce stream autoencoder windowing (SAW), an approach based on the online training of an autoencoder, while monitoring its reconstruction error via a sliding window of adaptive size. We will evaluate the performance of our method against synthetic data, in which the characteristics of drifts are known. We then show how our method improves the accuracy of existing classifiers for predictive systems compared to benchmarks on real data streams.

Logs are similar to natural languages. Recent deep learning algorithm Transformer Neural Network has shown outstanding performance in Natural Language Processing (NLP) tasks. Based on these, we adapt Transformer Neural Network to detect outliers from applications logs In an unsupervised way. We compared our algorithm against state-of-the-art log outlier detection algorithms on three widely used benchmark datasets. Our algorithm outperformed state-of-the-art log outlier detection algorithms.