Personal tools

PerOpteryx

From Wissensbasis

Jump to: navigation, search

PerOpteryx is an optimization framwork to improve component-based software architectures based on model-based quality prediction techniques. Currently, it is realised as a set of Eclipse plugins to automatically improve Palladio Component Model instances for performance, reliability, and cost properties. Conceptually, it is independent of the considered quality attributes, software architecture metamodelling language, and degrees of freedom; and can be applied to improve any software architecture models using any quantitative quality prediction technique.

Bat.jpg        Peropt-logo.jpg

Contents

Background

Bat.jpg
PerOpteryx stands for "Performance Optimizer" and is named after a bat, because it shall explore the architectural design space like a bat explores physical space with sonar.

The tool takes a complete PCM instance as input and produces new PCM instances by manipulating certain parameters of the model based on a genetic algorithm. As a new and distinctive feature, it uses domain-specific knowledge (e.g. performance tactics) to improve the generic evolutionary optimisation. PerOpteryx analyses the quality attributes (e.g. performance and costs) of the new instances and tries to find the Pareto front of optimal trade-off solutions. It helps software architects to make informed trade-off decisions between different, conflicting quality attributes.

PerOpteryx works on top of the PCM Bench and uses PCM2LQN, LQNS, and PCMMarkov for performance and reliability prediction. It also features a simple cost calculator that can be used to model any type of costs (e.g. development costs, procurement costs, operating costs, or total cost of ownership). Analysis tools for other quality attribute can be readily added as Eclipse plugins using Eclipse's extension point mechanism. PerOpteryx is implemented based on the Opt4J framework and uses the efficient, multi-objective NSGA-II genetic algorithm internally.

The tool is currently being developed at the Chair for Software Design and Quality of the Karlsruhe Institute of Technology in Germany. For details and questions about PerOpteryx, please contact Anne Koziolek.

Documentation

Publications

Publications discussing different aspects of the approach and case studies (see http://sdq.ipd.kit.edu/people/anne_koziolek/publications/ for full bibliographic details and bibtex entries).

Core PerOpteryx Publications

  • ICPE'12: An industrial case study of performance and cost design space exploration (PDF)
  • QoSA'11: Peropteryx: Automated application of tactics in multi-objective software architecture optimization. (PDF), details on case studies can be found at PerOpteryx/Tactics Case Study.
    • Improving the efficiency of the optimization using tactics derived from domain knowledge.
  • CBSE'11: Towards a generic quality optimisation framework for component-based system models. (PDF)
    • Described the generic design space formulation used to optimize software architectures modelled in any architecture modelling language for any quantitatively predictable quality attribute.
    • Metamodel-independent formulation of the degrees of freedom
  • WOSP/SIPEW'10: Automatically Improve Software Models for Performance, Reliability, and Cost Using Evolutionary Algorithms (PDF)
  • QUASOSS'09: Optimising multiple quality criteria of service-oriented software architectures (PDF)
    • Early workshop paper including reliability prediction into PerOpteryx
  • FESCA'09: Automatic, model-based software performance improvement for component-based software designs (PDF)
    • Early workshop paper proposing a hill-climbing approach, only considers performance, superseded by above papers.
  • WCOP'08: Performance-oriented Design Space Exploration (PDF)
    • First presentation of the idea at a workshop, only considers performance, superseded by above papers.

Thesis

Most detail is provided by the dissertation thesis on PerOpteryx, (PDF).

Extension to Requirements Engineering

  • REFSQ'12: Research preview: Prioritizing quality requirements based on software architecture evaluation feedback. (PDF)

Related Publications

Other publications related to PerOpteryx or extending it:

  • ICPE'11: Detection and solution of software performance antipatterns in palladio architectural models (DOI), ICPE best paper award.
  • Workshops and Symposia at MODELS 2010: Focussing multi-objective software architecture optimization using quality of service bounds (revised and extended version of QUASOSS paper below) (PDF)
  • QUASOSS'10: Using Quality of Service Bounds for Effective Multi-objective Software Architecture Optimization (PDF)
    • Initial work to extend PerOpteryx to consider bounds for quality requirements to focus the search on interesting regions
  • QoSA'10: A Hybrid Approach for Multi-Attribute QoS Optimisation in Component Based Software Systems. Joint Work with Danilo Ardagna and Raffaela Mirandola from Politecnico Milano, Italy. (PDF)
  • FASE'10: A process to effectively identify ”guilty” performance antipatterns. Joint work with Vittorio Cortellessa and Catia Trubiani from University of L'Aquila, Italy. (PDF)

Tool Documentation

The tool contains some help content. Click the quation mark in the Eclipse Run Configuation (shown below).

PerOpteryx run config.PNG

Degrees of Freedom

The current implementation supports these degrees of freedom

  • Component selection for components that offer at least the same interfaces than the current one and require exactly the same interfaces (deletion of AssemblyConnectors is not implemented yet).
  • Component allocation (no check that servers have to offer all required resource types yet implemented)
  • Continuous Processing Rate change: the processing rate of any resource can be continuously varied in a given interval. Note that the MTTF is adjusted linearly together with the rate: Higher rates lead to higher MTTF, too, because processors are more expensive and more carefully produced (this could as well be modelled the other way around, but is not configurable yet).
  • Scheduling Policies for processing resources can be adjusted
  • Capacity of passive resources can be adjusted.
  • Replication of servers: Replicas of servers can be created for load balancing (not in the core PerOpteryx release, but available on an svn branch), see ICPE'12 publication (PDF).

More degrees of freedom for component-based architectures are discussed in our WOSP paper and in Anne's thesis (see above).

These generic degrees of freedom (GDoF) are instantiated automatically for a concrete system at hand as shown in this example. Each GDoF has a matching rule that detects instances of itself:

PerOpteryx-DoFInstances.png

Each degree of freedom instance (DoFI) has a set of possible options. In this tiny example, we have two components to use at place C in the architecture, namely component C1 (grey) and component C2 (green). Let us further assume we have three components to use at place D, namely D1, D2, and D3. Additionally, we can allocate each of the components to other servers (let us assume we have one more server S2 not shown here). Finally, we could change the server speed of the servers S1 and S2. Thus, we get

  • Component selection DoFI at C with options C1 and C2
  • Component selection DoFI at D with options D1, D2, and D3
  • Component allocation DoFI at C with options S1 and S2
  • Component allocation DoFI at D with options S1 and S2
  • Server speed DoFI at S1 with options [1GHz, 3GHz]
  • Server speed DoFI at S2 with options [1GHz, 3GHz]

The choice of values are the genotype of a candidate. The next figure shows an excerpt of the DoFI.

PerOpteryx-EvaluateCandidate.png

Additionally, the figure shows how to evaluate the quality properties for a candidate, i.e. for a vector of chosen values from the design space spanned by the DoFI. A model transformation applies the chosen values (the genotype) to the initial system model (lower arrow). The transformation rules are defined generically by the GDoF. Then, the generated model (the phenotype) can be evaluated using standard techniques, resulting in the predicted quality properties. This translation from genotype to phenotype is made transparent for the optimisation algorithm, which just sees that a set of choices (a genotype) can be evaluated to quality properties (uper arrow).

The images are taken from the WOSP talks, which can be downloaded as pdf or animated powerpoint slides above.

Note that a DoFI does not only represent the update of a single variable in the model. For example, when replacing a component in the PCM, one has to update the place in the architecture (called AssemblyContext) to bind the new component and additionally has to update the connectors so that they connect the right interfaces. In principle, a DoFI can be any model transformation. The restriction is that the model transformation must not change the model too much so that other DoFI become invalid. This is discussed in more detail in my thesis, which is (hopefully) available soon.

The figure below shows another example for the instantiation of DoFI from GDoF and a given system model. The manual input is the system model. Then, with the generic degree of freedom (GDoF) rules, the DoFI are automatically instantiated. Additionally, candidates canbe automatically evaluated (see picture above).

PerOpteryx-OutlineDesignSpaceSection.png

Cost Model

PCM models can be annotated with costs in a cost model file. Types of costs are Component Cost, VariableProcessingResourceCost, VariableLinkingResourceCost, FixedProcessingResourceCost, and FixedLinkingResourceCost. Note that for the VariableProcessingResourceCost, it is possible to specify a function for the cost based on the processing rate of that resource, e.g. 0.7665*((procRate.VALUE/2)^6.2539)+145 . The keyword procRate.VALUE is then replaced by the current processing rate of a resource.

Details for published and submitted papers

Download / Install / Use

PerOpteryx-HowTo-help.jpg]

First, you need to configure the model files. Enter the files names in the "model files" tab as usual. The only new thing for PerOpteryx is the QMLCriteria Definition, which allows you to define which criteria (mean response time, reliability, costs) you want to optimise with your system. Enter the qmldeclaration that comes with the example model as shown below. Alternatively, check this page. For details how to create a QML declaration file for your own model, refer to Qais diploma thesis.

PerOpteryx-HowTo-model-files.jpg

In the second tab, you can configure the options for PerOpteryx. The screenshot below shows the most common setting: Choose a number of iterations that the search should run and the size of the population in each iteration. Choose "Evolutionary Search" in the drop down box below to optimise using genetic algorithms (the other two are for validation purposes).

The only other important setting in this tab are the "design decision options" at the bottom. You can either provide your own designdecision file (= degrees of freedom to use in the search) here or let PerOpteryx determine one.

  • Let PerOpteryx determine the designdecisions: Check the box "Only determine design decisions, do not optimise" to let PerOpteryx determine the designdecisions and let you review them before optimising. You need to provide a file name with full path here so that PerOpteryx knows where to write the designdecisions to. Open the file and check it, before proceeding to the third case in this list.
  • Let PerOpteryx determine the designdecisions and optimise them right away: Uncheck both checkboxes
  • Optimise your system with predefined designdecisions that are to be loaded from the given file. Check "Only optimise, using pre-existing design decisions" to make PerOpteryx optimise the system with the designdecisions loaded from the given file.

In the middle of this tab, you can make some specialised settings:

  • In the "Starting Population" field, you can provide a csv file with a staring population for PerOpteryx. If you do not provide any, the start population is randomly generated (the first candidate is always the system as you modelled it). A .csv file is expected. It has to have the same format as the optimisation results file (thus, run the optimisation first if you are not sure how they look like). You can, however, omit the objective values in the predefined candidates.

If you specify predefined candidates and set the maximum number of iterations to 0, then the tool will evaluate all candidates specified in the file, regardless of the specified "number of individuals per iteration" and of the specified search algorithm.

  • The other two possibilities to enter a file are 1) an option to provide objective values for a cache so that rerunning an optimisation can run faster (usually not needed) and 2) an option to continue an interrupted run, in that case not only the last population of the interrupted run needs to be provided as the start population (see above), but also the archive of the internal genetic algorithm NSGA-II needs to be provided. The archive is one of the result files of PerOpteryx.

PerOpteryx-HowTo-DSE-options.jpg

Finally, you can configure the quality analyses as usually.

In the performance analysis tab you can select SimuCom, LQNS or the LQN simulator. The figure below shows an example configuration for LQNS.

PerOpteryx-HowTo-performance-analysis.jpg

In the reliability analysis tab you can configure the Markov solver as usual. PerOpteryx-HowTo-reliability-analysis.jpg|400px]]

To analyse costs, you need a cost model for your system. See e.g. the example model shown in the figure below. Basically, you can assign costs to components in the repository (component costs) which will be added to the system cost if the component is used and thus allows to model costs for different component alternatives. Second, you can assign costs to servers. You can either assign fixed costs for a server, or Variable Costs that are adjusted depending on the processing rate of the server. The latter is useful if the processing rate of your servers is a designdecision (i.e., a degree of freedom) in your system. In that case you should use Variable Costs so that a faster resource indeed leads to higher costs. A cost model derived from data from Intel is 0.7665*((procRate.VALUE)^6.2539)+145 where procRate.VALUE is a keyword for PerOpteryx which is replaced by the current processing rate of the resource for a candidate. The processing rate in this case is assumed to be specified in GHz.

Check out the example cost model in the example (file ending .cost) to learn more.

PerOpteryx-HowTo-costs-analysis.jpg

The other tabs can be ignored. Their options should be unchecked in most cases.

Then, hit Run to start the optimisation! Note that to test it, it might be useful to configure a few iterations (e.g. 2) and LQNS, so that it finishes quickly.

PerOpteryx will output the results as csv files and designdecision files in a subfolder PerOpteryx_results of your current project (see figure below).

PerOpteryx-HowTo-where-results.jpg

For each iteration, three output files are created. Thus, even while the optimisation is still running, you can check preliminary results. The first number in the file name (before the underscore) is the iteration). The most important files are the archive*: They contain the results the optimisation found so far, i.e. the Pareto-optimal candidates of that iteration. The allCandidates* files contain all candidates evaluated up to that iteration. The population* files contain the population of the genetic algorithm in that iteration.

The csv files can be used again as input to the tool as described above. They look as follows:

PerOpteryx-HowTo-results-cvs.jpg

Each row contains a candidate architecture. The figure aboe shows the results in the archive (i.e., the Pareto-optimal candidates) after 10 iterations. The first columns contain the quality values for the candidates. Here, it is first POFOD, then costs, then mean response time. Then, the next columns are the design decisions of the candidate, i.e. which choices where made for the degrees of freedom for this candidate. In this example, we first have the Processing Rates of the three servers, then the allocation of the three components (the value in the field says to which server the component in the headline of the column is allocated to), and the next columns contain the choice of components, if any alternatives were given in the system. Here, two alternative components could be used in the AssemblyContext "Assembly_BookingSystem", there were two available options "BookingSystem" and "QuickBooking". The final columns give the utilisation of the servers.

You can draw a diagram in Excel to study the tradeoffs. Sort the results first if you have infinity values, because Excel cannot plot these. Then you can create a result plot like below.

PerOpteryx-HowTo-results-plot.jpg

Screenshots

  • System-Editor

  • Repository-Editor

  • Allocation-Editor

  • Design Space Exploration Options

  • Performance Analysis Options

  • Reliability Analysis Options

  • Peropteryx Running...

  • Peropteryx Results in Excel

  • Peropteryx 3D Pareto Front

  • Peropteryx 3D Pareto Front 2

  • Peropteryx 3D Pareto Front 3

  • Peropteryx 2D Pareto Front

Example Models

Below you can find several PCM model instances, which you can use for playing around with Peropteryx.

Minimal Example

This is a minimal PCM instance with all annotations to perform a successful run of Peropteryx. It consists of three components and three servers.

UML Visualization of the Minimal Example PCM instance         Layered Queueing Network generated from PCM instance

Business Reporting System

This is a larger PCM instance with 8 servers and 12 components. It models a Business Reporting System where clients can generate and download business reports from data stored in a database. The model has been used in this paper as a case study and is loosely based on a realistic system.

UML Visualization of the Business Reporting Server PCM instance         Layered Queueing Network generated from PCM instance

The LQN model has been created with the PCM2LQN tool (which is included in the latest PCM release). It contains multiple dummy processors and tasks, which were added to make the transformation from PCM more straightforward.

A comparison of the evolutionary optimisation with a random search for this case study is available on a separate page: PerOpteryx/BRSRandomSearch.

Quickstart

To get started with the optimisation, you need a valid QML model that defines your requirements on the PCM instance. This page shows a QML model quickstart.

Related Projects

Foundations

PerOpteryx applies a multiobjective genetic algorithm to the problem. Currently, NSGA2 as implemented in the Opt4J framework has been chosen.

Concerning multiobjective optimisation, the following work is useful as background information

  • Blum & Roli, Metaheuristics in combinatorial optimization: Overview and conceptual comparison (DOI), they discuss single-objective metaheuristics.
  • Ehrgott, Multicriteria Optimisation (Google Books link), he discusses multiobjective problems in general and presents algorithmic solutions.
  • Coello Coello, A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques (DBLP Entry Google Scholar search)
  • Sean Luke, 2009, Essentials of Metaheuristics (available at http://cs.gmu.edu/~sean/book/metaheuristics/)
  • Harman et al., Search Based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications, (PDF)

Development

PerOpteryx code is available at

Team

Anne Koziolek (Martens) Ralf Reussner Heiko Koziolek Steffen Becker
Anne Koziolek (Martens) Ralf Reussner Heiko Koziolek Steffen Becker 
keywords: Generic design space exploration framework, software architecture, quality optimization, quality improvement, degrees of freedom
Retrieved from "http://sdqweb.ipd.kit.edu/mediawiki-sdq-extern/index.php?title=PerOpteryx&oldid=16931"
KIT – Universität des Landes Baden-Württemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft