Abstract: Building and operating software-intensive systems often involves exploring decision spaces made up of large numbers of variables and complex relations among them. Understanding such spaces is often overwhelming to human decision makers, who have limited capacity to digest large amounts of information, making it difficult to distinguish the forest through the trees. In this article, we report on our experience in which we used dimensionality reduction techniques to enable decision makers in different domains (software architecture, smart manufacturing, automated planning for service robots) to focus on the elements of the decision space that explain most of the quality variation, filtering out noise, and thus reducing cognitive complexity.

Abstract: In software design, guaranteeing the correctness of run-time system behavior while achieving an acceptable balance among multiple quality attributes remains a challenging problem. Moreover, providing guarantees about the satisfaction of those requirements when systems are subject to uncertain environments is even more challenging. While recent developments in architectural analysis techniques can assist architects in exploring the satisfaction of quantitative guarantees across the design space, existing approaches are still limited because they do not explicitly link design decisions to satisfaction of quality requirements. Furthermore, the amount of information they yield can be overwhelming to a human designer, making it difficult to see the forest for the trees. In this paper we present ExTrA (Explaining Tradeoffs of software Architecture design spaces), an approach to analyzing architectural design spaces that addresses these limitations and provides a basis for explaining design tradeoffs. Our approach employs dimensionality reduction techniques employed in machine learning pipelines like Principal Component Analysis (PCA) and Decision Tree Learning (DTL) to enable architects to understand how design decisions contribute to the satisfaction of extra-functional properties across the design space. Our results show feasibility of the approach in two case studies and evidence that combining complementary techniques like PCA and DTL is a viable approach to facilitate comprehension of tradeoffs in poorly-understood design spaces.

Abstract: Self-adaptive systems commonly operate in heterogeneous contexts and need to consider multiple quality attributes. Human stakeholders often express their quality preferences by defining utility func-tions, which are used by self-adaptive systems to automatically generate adaptation plans. However, the adaptation space of realistic systems is large and it is obscure how utility functions impact the generated adaptation behavior, as well as structural, behavioral, and quality constraints. Moreover, human stakeholders are often not aware of the underlying tradeoffs between quality attributes. To address this issue, we present an approach that uses machine learning techniques (dimensionality reduction, clustering, and decision tree learning) to explain the reasoning behind automated planning. Our approach focuses on the tradeoffs between quality attributes and how the choice of weights in utility functions results in different plans being generated. We help humans understand quality attribute tradeoffs, identify key decisions in adaptation behavior, and explore how differences in utility functions result in different adaptation alternatives. We present two systems to demonstrate the approach's applicability and consider its potential application to 24 exemplar self-adaptive systems. Moreover, we describe our assessment of the tradeoff between the information reduction and the amount of explained variance retained by the results obtained with our approach.

Abstract: For realistic self-adaptive systems, multiple quality attributes need to be considered and traded off against each other. These quality attributes are commonly encoded in a utility function, for instance, a weighted sum of relevant objectives. Utility functions are typically subject to a set of constraints, i. e., hard requirements that should not be violated by the system. The research agenda for requirements engineering for self-adaptive systems has raised the need for decision-making techniques that consider the trade-offs and priorities of multiple objectives. Human stakeholders need to be engaged in the decision-making process so that constraints and the relative importance of each objective can be correctly elicited. This paper presents a method that supports multiple stakeholders in eliciting constraints, prioritizing relevant quality attributes, negotiating priorities, and giving input to define utility functions for self-adaptive systems. We developed tool support in the form of a blackboard system that aggregates information by different stakeholders, detects conflicts, proposes mechanisms to reach an agreement, and generates a utility function. We performed a think-aloud study with 14 participants to investigate negotiation processes and assess the approach's understandability and user satisfaction. Our study sheds light on how humans reason about and how they negotiate around quality attributes. The mechanisms for conflict detection and resolution were perceived as very useful. Overall, our approach was found to make the process of utility function definition more understandable and transparent.

Abstract: The emergence of powerful automated design tools in many domains is changing the nature of design, as human-intensive activities can be increasingly off-loaded to those tools. Rather than having a human consider only handful of options, as has been done historically, such tools now enable the generation of a large space of potential designs, exhibiting different tradeoffs among competing qualities of merit, and supporting systematic exploration of the design space. At the same time, this paradigm raises new challenges centered on enabling humans to effectively navigate that generated space in order to select a design that best meets their requirements. In this paper we describe our experience in the domain of generative manufacturing, in which we developed a novel design environment for airplane parts manufacturing that incorporates a number of sophisticated design tools and attempts to tackle the emergent problems of design space exploration that are faced by designers of those parts. We use this experience to highlight the challenges that we faced and reflect on their applicability more generally to tool-assisted software design environments.

Abstract: When designing and evolving software architectures, architects need to consider large design spaces of architectural decisions. These decisions tend to impact the quality attributes of the system, such as performance, security, or reliability. Relevant quality attributes might influence each other and usually need to be traded off by architects. When exploring a design space, it is often challenging for architects to understand what tradeoffs exist and how they are connected to architectural decisions. This is particularly problematic in architectural spaces generated by automated optimization tools, as the underlying tradeoffs behind the decisions that they make are unclear. This paper presents an approach to explore quality-attribute tradeoffs via clustering and visualization techniques. The approach allows architects to get an overview of the main tradeoffs and their links to architectural configurations. We evaluated the approach in a think-aloud study with 9 participants from academia and industry. Our findings show that the proposed techniques can be useful in understanding feasible tradeoffs and architectural changes affecting those tradeoffs in large design spaces.

Abstract: Self-adaptive systems typically operate in heterogeneous environments and need to optimize their behavior based on a variety of quality attributes to meet stakeholders' needs. During adaptation planning, these quality attributes are considered in the form of constraints, describing requirements that must be fulfilled, and utility functions, which are used to select an optimal plan among several alternatives. Up until now, most automated planning approaches are not designed to adapt quality attributes, their priorities, and their trade-offs at run time. Instead, both utility functions and constraints are commonly defined at design time. There exists a clear lack of run-time mechanisms that support their adaptation in response to changes in the environment or in stakeholders' preferences. In this paper, we present initial work that combines automated planning and adaptation of quality attributes to address this gap. The approach helps to semi-automatically adjust utility functions and constraints based on changes at run time. We present a preliminary experimental evaluation that indicates that our approach can provide plans with higher utility values while fulfilling changed or added constraints. We conclude this paper with our envisioned research outlook and plans for future empirical studies.

Abstract: [Context and motivation:] For realistic self-adaptive systems, multiple quality attributes need to be considered and traded off against each other. These quality attributes are commonly encoded in a utility function, for instance, a weighted sum of relevant objectives. [Question/problem:] The research agenda for requirements engineering for self-adaptive systems has raised the need for decision-making techniques that consider the trade-offs and priorities of multiple objectives. Human stakeholders need to be engaged in the decision-making process so that the relative importance of each objective can be correctly elicited. [Principal ideas/results:] This research preview paper presents a method that supports multiple stakeholders in prioritizing relevant quality attributes, negotiating priorities to reach an agreement, and giving input to define utility functions for self-adaptive systems. [Contribution:] The proposed method constitutes a lightweight solution for utility function definition. It can be applied by practitioners and researchers who aim to develop self-adaptive systems that meet stakeholders' requirements. We present details of our plan to study the application of our method using a case study.