Abstract: Microservice developers increasingly use resiliency patterns such as Retry and Circuit Breaker to cope with remote services that are likely to fail. However, there is still little research on how the invocation delays typically introduced by those resiliency patterns may impact application performance under varying workloads and failure scenarios. This paper presents a novel approach and benchmark tool for experimentally evaluating the performance impact of existing resiliency patterns in a controlled setting. The main novelty of this approach resides in the ability to declaratively specify and automatically generate multiple testing scenarios involving different resiliency patterns, which one can implement using any programming language and resilience library. The paper illustrates the benefits of the proposed approach and tool by reporting on an experimental study of the performance impact of the Retry and Circuit Breaker resiliency patterns in two mainstream programming languages (C# and Java) using two popular resilience libraries (Polly and Resilience4j), under multiple service workloads and failure rates. Our results show that, under low to moderate failure rates, both resiliency patterns effectively reduce the load over the application's target service with barely any impact on the application's performance. However, as the failure rate increases, both patterns significantly degrade the application's performance, with their effect varying depending on the service's workload and the patterns' programming language and resilience library.

Abstract: Mobile malware is a malicious code specifically designed to target mobile devices to perform multiple types of fraud. The number of attacks reported each day is increasing constantly and is causing an impact not only at the end-user level but also at the network operator level. Malware like FluBot contributes to identity theft and data loss but also enables remote Command & Control (C2) operations, which can instrument infected devices to conduct Distributed Denial of Service (DDoS) attacks. Current mobile device-installed solutions are not effective, as the end user can ignore security warnings or install malicious software. This article designs and evaluates MONDEO-Tactics5G-a multistage botnet detection mechanism that does not require software installation on end-user devices, together with tactics for 5G network operators to manage infected devices. We conducted an evaluation that demonstrates high accuracy in detecting FluBot malware, and in the different adaptation strategies to reduce the risk of DDoS while minimising the impact on the clients' satisfaction by avoiding disrupting established sessions.

Abstract: Building and operating software-intensive systems often involves exploring decision spaces made up of large numbers of variables and complex relations among them. Understanding such spaces is often overwhelming to human decision makers, who have limited capacity to digest large amounts of information, making it difficult to distinguish the forest through the trees. In this article, we report on our experience in which we used dimensionality reduction techniques to enable decision makers in different domains (software architecture, smart manufacturing, automated planning for service robots) to focus on the elements of the decision space that explain most of the quality variation, filtering out noise, and thus reducing cognitive complexity.

Abstract: Robotic systems have subsystems with a combinatorially large configuration space and hundreds or thousands of possible software and hardware configuration options interacting non-trivially. The configurable parameters are set to target specific objectives, but they can cause functional faults when incorrectly configured. Finding the root cause of such faults is challenging due to the exponentially large configuration space and the dependencies between the robot's configuration settings and performance. This paper proposes CaRE - a method for diagnosing the root cause of functional faults through the lens of causality. CaRE abstracts the causal relationships between various configuration options and the robot's performance objectives by learning a causal structure and estimating the causal effects of options on robot performance indicators. We demonstrate CaRE's efficacy by finding the root cause of the observed functional faults and validating the diagnosed root cause by conducting experiments in both physical robots (Husky and Turtlebot 3) and in simulation (Gazebo). Furthermore, we demonstrate that the causal models learned from robots in simulation (e. g., Husky in Gazebo) are transferable to physical robots across different platforms (e. g., Husky and Turtlebot 3).

Abstract: Although tests are traditionally used as mandatory summative assessments at the end of the instruction period and their purpose is measuring how well a student has learned a given subject, they can also play an important role in formative assessment when they are employed as an instrument to aid in the acquisition and retention of knowledge. This study evaluates the role of tests as weekly formative assessments and investigates the impact of their repetition (i.e., administering the same test more than once) on academic performance of students in a postgraduate formal methods course for professional software engineers. Our hypothesis is that repetition in formative assessments leads to increased academic performance, as well as to an improvement in the effectiveness of gauging student progress. Based on an analysis of data from a population of 194 students that took the course during the period 2013-2017, we report results that point at the validity of our hypothesis.

Abstract: In software design, guaranteeing the correctness of run-time system behavior while achieving an acceptable balance among multiple quality attributes remains a challenging problem. Moreover, providing guarantees about the satisfaction of those requirements when systems are subject to uncertain environments is even more challenging. While recent developments in architectural analysis techniques can assist architects in exploring the satisfaction of quantitative guarantees across the design space, existing approaches are still limited because they do not explicitly link design decisions to satisfaction of quality requirements. Furthermore, the amount of information they yield can be overwhelming to a human designer, making it difficult to see the forest for the trees. In this paper we present ExTrA (Explaining Tradeoffs of software Architecture design spaces), an approach to analyzing architectural design spaces that addresses these limitations and provides a basis for explaining design tradeoffs. Our approach employs dimensionality reduction techniques employed in machine learning pipelines like Principal Component Analysis (PCA) and Decision Tree Learning (DTL) to enable architects to understand how design decisions contribute to the satisfaction of extra-functional properties across the design space. Our results show feasibility of the approach in two case studies and evidence that combining complementary techniques like PCA and DTL is a viable approach to facilitate comprehension of tradeoffs in poorly-understood design spaces.

Abstract: Self-adaptive systems commonly operate in heterogeneous contexts and need to consider multiple quality attributes. Human stakeholders often express their quality preferences by defining utility func-tions, which are used by self-adaptive systems to automatically generate adaptation plans. However, the adaptation space of realistic systems is large and it is obscure how utility functions impact the generated adaptation behavior, as well as structural, behavioral, and quality constraints. Moreover, human stakeholders are often not aware of the underlying tradeoffs between quality attributes. To address this issue, we present an approach that uses machine learning techniques (dimensionality reduction, clustering, and decision tree learning) to explain the reasoning behind automated planning. Our approach focuses on the tradeoffs between quality attributes and how the choice of weights in utility functions results in different plans being generated. We help humans understand quality attribute tradeoffs, identify key decisions in adaptation behavior, and explore how differences in utility functions result in different adaptation alternatives. We present two systems to demonstrate the approach's applicability and consider its potential application to 24 exemplar self-adaptive systems. Moreover, we describe our assessment of the tradeoff between the information reduction and the amount of explained variance retained by the results obtained with our approach.

Abstract: The problem of mitigating uncertainty in self-adaptation has driven much of the research proposed in the area of software engineering for self-adaptive systems in the last decade. Although many solutions have already been proposed, most of them tend to tackle specific types, sources, and dimensions of uncertainty (e.g., in goals, resources, adaptation functions) in isolation. A special concern are the aspects associated with uncertainty modeling in an integrated fashion. Different uncertainties are rarely independent and often compound, affecting the satisfaction of goals and other system properties in subtle and often unpredictable ways. Hence, there is still limited understanding about the specific ways in which uncertainties from various sources interact and ultimately affect the properties of self-adaptive, software-intensive systems. In this SoSym expert voice, we introduce the Uncertainty Interaction Problem as a way to better qualify the scope of the challenges with respect to representing different types of uncertainty while capturing their interaction in models employed to reason about self-adaptation. We contribute a characterization of the problem and discuss its relevance in the context of case studies taken from two representative application domains. We posit that the Uncertainty Interaction Problem should drive future research in software engineering for autonomous and self-adaptive systems, and therefore, contribute to evolving uncertainty modeling towards holistic approaches that would enable the construction of more resilient self-adaptive systems.

Abstract: Designing software in a way that guarantees run-time behavior while achieving an acceptable balance among multiple quality attributes is an open problem. Providing guarantees about the satisfaction of the same requirements under uncertain environments is even more challenging. Tools and techniques to inform engineers about poorly-understood design spaces in the presence of uncertainty are needed, so that engineers can explore the design space, especially when tradeoffs are crucial. To tackle this problem, we describe an approach that combines synthesis of spaces of system design alternatives from formal specifications of architectural styles with probabilistic formal verification. The main contribution of this paper is a formal framework for specification-driven synthesis and analysis of design spaces that provides formal guarantees about the correctness of system behaviors and satisfies quantitative properties (e.g., defined over system qualities) subject to uncertainty, which is treated as a first-class entity. We illustrate our approach in two case studies: a service-based adaptive system and a mobile robotics architecture. Our results show how the framework can provide useful insights into how average case probabilistic guarantees can differ from worst case guarantees, emphasizing the relevance of combining quantitative formal verification methods with structural synthesis, in contrast with techniques based on simulation and dynamic analysis that can only provide estimates about average case probabilistic properties.

Abstract: We have developed model-based adaptation, an approach that leverages models of software and its environment to enable automated adaptation. The goal of our approach is to build long-lasting mobile software systems that can effectively adapt to changes in their environment.

Abstract: Adaptive systems are expected to adapt to unanticipated run-time events using imperfect information about themselves, their environment, and goals. This entails handling the effects of uncertainties in decision-making, which are not always considered as a first-class concern. This paper contributes a formal analysis technique that explicitly considers uncertainty in sensing when reasoning about the best way to adapt, together with uncertainty reduction mechanisms to improve system utility. We illustrate our approach on a Denial of Service (DoS) attack scenario and present results that demonstrate the benefits of uncertainty-aware decision-making in comparison to using an uncertainty-ignorant approach, both in the presence and absence of uncertainty reduction mechanisms.

Abstract: Self-adaptation improves the resilience of software-intensive systems, enabling them to adapt their structure and behavior to run-time changes (e.g., in workload and resource availability). Many of these approaches reason about the best way of adapting by synthesizing adaptation plans online via planning or model checking tools. This method enables the exploration of a rich solution space, but optimal solutions and other guarantees (e.g., constraint satisfaction) are computationally costly, resulting in long planning times during which changes may invalidate plans. An alternative to online planning involves selecting at run time the adaptation best suited to the current system and environment conditions from among a predefined repertoire of adaptation strategies that capture repair and optimization tasks. This method does not incur run-time overhead but requires additional effort from engineers, who have to specify strategies and lack support to systematically assess their quality. In this article, we present MOSAICO, an approach for offline synthesis of adaptation strategy repertoires that makes a novel use of discrete abstractions of the state space to flexibly adapt extra-functional behavior in a scalable manner. The approach supports making trade-offs: (i) among multiple extra-functional concerns, and (ii) between computation time and adaptation quality (varying abstraction resolution). Our results show a remarkable improvement on system qualities in contrast to manually-specified repertoires. More interestingly, moderate increments in abstraction resolution can lead to pronounced quality improvements, whereas high resolutions yield only negligible improvement over medium resolutions.

Abstract: Proactive latency-aware adaptation is an approach for self-adaptive systems that considers both the current and anticipated adaptation needs when making adaptation decisions, taking into account the latency of the available adaptation tactics. Since this is a problem of selecting adaptation actions in the context of the probabilistic behavior of the environment, Markov decision processes (MDP) are a suitable approach. However, given all the possible interactions between the different and possibly concurrent adaptation tactics, the system, and the environment, constructing the MDP is a complex task. Probabilistic model checking has been used to deal with this problem, but it requires constructing the MDP every time an adaptation decision is made to incorporate the latest predictions of the environment behavior. In this article, we describe PLA-SDP, an approach that eliminates that run-time overhead by constructing most of the MDP offine. At run time, the adaptation decision is made by solving the MDP through stochastic dynamic programming, weaving in the environment model as the solution is computed. We also present extensions that support different notions of utility, such as maximizing reward gain subject to the satisfaction of a probabilistic constraint, making PLA-SDP applicable to systems with different kinds of adaptation goals.

Abstract: An increasingly important requirement for certain classes of software-intensive systems is the ability to self-adapt their structure and behavior at run-time when reacting to changes that may occur to the system, its environment, or its goals. A major challenge related to self-adaptive software systems is the ability to provide assurances of their resilience when facing changes. Since in these systems, the components that act as controllers of a target system incorporate highly complex software, there is the need to analyze the impact that controller failures might have on the services delivered by the system. In this paper, we present a novel approach for evaluating the resilience of self-adaptive software systems by applying robustness testing techniques to the controller to uncover failures that can affect system resilience. The approach for evaluating resilience, which is based on probabilistic model checking, quantifies the probability of satisfaction of system properties when the target system is subject to controller failures. The feasibility of the proposed approach is evaluated in the context of an industrial middleware system used to monitor and manage highly populated networks of devices, which was implemented using the Rainbow framework for architecture-based self-adaptation.

Abstract: Self-adaptive systems must decide which adaptations to apply and when. In reactive approaches, adaptations are chosen and executed after some issue in the system has been detected (e.g., unforeseen attacks or failures). In proactive approaches, predictions are used to prepare the system for some future event (e.g., traffic spikes during holidays). In both cases, the choice of adaptation is based on the estimated impact it will have on the system. Current decision-making approaches assume that the impact will be instantaneous, whereas it is common that adaptations take time to produce their impact. Ignoring this latency is problematic because adaptations may not achieve their effect in time for a predicted event. Furthermore, lower impact but quicker adaptations may be ignored altogether, even if over time the accrued impact is actually higher. In this article, we introduce a novel approach to choosing adaptations that considers these latencies. To show how this improves adaptation decisions, we use a two-pronged approach: (i) model checking of Stochastic Multiplayer Games (SMGs) enables us to understand best- and worst-case scenarios of optimal latency-aware and non-latency-aware adaptation without the need to develop specific adaptation algorithms. However, since SMGs do not provide an algorithm to make choices at runtime, we propose a (ii) latency-aware adaptation algorithm to make decisions at runtime. Simulations are used to explore more detailed adaptation behavior and to check if the performance of the algorithm falls within the bounds predicted by SMGs. Our results show that latency awareness improves adaptation outcomes and also allows a larger set of adaptations to be exploited.

Abstract: Self-adaptive systems have the ability to adapt their behavior to dynamic operating conditions. In reaction to changes in the environment, these systems determine the appropriate corrective actions based in part on information about which action will have the best impact on the system. Existing models used to describe the impact of adaptations are either unable to capture the underlying uncertainty and variability of such dynamic environments, or are not compositional and described at a level of abstraction too low to scale in terms of specification effort required for non-trivial systems. In this paper, we address these shortcomings by describing an approach to the specification of impact models based on architectural system descriptions, which at the same time allows us to represent both variability and uncertainty in the outcome of adaptations, hence improving the selection of the best corrective action. The core of our approach is a language equipped with a formal semantics defined in terms of Discrete Time Markov Chains that enables us to describe both the impact of adaptation tactics, as well as the assumptions about the environment. To validate our approach, we show how employing our language can improve the accuracy of predictions used for decision-making in the Rainbow framework for architecture-based self-adaptation.

Abstract: Complex software-intensive systems are increasingly relied upon for all kinds of activities in society, leading to the requirement that these systems should be resilient to changes that may occur to the system, its environment, or its goals. Traditionally, resilience has been achieved either through: (i) low-level mechanisms embedded in the implementation (e.g., exception handling, timeouts, redundancies), which are unable to detect subtle but important anomalies (e.g., progressive performance degradation); or (ii) human oversight, which is costly and unreliable. Architecture-based self-adaptation (ABSA) is regarded as a promising approach to improve the resilience and reduce the development/operation costs of such systems. Although researchers have illustrated the benefits of ABSA through a number of small-scale case studies, it remains to be seen whether ABSA is truly effective in handling changes at run-time in industrial-scale systems. In this paper, we report on our experience applying an ABSA framework (Rainbow) to a large-scale commercial software system, called Data Acquisition and Control Service (DCAS), which is used to monitor and manage highly populated networks of devices in renewable energy production plants. In the approach followed, we have replaced some of the existing adaptive mechanisms embedded in DCAS by those advocated by ABSA proponents. This has allowed us to assess the development costs associated with the reengineering of adaptive mechanisms when using an ABSA solution, and to make effective comparisons, in terms of operational performance, between a baseline industrial system and one that uses ABSA. Our results show that using the ABSA concepts as embodied in Rainbow enabled an independent team of developers to: (i) effectively implement the adaptation behavior required from such industrial systems; and (ii) obtain important benefits in terms of maintainability and extensibility of adaptation mechanisms.

Abstract: Design decisions made during early development stages of self-adaptive systems tend to have a significant impact upon system properties at runtime (e.g., safety, QoS). However, understanding the implications of these decisions a priori is difficult due to the different types and degrees of uncertainty that affect such systems (e.g., simplifying assumptions, human-in-the-loop). To provide some assurances about self-adaptive system designs, evidence can be gathered from activities such as simulations and prototyping, but these demand a significant effort and do not provide a systematic way of dealing with uncertainty. In this chapter, we describe an approach based on model checking of stochastic multiplayer games (SMGs) that enables developers to approximate the behavioral envelope of a self-adaptive system by analyzing best- and worst-case scenarios of alternative designs for self-adaptation mechanisms. Compared to other sources of evidence, such as simulations or prototypes, our approach provides developers with a preliminary understanding of adaptation behavior with less effort, and without the need to have any specific adaptation algorithms or infrastructure in place. We illustrate our approach by showing how it can be used to mitigate different types of uncertainty in contexts such as self-protecting systems, proactive latency-aware adaptation, and human-in-the-loop.

Abstract: Providing assurances for self-adaptive systems is challenging. A primary underlying problem is uncertainty that may stem from a variety of different sources, ranging from incomplete knowledge to sensor noise and uncertain behavior of humans in the loop. Providing assurances that the self-adaptive system complies with its requirements calls for an enduring process spanning the whole lifetime of the system. In this process, humans and the system jointly derive and integrate new evidence and arguments, which we coined perpetual assurances for self-adaptive systems. In this paper, we provide a background framework and the foundation for perpetual assurances for self-adaptive systems. We elaborate on the concrete challenges of offering perpetual assurances, requirements for solutions, realization techniques and mechanisms to make solutions suitable. We also present benchmark criteria to compare solutions. We then present a concrete exemplar that researchers can use to assess and compare approaches for perpetual assurances for self-adaptation

Abstract: Software systems are increasingly called upon to autonomously manage their goals in changing contexts and environments, and under evolving requirements. In some circumstances, autonomous systems cannot be fully automated but instead cooperate with human operators to maintain and adapt themselves. Furthermore, there are times when a choice should be made between doing a manual or automated repair. Involving operators in self-adaptation should itself be adaptive, and consider aspects such as the training, attention, and ability of operators. Not only do these aspects change from person to person, but they may change with the same person. These aspects make the choice of whether to involve humans non-obvious. Self-adaptive systems should trade-off whether to involve operators, taking these aspects into consideration along with other business qualities it is attempting to achieve. In this chapter, we identify the various roles that operators can perform in cooperating with self-adaptive systems. We focus on humans as effectors - doing tasks which are difficult or infeasible to automate. We describe how we modified our self-adaptive framework, Rainbow, to involve operators in this way, which involved choosing suitable human models and integrating them into the existing utility trade-off decision models of Rainbow. We use probabilistic modeling and quantitative verification to analyze the trade-offs of involving humans in adaptation, and complement our study with experiments to show how different business preferences and modalities of human involvement may result in different outcomes.

Abstract: The emergence of powerful automated design tools in many domains is changing the nature of design, as human-intensive activities can be increasingly off-loaded to those tools. Rather than having a human consider only handful of options, as has been done historically, such tools now enable the generation of a large space of potential designs, exhibiting different tradeoffs among competing qualities of merit, and supporting systematic exploration of the design space. At the same time, this paradigm raises new challenges centered on enabling humans to effectively navigate that generated space in order to select a design that best meets their requirements. In this paper we describe our experience in the domain of generative manufacturing, in which we developed a novel design environment for airplane parts manufacturing that incorporates a number of sophisticated design tools and attempts to tackle the emergent problems of design space exploration that are faced by designers of those parts. We use this experience to highlight the challenges that we faced and reflect on their applicability more generally to tool-assisted software design environments.

Abstract: Machine Learning (ML) is increasingly used in domains such as cyber-physical systems and enterprise systems. These systems typically operate in non-static environments, prone to unpredictable changes that can adversely impact the accuracy of the ML models, which are usually in the critical path of the systems. Mispredictions of ML components can thus affect other components in the system, and ultimately impact overall system utility in non-trivial ways. From this perspective, self-adaptation techniques appear as a natural solution to reason about how to react to environment changes via adaptation tactics that can potentially improve the quality of ML models (e.g., model retrain), and ultimately maximize system utility. However, adapting ML components is non-trivial, since adaptation tactics have costs and it may not be clear in a given context whether the benefits of ML adaptation outweigh its costs. In this paper, we present a formal probabilistic framework, based on model checking, that incorporates the essential governing factors for reasoning at an architectural level about adapting ML classifiers in a system context. The proposed framework can be used in a self-adaptive system to create adaptation strategies that maximize rewards of a multidimensional utility space. Resorting to a running example from the enterprise systems domain, we show how the proposed framework can be employed to determine the gains achievable via ML adaptation and to find the boundary that renders adaptation worthwhile.

Abstract: In software design, guaranteeing the correctness of run-time system behavior while achieving an acceptable balance among multiple quality attributes remains a challenging problem. Moreover, providing guarantees about the satisfaction of those requirements when systems are subject to uncertain environments is even more challenging. While recent developments in architectural analysis techniques can assist architects in exploring the satisfaction of quantitative guarantees across the design space, existing approaches are still limited because they do not explicitly link design decisions to satisfaction of quality requirements. Furthermore, the amount of information they yield can be overwhelming to a human designer, making it difficult to distinguish the forest through the trees. In this paper, we present an approach to analyzing architectural design spaces that addresses these limitations and provides a basis to enable the explainability of design tradeoffs. Our approach combines dimensionality reduction techniques employed in machine learning pipelines with quantitative verification to enable architects to understand how design decisions contribute to the satisfaction of strict quantitative guarantees under uncertainty across the design space. Our results show feasibility of the approach in two case studies and evidence that dimensionality reduction is a viable approach to facilitate comprehension of tradeoffs in poorly-understood design spaces.

Abstract: Many self-adaptive systems benefit from human involvement, where a human operator can provide expertise not available to the system and perform adaptations involving physical changes that cannot be automated. However, a lack of transparency and intelligibility of system goals and the autonomous behaviors enacted to achieve them may hinder a human operator's effort to make such involvement effective. Explanation is sometimes helpful to allow the human to understand why the system is making certain decisions. However, explanations come with costs in terms of, e.g., delayed actions. Hence, it is not always obvious whether explanations will improve the satisfaction of system goals and, if so, when to provide them to the operator. In this work, we define a formal framework for reasoning about explanations of adaptive system behaviors and the conditions under which they are warranted. Specifically, we characterize explanations in terms of their impact on a human operator's ability to effectively engage in adaptive actions. We then present a decision-making approach for planning in self-adaptation that leverages a probabilistic reasoning tool to determine when the explanation should be used in an adaptation strategy in order to improve overall system utility. We illustrate our approach in a representative scenario for the application of an adaptive news website in the context of potential denial-of-service attacks.

Abstract: Self-adaptive systems increasingly need to reason about and adapt both structural and behavioral system aspects, such as in mobile service robots, which must reason about missions that they need to achieve and the architecture of the software executing them. Deciding how to best adapt these systems to run time changes is challenging because it entails considering mutual dependencies between the software architecture that the system is running and the outcome of plans for completing tasks, while also considering multiple trade-offs and uncertainties. Considering all these aspects in planning for adaptation often yields large solution spaces which cannot be adequately explored at run time. We address this challenge by proposing a planning approach able to consider the impact of mutual dependencies between software architecture and task planning on the satisfaction of mission goals. The approach is able to reason quantitatively about the outcome of adaptation decisions handling both the reconfiguration of the system's architecture and adaptation of task plans under uncertainty and in a rich trade-off space. Our results show: (i) feasibility of run-time decision-making for self-adaptation in an otherwise intractable solution space by dividing-and-conquering adaptation into architecture reconfiguration and task planning sub-problems, and (ii) improved quality of adaptation decisions with respect to decision making that does not consider dependencies between architecture and task planning.

Abstract: Two of the main paradigms used to build adaptive software employ different types of properties to capture relevant aspects of the system’s run-time behavior. On the one hand, control systems consider properties that concern static aspects like stability, as well as dynamic properties that capture the transient evolution of variables such as settling time. On the other hand, self-adaptive systems consider mostly non-functional properties that capture concerns such as performance, reliability, and cost. In general, it is not easy to reconcile these two types of properties or identify under which conditions they constitute a good fit to provide run-time guarantees. There is a need of identifying the key properties in the areas of control and self-adaptation, as well as of characterizing and mapping them to better understand how they relate and possibly complement each other. In this paper, we take a first step to tackle this problem by: (1) identifying a set of key properties in control theory, (2) illustrating the formalization of some of these properties employing temporal logic languages commonly used to engineer self-adaptive software systems, and (3) illustrating how to map key properties that characterize self-adaptive software systems into control properties, leveraging their formalization in temporal logics. We illustrate the different steps of the mapping on an exemplar case in the cloud computing domain and conclude with identifying open challenges in the area.

Abstract: Microservice application developers try to mitigate the impact of partial outages typically by implementing service-to-service interactions that use well-known resiliency patterns, such as Retry, Fail Fast, and Circuit Breaker. However, those resiliency patterns–as well as their available open-source implementations–are often documented informally, leaving it up to application developers to figure out when and how to use those patterns in the context of a particular microservice application. In this paper, we take a first step towards improving on this situation by introducing a model checking-based approach in which we use the PRISM probabilistic model checker to analyze the behavior of the Retry and Circuit Breaker resiliency patterns as continuous-time Markov chains (CTMC). This approach has enabled us to quantify the impact of applying each resiliency pattern on multiple quality attributes, as well as to determine how to best tune their parameters to deal with varying service availability conditions, in the context of a simple client-service interaction scenario.

Abstract: Modern cyber-physical systems (e.g., robotics systems) are typically composed of physical and software components, the characteristics of which are likely to change over time. Assumptions about parts of the system made at design time may not hold at run time, especially when a system is deployed for long periods (e.g., over decades). Self-adaptation is designed to find reconfigurations of systems to handle such run-time inconsistencies. Planners can be used to find and enact optimal reconfigurations in such an evolving context. However, for systems that are highly configurable, such planning becomes intractable due to the size of the adaptation space. To overcome this challenge, in this paper we explore an approach that (a) uses machine learning to find Pareto-optimal configurations without needing to explore every configuration, and (b) restricts the search space to such configurations to make planning tractable. We explore this in the context of robot missions that need to consider task timeliness and energy consumption. An independent evaluation shows that our approach results in high quality adaptation plans in uncertain and adversarial environments.

Abstract: Why is it so difficult to build self-adaptive systems by reusing existing self-adaptation services and frameworks? In this paper, we argue that one possible explanation is that there is a fundamental mismatch between the adaptation needs of modern software systems, and the architectural models and adaptation mechanisms supported by current self-adaptation solutions. We identify and discuss the main reasons leading to this problem by looking into a number of representative self-adaptation solutions that have been proposed in recent years, including open source frameworks and cloud-based services, from two perspectives: generality, i.e., their ability to support a variety of architectural models and adaptation mechanisms, and reusability, i.e., their ability to be reused without requiring substantial effort from software developers. We then make the case that recent industry progress toward microservices and their enabling technologies can open the way to the development of more general and reusable self-adaptation solutions.

Abstract: Self-adaptive systems depend on models of themselves and their environment to decide whether and how to adapt, but these models are often affected by uncertainty. While current adaptation decision approaches are able to model and reason about this uncertainty, they do not consider ways to reduce it. This presents an opportunity for improving decision-making in self-adaptive systems, because reducing uncertainty results in a better characterization of the current and future states of the system and the environment (at some cost), which in turn supports making better adaptation decisions. We propose uncertainty reduction as the natural next step in uncertainty management in the field of self-adaptive systems. This requires both an approach to decide when to reduce uncertainty, and a catalog of tactics to reduce different kinds of uncertainty. We present an example of such a decision, examples of uncertainty reduction tactics, and describe how uncertainty reduction requires changes to the different activities in the typical self-adaptation loop.

Abstract: Modern computing systems are often formed by multiple components that interact with each other through the use of shared resources (e.g., CPU, network bandwidth, storage). In this paper, we consider a representative scenario of one such system in the context of an Internet of Things application. The system consists of a network of self-adaptive cameras that share a communication channel, transmitting streams of frames to a central node. The cameras can modify a quality parameter to adapt the amount of information encoded and to affect their bandwidth requirements and usage. A critical design choice for such a system is scheduling channel access, i.e., how to determine the amount of channel capacity that should be used by each of the cameras at any point in time. Two main issues have to be considered for the choice of a bandwidth allocation scheme: (i) camera adaptation and network access scheduling may interfere with one another, (ii) bandwidth distribution should be triggered only when necessary, to limit additional overhead. This paper proposes the first formally verified event-triggered adaptation scheme for bandwidth allocation, designed to minimize additional overhead in the network. Desired properties of the system are verified using model checking. The paper also describes experimental results obtained with an implementation of the scheme.

Abstract: Self-Adaptive systems are expected to adapt to unanticipated run-time events using imperfect information about their environment. This entails handling the effects of uncertainties in decision-making, which are not always considered as a first-class concern. This paper contributes a formal analysis technique that explicitly considers uncertainty in sensing when reasoning about the best way to adapt. We illustrate our approach on a Denial of Service (DoS) attack scenario and present some preliminary results that show the benefits of uncertainty-aware decision-making with respect to using an uncertainty-ignorant approach.

Abstract: Modern software-intensive systems must often guarantee certain quality requirements under changing run-time conditions and high levels of uncertainty. Self-adaptation has proven to be an effective way to engineer systems that can address such challenges, but many of these approaches are purely reactive and adapt only after a failure has taken place. To overcome some of the limitations of reactive approaches (e.g., lagging behind environment changes and favoring short-term improvements), recent proactive self-adaptation mechanisms apply ideas from control theory, such as model predictive control (MPC), to improve adaptation. When selecting which MPC approach to apply, the improvement that can be obtained with each approach is scenario-dependent, and so guidance is needed to better understand how to choose an approach for a given situation. In this paper, we compare CobRA and PLA, two approaches that are inspired by MPC. CobRA is a requirements-based approach that applies control theory, whereas PLA is architecture-based and applies stochastic analysis. We compare the two approaches applied to RUBiS, a benchmark system for web and cloud application performance, discussing the required expertise needed to use both approaches and comparing their run-time performance with respect to different metrics.

Abstract: Approaches to self-adaptation face a fundamental trade-off between quality and timeliness in decision-making. Due to this trade-off, designers of self-adaptive systems often have to find a fixed and suboptimal compromise between these two requirements. Recent work has proposed the hybrid planning approach that can resolve this trade-off dynamically and potentially in an optimal way. The promise of hybrid planning is to combine multiple planners at run time to produce adaptation plans of the highest quality within given time constraints. However, since decision-making approaches are complex and diverse, the problem of combining them is even more difficult, and no frameworks for hybrid planning. This paper makes an important step in simplifying the problem of hybrid planning by formalizing it and decomposing it into four simpler subproblems. These formalizations will serve as a foundation for creating and evaluating engineering solutions to the hybrid planning problem.

Abstract: Large software systems have to contend with a significant number of users who interact with different components of the system in various ways. The sequences of components that are used as part of an interaction define sets of behaviors that users have with the system. These can be large in number. Among these users, it is possible that there are some who exhibit anomalous behaviors - for example, they may have found back doors into the system and are doing something malicious. These anomalous behaviors can be hard to distinguish from normal behavior because of the number of interactions a system may have, or because traces may deviate only slightly from normal behavior. In this paper we describe a model-based approach to cluster sequences of user behaviors within a system and to find suspicious, or anomalous, sequences. We exploit the underlying software architecture of a system to define these sequences. We further show that our approach is better at detecting suspicious activities than other approaches, specifically those that use unigrams and bigrams for anomaly detection. We show this on a simulation of a large scale system based on Amazon Web application style architecture.

Abstract: Software architecture modeling is important for analyzing system quality attributes, particularly security. However, such analyses often assume that the architecture is completely known in advance. In many modern domains, especially those that use plugin-based frameworks, it is not possible to have such a complete model because the software system continuously changes. The Android mobile operating system is one such framework, where users can install and uninstall apps at run time. We need ways to model and analyze such architectures that strike a balance between supporting the dynamism of the underlying platforms and enabling analysis, particularly throughout a system�s lifetime. In this paper, we describe a formal architecture style that captures the modifiable architectures of Android systems, and that supports security analysis as a system evolves. We illustrate the use of the style with two security analyses: a predicatebased approach defined over architectural structure that can detect some common security vulnerabilities, and inter-app permission leakage determined by model checking. We also show how the evolving architecture of an Android device can be obtained by analysis of the apps on a device, and provide some performance evaluation that indicates that the architecture can be amenable for use throughout the system�s lifetime.

Abstract: Run-time generation of adaptation plans is a powerful mechanism that helps a self-adaptive system to meet its goals in a dynamically changing environment. In the past, researchers have demonstrated successful use of various automated planning techniques to generate adaptation plans at run time. However, for a planning technique, there is often a trade-off between timeliness and optimality of the solution. For some self-adaptive systems, ideally, one would like to have a planning approach that is both quick and finds an optimal adaptation plan. To find the right balance between these conflicting requirements, this paper introduces a hybrid planning approach that combines more than one planner to obtain the benefits of each. In this paper, to instantiate a hybrid planner we combine deterministic planning with Markov Decision Process (MDP) planning to obtain the best of both worlds: deterministic planning provides plans quickly when timeliness is critical, while allowing MDP planning to generate optimal plans when the system has sufficient time to do so. We validate the hybrid planning approach using a realistic workload pattern in a simulated cloud-based self-adaptive system.

Abstract: Proactive latency-aware adaptation is an approach for self-adaptive systems that improves over reactive adaptation by considering both the current and anticipated adaptation needs of the system, and taking into account the latency of adaptation tactics so that they can be started with the necessary lead time. Making an adaptation decision with these characteristics requires solving an optimization problem to select the adaptation path that maximizes an objective function over a finite look-ahead horizon. Since this is a problem of selecting adaptation actions in the context of the probabilistic behavior of the environment, Markov decision processes (MDP) are a suitable approach. However, given all the possible interactions between the different and possibly concurrent adaptation tactics, the system, and the environment, constructing the MDP is a complex task. Probabilistic model checking can be used to deal with this problem since it takes as input a formal specification of the stochastic system, which is internally translated into an MDP, and solved. One drawback of this solution is that the MDP has to be constructed every time an adaptation decision has to be made to incorporate the latest predictions of the environment behavior. In this paper we present an approach that eliminates that run-time overhead by constructing most of the MDP offline, also using formal specification. At run time, the adaptation decision is made by solving the MDP through stochastic dynamic programming, weaving in the stochastic environment model as the solution is computed. Our experimental results show that this approach reduces the adaptation decision time by an order of magnitude compared to the probabilistic model checking approach, while producing the same results.

Abstract: Planning in CPSs requires temporal reasoning to handle the dynamics of the environment, including human behavior, as well as temporal constraints on system goals and durations of actions that systems and human actors may take. The discrete abstraction of time in a state space planning should have a time sampling parameter value that satisfi es some relation to achieve a certain precision. In particular, the sampling period should be small enough to allow the dynamics of the problem domain to be modeled with sufficient precision. Meanwhile, in many cases, events in the far future (relative to the sampling period) may be relevant to the decision making earlier in the planning timeline; therefore, a longer planning look-ahead horizon can yield a closer-to-optimal plan. Unfortunately, planning with a uniform fine-grained discrete abstraction of time and a long look-ahead horizon is typically computationally infeasible. In this paper, we propose a multiscale temporal planning approach - formulated as MDP planning - to preserve the required time fidelity of the problem domain and at the same time approximate a globally optimal plan. We illustrate our approach in a middleware used to monitor large sensor networks.

Abstract: The term big data arose under the explosive increase of global data as a technology that is able to store and process big and varied volumes of data, providing both enterprises and science with deep insights over its clients/experiments. Cloud computing provides a reliable, fault-tolerant, available and scalable environment to harbour big data distributed management systems. Within the context of this paper we present an overview of both technologies and cases of success when integrating big data and cloud frameworks. Although big data solves much of our current problems it still presents some gaps and issues that raise concern and need improvement. Security, privacy, scalability, data governance policies, data heterogeneity, disaster recovery mechanisms, and other challenges are yet to be addressed. Other concerns are related to cloud computing and its ability to deal with exabytes of information or address exaflop computing efficiently. This paper presents an overview of both cloud and big data technologies describing the current issues with these technologies.

Abstract: In an organization, the interactions users have with software leaves patterns or traces of the parts of the systems accessed. These interactions can be associated with the underlying software architecture. The first step in detecting problems like insider threat is to detect those traces that are anomalous. In this paper, we present a method to find anomalous users leveraging these interaction traces, categorized by user roles. We propose a model based approach to cluster user sequences and find outliers. Such a technique could be useful in finding potentially anomalous users, insiders, or compromised accounts.We show that the approach works on a simulation of a large scale system based on and Amazon Web application style.

Abstract: Modern frameworks are required to be extendable as well as secure. However, these two qualities are often at odds. In this poster we describe an approach that uses a combination of static analysis and run-time management, based on software architecture models, that can improve security while maintaining framework extendability. We implement a prototype of the approach for the Android platform. Static analysis identifi es the architecture and communication patterns among the collection of apps on an Android device and which communications might be vulnerable to attack. Run-time mechanisms monitor these potentially vulnerable communication patterns, and adapt the system to either deny them, request explicit approval from the user, or allow them.

Abstract: Modern software systems are often compositions of entities that increasingly use self-adaptive capabilities to improve their behavior to achieve systematic quality goals. Self-adaptive managers for each component system attempt to provide locally optimal results, but if they cooperated and potentially coordinated their efforts it might be possible to obtain more globally optimal results. The emergent properties that result from such composition and cooperation of self-adaptive systems are not well understood, difficult to reason about, and present a key challenge in the evolution of modern software systems. For example, the effects of coordination patterns and protocols on emergent properties such as the resiliency of the collectives need to be understood when designing these systems. In this paper we propose that probabilistic model checking of stochastic multiplayer games (SMG) provides a promising approach to analyze, understand, and reason about emergent properties in collectives of adaptive systems (CAS). Probabilistic Model Checking of SMGs is a technique particularly suited to analyzing emergent properties in CAS, since SMG models capture: (i) the uncertainty and variability intrinsic to the CAS and its execution environment in the form of probabilistic and nondeterministic choices, and (ii) the competitive/cooperative aspects of the interplay among the constituent systems of the CAS. Analysis of SMGs allows us to reason about things like the worst case scenarios, which constitutes a new contribution to understanding emergent properties in CAS. We investigate the use of SMGs to show how they can be useful in analyzing the impact of communication topology for collections of fully cooperative systems defending against an external attack.

Abstract: Self-adaptive systems tend to be reactive and myopic, adapting in response to changes without anticipating what the subsequent adaptation needs will be. Adapting reactively can result in ineffi?ciencies due to the system performing a suboptimal sequence of adaptations. Furthermore, when adaptations have latency, and take some time to produce their e?ffect, they have to be started with su?fficient lead time so that they complete by the time their eff?ect is needed. Proactive latency-aware adaptation addresses these issues by making adaptation decisions with a look-ahead horizon and taking adaptation latency into account. In this paper we present an approach for proactive latency-aware adaptation under uncertainty that uses probabilistic model checking for adaptation decisions. The key idea is to use a formal model of the adaptive system in which the adaptation decision is left underspeci?fied through nondeterminism, and have the model checker resolve the nondeterministic choices so that the accumulated utility over the horizon is maximized. The adaptation decision is optimal over the horizon, and takes into account the inherent uncertainty of the environment predictions needed for looking ahead. Our results show that the decision based on a look-ahead horizon, and the factoring of both tactic latency and environment uncertainty, considerably improve the eff?ectiveness of adaptation decisions.

Abstract: Self-adaptive systems overcome many of the limitations of human supervision in complex software-intensive systems

by endowing them with the ability to automatically adapt their

structure and behavior in the presence of runtime changes.

However, adaptation in some classes of systems (e.g., safety- critical) can benefit by receiving information from humans (e.g.,

acting as sophisticated sensors, decision-makers), or by involving

them as system-level effectors to execute adaptations (e.g., when

automation is not possible, or as a fallback mechanism). However,

human participants are influenced by factors external to the

system (e.g., training level, fatigue) that affect the likelihood

of success when they perform a task, its duration, or even if

they are willing to perform it in the first place. Without careful

consideration of these factors, it is unclear how to decide when to

involve humans in adaptation, and in which way. In this paper,

we investigate how the explicit modeling of human participants

can provide a better insight into the trade-offs of involving

humans in adaptation. We contribute a formal framework to

reason about human involvement in self-adaptation, focusing on

the role of human participants as actors (i.e., effectors) during

the execution stage of adaptation. The approach consists of:

(i) a language to express adaptation models that capture factors

affecting human behavior and its interactions with the system,

and (ii) a formalization of these adaptation models as stochastic

multiplayer games (SMGs) that can be used to analyze human- system-environment interactions. We illustrate our approach in

an adaptive industrial middleware used to monitor and manage

sensor networks in renewable energy production plants.

Abstract: Architecture-based approaches to self-adaptation rely on architectural descriptions to reason about the best way of adapting the structure and behavior of software-intensive systems at run time, either by choosing among a set of predefined adaptation strategies, or by automatically generating adaptation plans. Predefined strategy selection has allow computational overhead and facilitates dealing with uncertainty (e.g., by accounting explicitly for contingencies derived from unexpected outcomes of actions), but requires additional designer effort regarding the specification of strategies and is unable to guarantee optimal solutions. In contrast, run time plan generation is able to explore a richer solution space and provide optimal solutions in some cases, but is more limited when dealing with uncertainty, and incurs higher computational overheads. In this paper, we propose an approach to optimal adaptation plan generation for architecture-based self-adaptation via model checking of stochastic multiplayer games (SMGs). Our approach enables: (i) trade-off analysis among different qualities by means of utility functions and preferences, and (ii) explicit modeling of uncertainty in the outcome of adaptation actions and the behavior of the environment. Basing on the concepts embodied in the Rainbow framework for self-adaptation, we illustrate our approach in Znn.com, a case study that reproduces the infrastructure for a news website.

Abstract: Self-adaptive systems have the ability to adapt their behavior to dynamic operation conditions. In reaction to changes in the environment, these systems determine the appropriate corrective actions based in part on information about which action will have the best impact on the system. Existing models used to describe the impact of adaptations are either unable to capture the underlying uncertainty and variability of such dynamic environments, or are not compositional and described at a level of abstraction too low to scale in terms of specification effort required for non-trivial systems. In this paper, we address these shortcomings by describing an approach to the specification of impact models based on architectural system descriptions, which at the same time allows us to represent both variability and uncertainty in the outcome of adaptations, hence improving the selection of the best corrective action. The core of our approach is an impact model language equipped with a formal semantics defined in terms of Discrete Time Markov Chains. To validate our approach, we show how employing our language can improve the accuracy of predictions used for decision-making in the Rainbow framework for architecture-based self-adaptation.

Abstract: Architecture-based self-adaptation is considered as a promising approach to drive down the development and opera-tion costs of complex software systems operating in ever changing environments. However, there is still a lack of evidence supporting the arguments for the beneficial impact of architecture-based self-adaptation on resilience with respect to other customary approaches, such as embedded code-based adaptation. In this paper, we report on an empirical study about the impact on resilience of incorporating architecture-based self-adaptation in an industrial middleware used to collect data in highly populated networks of devices. To this end, we compare the results of resilience evaluation between the original version of the middleware, in which adaptation mechanisms are embedded at the code-level, and a modifi ed version of that middleware in which the adaptation mechanisms are implemented using Rainbow, a framework for architecture-based self-adaptation. Our results show improved levels of resilience in architecture-based compared to embedded code-based self-adaptation.

Abstract: Although different approaches to decision-making in self-adaptive systems have shown their effectiveness in the past by factoring in predictions about the system and its environment (e. g., resource availability), no proposal considers the latency associated with the execution of tactics upon the target system. However, different adaptation tactics can take different amounts of time until their eff ects can be observed. In reactive adaptation, ignoring adaptation tactic latency can lead to suboptimal adaptation decisions (e. g., activating a server that takes more time to boot than the transient spike in traffic that triggered its activation). In proactive adaptation, taking adaptation latency into account is necessary to get the system into the desired state to deal with an upcoming situation. In this paper, we introduce a formal analysis technique based on model checking of stochastic multiplayer games (SMGs) that enables us to quantify the potential benefi ts of employing different types of algorithms for self-adaptation. In particular, we apply this technique to show the potential benefi t of considering adaptation tactic latency in proactive adaptation algorithms. Our results show that factoring in tactic latency in decision making improves the outcome of adaptation. We also present an algorithm to do proactive adaptation that considers tactic latency, and show that it achieves higher utility than an algorithm that under the assumption of no latency is optimal.

Abstract: Security features are often hardwired into software applications, making it difficult to adapt security responses to reflect changes in runtime context and new attacks. In prior work, we proposed the idea of architecture-based self-protection as a way of separating adaptation logic from application logic and providing a global perspective for reasoning about security adaptations in the context of other business goals. In this paper, we present an approach, based on this idea, for combating denial-of-service (DoS) attacks. Our approach allows DoS-related tactics to be composed into more sophisticated mitigation strategies that encapsulate possible responses to a security problem. Then, utility-based reasoning can be used to consider diff erent business contexts and qualities. We describe how this approach forms the underpinnings of a scientific approach to self-protection, allowing us to reason about how to make the best choice of mitigation at runtime. Moreover, we also show how formal analysis can be used to determine whether the mitigations cover the range of conditions the system is likely to encounter, and the effect of mitigations on other quality attributes of the system. We evaluate the approach using the Rainbow self-adaptive framework and show how Rainbow chooses DoS mitigation tactics that are sensitive to diff erent business contexts.

Abstract: Although architecture-based self-adaptation has been widely used, there is still little understanding about the validity and tradeoffs of incorporating it into real-world software-intensive systems which already feature built-in adaptation mechanisms. In this paper, we report on our experience in integrating Rainbow, a platform for architecture-based self-adaptation, and an industrial middleware employed to monitor and manage highly populated networks of devices. Concretely, we reflect on aspects such as the effort required for framework customization and legacy code refactoring, performance improvement, and the impact of architecture-based self-adaptation on system evolution.

Abstract: Self-adaptive systems choose adaptations to perform on a running system. Typically, the self-adaptive system must choose the "best" adaptation to perform in a given circumstance from a set of adaptations that may apply. Utility functions are typically used to encode some measure of goodness or badness the result of choosing a particular adaptation. Within the area of utility theory, there are a set of theories that could be used to help choose the best adaptation, that vary in the assumptions and requirements made about the system, the environment, and the business context of use. By understanding some of the formalities and advanced concepts in Utility Theory, engineers and administrators of self-adaptive systems can create more effective utility functions for their purposes, enable new forms of analysis, and potentially move into new frontiers in self-adaptive systems. In this report, we survey some of the more interesting topics in Utility Theory relevant to self-adaptive systems include objective and subjective expected utility, stochastic and fuzzy utility, and changing and state dependent utility functions.

Abstract: Self-Adaptive systems are expected to adapt to unanticipated run-time events using imperfect information about their environment. This entails handling the effects of uncertainties in decision-making, which are not always considered as a first-class concern. This technical report summarizes a set of existing techniques and insights into addressing uncertainty in self-adaptive systems and outlines a future research agenda on uncertainty management in self-adaptive systems. The material in this report is strongly informed by our own research in the area, and is therefore not necessarily representative of other works.

Abstract: The fundamental premise behind Moving Target Defense (MTD) is to create a dynamic and shifting system that is more difficult to attack than a static system because a constantly changing attack surface at least reduces the chance of an attacker finding and exploiting the weakness. However, MTD approaches are typically chosen without regard to other qualities of the system, such as performance or cost. This report explores the use of self-adaptive systems, in particular those based on the architecture of the running system. A systems software architecture can be used to trade off different quality dimensions of the system. In particular, this report describes the first steps in reasoning formally about MTD approaches, and elevating this reasoning to an architectural level, along three thrusts: (1) creating an initial catalog of MTD tactics that can be used at the architectural level, along with the impacts on security and other quality concerns, (2) using this information to inform proactive self-adaptation that uses predictions of tactic duration to improve the self-adaptation, and (3) using stochastic multiplayer games to verify the the behavior of a variety of MTD scenarios, from uninformed to predictive-reactive. This work is applied in the context of the Rainbow self-adaptive framework.

Abstract: Planning approaches in self-adaptation face the fundamental trade-off between quality and timeliness of adaptation plans. Due to this trade-off, today designers often have to compromise between a planning approach that is quick to find a plan and an approach that is slow but finds a quality plan. To deal with this trade-off, researchers have proposed a hybrid planning approach that combines more than one planning approaches to find a balance between quality and timeliness. However, the diversity of planning approaches makes the problem of hybrid planning complex and multi-faceted. This paper advances the theory of hybrid planning by formalizing the central concepts and four subproblems of hybrid planning. This formalization can serve as a foundation for not only understanding and comparing existing hybrid planners, but also for developing new hybrid planners in the future. Moreover, to illustrate practicality of the formal model, the paper uses it to analyze two hybrid planning instantiations that have shown to effective in their respective contexts. Furthermore, grounded on the formal model, the paper formalizes explicit/implicit assumptions about these instantiations that must hold for them to be sound.