Cloud computing provisions scalable resources for high performance industrial applications. Cloud providers usually offer two types of usage plans: reserved and on-demand. Reserved plans offer cheaper resources for long-term contracts while on-demand plans are available for short or long periods but are more expensive. To satisfy incoming user demands with reasonable costs, cloud resources should be allocated efficiently. Most existing works focus on either cheaper solutions with reserved resources that may lead to under-provisioning or over-provisioning, or costly solutions with on-demand resources. Since inefficiency of allocating cloud resources can cause huge provisioning costs and fluctuation in cloud demand, resource allocation becomes a highly challenging problem. In this paper, we propose a hybrid method to allocate cloud resources according to the dynamic user demands. This method is developed as a two-phase algorithm that consists of reservation and dynamic provision phases. In this way, we minimize the total deployment cost by formulating each phase as an optimization problem while satisfying quality of service. Due to the uncertain nature of cloud demands, we develop a stochastic optimization approach by modeling user demands as random variables. Our algorithm is evaluated using different experiments and the results show its efficiency in dynamically allocating cloud resources.

Dynamic Cloud Resource Allocation Considering Demand Uncertainty

Cloud computing heavily relies on virtualization, as with cloud computing virtual resources are typically leased to the consumer, for example as virtual machines. Efficient management of these virtual resources is of great importance, as it has a direct impact on both the scalability and the operational costs of the cloud environment. Recently, containers are gaining popularity as virtualization technology, due to the minimal overhead compared to traditional virtual machines and the offered portability. Traditional resource management strategies however are typically designed for the allocation and migration of virtual machines, so the question arises how these strategies can be adapted for the management of a containerized cloud. Apart from this, the cloud is also no longer limited to the centrally hosted data center infrastructure. New deployment models have gained maturity, such as fog and mobile edge computing, bringing the cloud closer to the end user. These models could also benefit from container technology, as the newly introduced devices often have limited hardware resources. In this survey, we provide an overview of the current state of the art regarding resource management within the broad sense of cloud computing, complementary to existing surveys in literature. We investigate how research is adapting to the recent evolutions within the cloud, being the adoption of container technology and the introduction of the fog computing conceptual model. Furthermore, we identify several challenges and possible opportunities for future research.

/pdf/resource-management-in-a-containerized-cloud-status-and-50xg94ulsn.pdf

Resource Management in a Containerized Cloud: Status and Challenges

Summary
The high volume of monitoring information generated by large-scale cloud infrastructures poses a challenge to the capacity of cloud providers in detecting anomalies in the infrastructure. Traditional anomaly detection methods are resource-intensive and computationally complex for training and/or detection, what is undesirable in very dynamic and large-scale environment such as clouds. Isolation-based methods have the advantage of low complexity for training and detection and are optimized for detecting failures. In this work, we explore the feasibility of Isolation Forest, an isolation-based anomaly detection method, to detect anomalies in large-scale cloud data centers. We propose a method to code time-series information as extra attributes that enable temporal anomaly detection and establish its feasibility to adapt to seasonality and trends in the time-series and to be applied online and in real-time.

On the effectiveness of isolation‐based anomaly detection in cloud data centers

We demonstrate the ability of deep architectures, specifically convolutional neural networks, to learn and differentiate the lexical features of different programming languages presented in coding video tutorials found on the Internet We analyze over 17,000 video frames containing examples of Java, Python, and other textual and non-textual objects Our results indicate that not only can computer vision models based on deep architectures be taught to differentiate among programming languages with over 98% accuracy, but can learn language-specific lexical features in the process This provides a powerful mechanism for carrying out program comprehension research on repositories where source code is represented with imagery rather than text, while simultaneously avoiding the computational overhead of optical character recognition

https://dl.acm.org/doi/pdf/10.1145/3196321.3196359

Learning lexical features of programming languages from imagery using convolutional neural networks

The rapid growth of cloud computing environment with many clients ranging from personal users to big corporate or business houses has become a challenge for cloud organizations to handle the massive volume of data and various resources in the cloud. Inefficient management of resources can degrade the performance of cloud computing. Therefore, resources must be evenly allocated to different stakeholders without compromising the organization’s profit as well as users’ satisfaction. A customer’s request cannot be withheld indefinitely just because the fundamental resources are not free on the board. In this paper, a combined resource allocation security with efficient task scheduling in cloud computing using a hybrid machine learning (RATS-HM) technique is proposed to overcome those problems. The proposed RATS-HM techniques are given as follows: First, an improved cat swarm optimization algorithm-based short scheduler for task scheduling (ICS-TS) minimizes the make-span time and maximizes throughput. Second, a group optimization-based deep neural network (GO-DNN) for efficient resource allocation using different design constraints includes bandwidth and resource load. Third, a lightweight authentication scheme, i.e., NSUPREME is proposed for data encryption to provide security to data storage. Finally, the proposed RATS-HM technique is simulated with a different simulation setup, and the results are compared with state-of-art techniques to prove the effectiveness. The results regarding resource utilization, energy consumption, response time, etc., show that the proposed technique is superior to the existing one.

/pdf/a-joint-resource-allocation-security-with-efficient-task-1a526lnj.pdf

A Joint Resource Allocation, Security with Efficient Task Scheduling in Cloud Computing Using Hybrid Machine Learning Techniques

In IT service delivery and support, the cloud paradigm has introduced the problem of resource over-provisioning through rapid automation (or orchestration) of manual IT operations. Due to the elastic nature of cloud computing, this shortcoming ends up significantly reducing the real benefit, viz., the cost-effectiveness of cloud adoption for Cloud Service Consumers (CSC). Similarly, detecting and eliminating such over-provisioning of cloud resources without affecting the quality of service (QoS) is extremely difficult for Cloud Service Providers (CSPs) since they have visibility only into the state of the IT services (cloud resources) but none into the actual performance of business services. In this paper, we propose Tier-centric Business Impact and Cost Analysis (T-BICA), a tier-centric optimal resource allocation algorithm, to address the problem of rapid provisioning of IT resources in modern enterprise cloud environments, through extensive data gathering and performance analyses of business services in a simulated environment emulating a mature cloud service provider. We have derived improved analytics to address the issues and to accelerate real cloud adoption for large enterprises within the context of meeting (or exceeding) business service level objectives (SLOs) and minimizing the cloud subscription cost (OpEx) for the business. While investigating the problem, we consider the time and the cost of delivering business service in medium- to large-size enterprise environments, quantifying the negative impact of IT resource over-provisioning (due to highly mature IT services centric orchestration capabilities) on the business, and indicate how the suggested cloud analytics could assist in reducing total cost of ownership (TCO) of the business service. From our analysis of the test data, we have observed that our suggested approach and analytic reduces the cost of delivering business services by 65.19 percent, and improves the performance (total time to deliver) by 74.18 percent when compared to the existing modern cloud management and resource provisioning approach. Using T-BICA dramatically reduces upfront costs (CapEx) for CSPs (from the capacity procurement and management points of view) through efficient on-demand resource de-provisioning, without affecting business SLOs and IT service level agreements (SLAs). The improved dynamic allocation of resources also makes for better efficiency of utilization, which in turn has desirable consequences for sustainability, and makes this an approach for “Green” IT.

Tier-Centric Resource Allocation in Multi-Tier Cloud Systems

With dozens of popular programming languages used worldwide, the number of source code files of programs available online for public use is massive. However most blogs, forums or online Q&A websites have poor searchability for specific programming language source code. Naive thumb rules based on the file extension if any are invariably used for syntax highlighting, indentation and other ways to improve readability of the code by programming language editors. A more systematic way to identify the language in which a given source file was written would be of immense value. We believe that simple Bayesiam models would be adequate for this given the intrinsic syntactic structure of any programming language. In this paper, we present Bayesian learning models for correctly identifying the programming language in which a given piece of source code was written, with high probability. We have used 20000 source code files across 10 programming languages to train and test the model using the following Bayesian classifier models – Naive Bayes, Bayesian Network and Multinomial Naive Bayes. Lastly, we show a performance comparison among the three models in terms of classification accuracy on the test data.

Detecting Programming Language from Source Code Using Bayesian Learning Techniques

Invariants represent properties of a system that are expected to hold when everything goes well. Thus, the violation of an invariant most likely corresponds to the occurrence of an anomaly in the system. In this paper, we discuss the accuracy and the completeness of an anomaly detection system based on invariants. The case study we have taken is a back-end operation of a SaaS platform. Results show the rationality of the approach and discuss the impact of the invariant mining strategy on the detection capabilities, both in terms of accuracy and of time to reveal violations.

Using Invariants for Anomaly Detection: The Case Study of a SaaS Application

This paper addresses the problem of “deep matching” — or matching different classes of entities based on latent underlying semantics, rather than just their visible attributes. An example of this is the “automatic task assignment” problem where several tasks have to be assigned to people with varied skill-sets and experiences. Datasets showing types of entities (tasks and people) along with their involvement of other concepts, are used as the basis for deep matching. This paper describes a work in progress, of a deep matching application called SortingHat. We analyze issue tracking data of a large corporation containing task descriptions and assignments to people that were computed manually. We identify several entities and concepts from the dataset and build a co-occurrence graph as the basic data structure for computing deep matches. We then propose a set of query primitives that can establish several forms of semantic matching across different classes of entities.

SortingHat: A framework for deep matching between classes of entities

Apache Hadoop is an open source framework for processing massive amount of data in a distributed environment. Hadoop services use a polling mechanism for event notifications. In this paper, we propose a distributed notification system for Hadoop based on the Publish-Subscribe model. Such a notification system can be used for message-passing among Hadoop services. It can also be used to chain multiple MapReduce jobs based on events occuring in a Hadoop cluster. This results in reduced cluster load and network bandwidth requirement. We have used two popular Publish-Subscribe-based messaging systems—Apache ActiveMQ and Apache Kafka—for implementation. Lastly, we have executed performance tests on both these messaging systems to monitor time taken for message delivery and reception.

Jyotiska Nath Khasnabish

Papers

Tier-Centric Resource Allocation in Multi-Tier Cloud Systems

Detecting Programming Language from Source Code Using Bayesian Learning Techniques

Using Invariants for Anomaly Detection: The Case Study of a SaaS Application

SortingHat: A framework for deep matching between classes of entities

Implementing a Publish-Subscribe Distributed Notification System on Hadoop