The talk describes the £ 120M UK ‘e-Science’ initiative and begins by defining what is meant by the term e-Science. The majority of the £ 120M, some £ 85M, is for the support of large-scale e-Science projects in many areas of science and engineering. The infrastructure needed to support such projects needs to allow sharing of distributed and heterogeneous computational and data resources and support effective collaboration between groups of scientists. Such an infrastructure is commonly referred to as the Grid. The remaining funds, some £ 35M, constitute the e-Science ‘Core Program’ and is intended to encourage the development of robust and generic Grid middleware in collaboration with industry. The key elements of the Core Program will be described including the construction of a UK e-Science Grid. In addition, the talk will include a description of the pilot projects that have so far received funding. These span a range of disciplines from particle physics and astronomy to engineering and healthcare. We conclude with some remarks about the need to develop a data architecture for the Grid that will allow federated access to relational databases as well as flat files. H. Zima et al. (Eds.): ISHPC 2002, LNCS 2327, p. 6, 2002. c © Springer-Verlag Berlin Heidelberg 2002 SPEC HPC2002: The Next High-Performance Computer Benchmark

High Performance Computing

A Survey on Incorporating Domain Knowledge into Deep Learning for Medical Image Analysis

Parametric deep energy approach for elasticity accounting for strain gradient effects

Along with the popularity of the Internet of Things (IoT) techniques with several computational paradigms, such as cloud and edge computing, microservice has been viewed as a promising architecture in large-scale application design and deployment. Due to the limited computing ability of edge devices in distributed IoT, only a small scale of data can be used for model training. In addition, most of the machine-learning-based intrusion detection methods are insufficient when dealing with imbalanced dataset under limited computing resources. In this article, we propose an optimized intra/inter-class-structure-based variational few-shot learning (OICS-VFSL) model to overcome a specific out-of-distribution problem in imbalanced learning, and to improve the microservice-oriented intrusion detection in distributed IoT systems. Following a newly designed VFSL framework, an intra/inter-class optimization scheme is developed using reconstructed feature embeddings, in which the intra-class distance is optimized based on the approximation during a variation Bayesian process, while the inter-class distance is optimized based on the maximization of similarities during a feature concatenation process. An intelligent intrusion detection algorithm is, then, introduced to improve the multiclass classification via a nonlinear neural network. Evaluation experiments are conducted using two public datasets to demonstrate the effectiveness of our proposed model, especially in detecting novel attacks with extremely imbalanced data, compared with four baseline methods. 

/pdf/variational-few-shot-learning-for-microservice-oriented-3g34zist.pdf

Variational Few-Shot Learning for Microservice-Oriented Intrusion Detection in Distributed Industrial IoT

Automated construction of deep neural networks (DNNs) has become a research hot spot nowadays because DNN’s performance is heavily influenced by its architecture and parameters, which are highly task-dependent, but it is notoriously difficult to find the most appropriate DNN in terms of architecture and parameters to best solve a given task. In this work, we provide an insight into the automated DNN construction process by formulating it into a multilevel multiobjective large-scale optimization problem with constraints, where the nonconvex, nondifferentiable, and black-box nature of this problem make evolutionary algorithms (EAs) to stand out as a promising solver. Then, we give a systematical review of existing evolutionary DNN construction techniques from different aspects of this optimization problem and analyze the pros and cons of using EA-based methods in each aspect. This work aims to help DNN researchers to better understand why, where, and how to utilize EAs for automated DNN construction and meanwhile, help EA researchers to better understand the task of automated DNN construction so that they may focus more on EA-favored optimization scenarios to devise more effective techniques.

A Survey on Evolutionary Construction of Deep Neural Networks

Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks However, finding a good set of hyperparameters for a CNN remains a challenging task It usually takes an expert with deep knowledge, and trials and errors Genetic algorithms have been used in hyperparameter optimizations However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth As the depth increases, the number of hyperparameters grows exponentially, and searching becomes exponentially harder It is important to have an efficient algorithm that can find a good model in reasonable time In this article, we propose to use a variable length genetic algorithm (GA) to systematically and automatically tune the hyperparameters of a CNN to improve its performance Experimental results show that our algorithm can find good CNN hyperparameters efficiently It is clear from our experiments that if more time is spent on optimizing the hyperparameters, better results could be achieved Theoretically, if we had unlimited time and CPU power, we could find the optimized hyperparameters and achieve the best results in the future

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm.

Improving performance of deep learning models and reducing their training times are ongoing challenges in deep neural networks. There are several approaches proposed to address these challenges, one of which is to increase the depth of the neural networks. Such deeper networks not only increase training times, but also suffer from vanishing gradients problem while training. In this work, we propose gradient amplification approach for training deep learning models to prevent vanishing gradients and also develop a training strategy to enable or disable gradient amplification method across several epochs with different learning rates. We perform experiments on VGG-19 and Resnet models (Resnet-18 and Resnet-34), and study the impact of amplification parameters on these models in detail. Our proposed approach improves performance of these deep learning models even at higher learning rates, thereby allowing these models to achieve higher performance with reduced training time.

/pdf/gradient-amplification-an-efficient-way-to-train-deep-neural-4hhegjydbd.pdf

Gradient Amplification: An Efficient Way to Train Deep Neural Networks

Improving performance of deep learning models and reducing their training times are ongoing challenges in deep neural networks There are several approaches proposed to address these challenges one of which is to increase the depth of the neural networks Such deeper networks not only increase training times, but also suffer from vanishing gradients problem while training In this work, we propose gradient amplification approach for training deep learning models to prevent vanishing gradients and also develop a training strategy to enable or disable gradient amplification method across several epochs with different learning rates We perform experiments on VGG-19 and resnet (Resnet-18 and Resnet-34) models, and study the impact of amplification parameters on these models in detail Our proposed approach improves performance of these deep learning models even at higher learning rates, thereby allowing these models to achieve higher performance with reduced training time

Gradient Amplification: An efficient way to train deep neural networks

This paper reviews recent research works in infant cry signal analysis and classification tasks. A broad range of literatures are reviewed mainly from the aspects of data acquisition, cross domain signal processing techniques, and machine learning classification methods. We introduce pre-processing approaches and describe a diversity of features such as MFCC, spectrogram, and fundamental frequency, etc. Both acoustic features and prosodic features extracted from different domains can discriminate frame-based signals from one another and can be used to train machine learning classifiers. Together with traditional machine learning classifiers such as KNN, SVM, and GMM, newly developed neural network architectures such as CNN and RNN are applied in infant cry research. We present some significant experimental results on pathological cry identification, cry reason classification, and cry sound detection with some typical databases. This survey systematically studies the previous research in all relevant areas of infant cry and provides an insight on the current cutting-edge works in infant cry signal analysis and classification. We also propose future research directions in data processing, feature extraction, and neural network classification fields to better understand, interpret, and process infant cry signals.

/pdf/a-review-of-infant-cry-analysis-and-classification-4gdq9ucfus.pdf

A review of infant cry analysis and classification

As the complexity of deep learning models grows, the difficulty and time of training them also increases. Depending on the task, the complexity of the model, and the hardware resources available, the amount of training time could take hours, weeks or even months. To decrease the training time, we propose a method to intelligently freeze layers during the training process. Our method involves designing a formula to calculate normalized gradient differences for all layers with weights in the model, and then use the calculated values to decide how many layers should be frozen. We implemented our method on top of stochastic gradient descent, and performed experiments on standard image classification dataset CIFAR-10. Results show that our method can accelerate training on VGG nets, ResNets, and DenseNets while having similar test accuracy.

Chunyan Ji

Papers

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm.

Gradient Amplification: An Efficient Way to Train Deep Neural Networks

Gradient Amplification: An efficient way to train deep neural networks

A review of infant cry analysis and classification

Fast Deep Learning Training through Intelligently Freezing Layers