Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Pattern Recognition and Machine Learning

Matrix Factorization Techniques for Recommender Systems

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.

https://link.springer.com/content/pdf/10.1007/s10618-019-00619-1.pdf

Deep learning for time series classification: a review

Nonlinear Programming: A Unified Approach

The notion of fuzziness as defined in this paper relates to situations in which the source of imprecision is not a random variable or a stochastic process, but rather a class or classes which do not possess sharply defined boundaries, e.g., the “class of bald men,” or the “class of numbers which are much greater than 10,” or the “class of adaptive systems,” etc. A basic concept which makes it possible to treat fuzziness in a quantitative manner is that of a fuzzy set, that is, a class in which there may be grades of membership intermediate between full membership and non-membership. Thus, a fuzzy set is characterized by a membership function which assigns to each object its grade of membership (a number lying between 0 and 1) in the fuzzy set. After a review of some of the relevant properties of fuzzy sets, the notions of a fuzzy system and a fuzzy class of systems are introduced and briefly analyzed. The paper closes with a section dealing with optimization under fuzzy constraints in which an approach to...

Fuzzy sets and systems

The main purpose of this paper is to use off-the-shelf devices to develop a fall detection system. In human body identification, human body silhouette is adopted to improve privacy protection, and vertical projection histograms of the silhouette image and statistical scheme are used to reduce the effect of human body upper limb activities. The kNN classification algorithm is used to classify the postures using the ratio and difference of human body silhouette bounding box height and width. Meanwhile, since time difference is a vital factor to differentiate fall incident event and lying down event, the critical time difference is obtained from the experiment and verified by statistical hypothesis testing. With the help of the kNN classifier and the critical time difference, a fall incident detection system is developed to detect fall incident events. The experiment shows that it could reduce the effect of upper limb activities and the system has a correct rate of 84.44% on fall detection and lying down event detection.

https://ir.nctu.edu.tw:443/bitstream/11536/32136/1/000279408200052.pdf

A fall detection system using k-nearest neighbor classifier

Time series classification is an important research topic in machine learning and data mining communities, since time series data exist in many application domains. Recent studies have shown that machine learning algorithms could benefit from good feature representation, explaining why deep learning has achieved breakthrough performance in many tasks. In deep learning, the convolutional neural network (CNN) is one of the most well-known approaches, since it incorporates feature learning and classification task in a unified network architecture. Although CNN has been successfully applied to image and text domains, it is still a challenge to apply CNN to time series data. This paper proposes a tensor scheme along with a novel deep learning architecture called multivariate convolutional neural network (MVCNN) for multivariate time series classification, in which the proposed architecture considers multivariate and lag-feature characteristics. We evaluate our proposed method with the prognostics and health management (PHM) 2015 challenge data, and compare with several algorithms. The experimental results indicate that the proposed method outperforms the other alternatives using the prediction score, which is the evaluation metric used by the PHM Society 2015 data challenge. Besides performance evaluation, we provide detailed analysis about the proposed method.

Time Series Classification With Multivariate Convolutional Neural Network

In this paper, we design and develop a movie-rating and review-summarization system in a mobile environment. The movie-rating information is based on the sentiment-classification result. The condensed descriptions of movie reviews are generated from the feature-based summarization. We propose a novel approach based on latent semantic analysis (LSA) to identify product features. Furthermore, we find a way to reduce the size of summary based on the product features obtained from LSA. We consider both sentiment-classification accuracy and system response time to design the system. The rating and review-summarization system can be extended to other product-review domains easily.

https://ir.nctu.edu.tw:443/bitstream/11536/16001/1/000303069100010.pdf

Movie Rating and Review Summarization in Mobile Environment

In the past decades, many optimization methods have been devised and applied to job shop scheduling problem (JSSP) to find the optimal solution. Many methods assumed that the scheduling results were applied to static environments, but the whole environments in the real world are always dynamic. Moreover, many unexpected events such as machine breakdowns and material problems may be present to adversely affect the initial job scheduling. This work views JSSP as a sequential decision making problem and proposes to use deep reinforcement learning to cope with this problem. The combination of deep learning and reinforcement learning avoids handcraft features as used in traditional reinforcement learning, and it is expected that the combination will make the whole learning phase more efficient. Our proposed model comprises actor network and critic network, both including convolution layers and fully connected layer. Actor network agent learns how to behave in different situations, while critic network helps agent evaluate the value of statement then return to actor network. This work proposes a parallel training method, combining asynchronous update as well as deep deterministic policy gradient (DDPG), to train the model. The whole network is trained with parallel training on a multi-agent environment and different simple dispatching rules are considered as actions. We evaluate our proposed model on more than ten instances that are present in a famous benchmark problem library - OR library. The evaluation results indicate that our method is comparative in static JSSP benchmark problems, and achieves a good balance between makespan and execution time in dynamic environments. Scheduling score of our method is 91.12% in static JSSP benchmark problems, and 80.78% in dynamic environments.

/pdf/actor-critic-deep-reinforcement-learning-for-solving-job-56b4sjbf6l.pdf

Actor-critic deep reinforcement learning for solving job shop scheduling problems

Universum, a collection of nonexamples that do not belong to any class of interest, has become a new research topic in machine learning. This paper devises a semi-supervised learning with Universum algorithm based on boosting technique, and focuses on situations where only a few labeled examples are available. We also show that the training error of AdaBoost with Universum is bounded by the product of normalization factor, and the training error drops exponentially fast when each weak classifier is slightly better than random guessing. Finally, the experiments use four data sets with several combinations. Experimental results indicate that the proposed algorithm can benefit from Universum examples and outperform several alternative methods, particularly when insufficient labeled examples are available. When the number of labeled examples is insufficient to estimate the parameters of classification functions, the Universum can be used to approximate the prior distribution of the classification functions. The experimental results can be explained using the concept of Universum introduced by Vapnik, that is, Universum examples implicitly specify a prior distribution on the set of classification functions.

Chien-Liang Liu

Papers

A fall detection system using k-nearest neighbor classifier

Time Series Classification With Multivariate Convolutional Neural Network

Movie Rating and Review Summarization in Mobile Environment

Actor-critic deep reinforcement learning for solving job shop scheduling problems

Semi-Supervised Text Classification With Universum Learning