https://axon.cs.byu.edu/~martinez/classes/678/Presentations/Yao.pdf

Extreme learning machine: Theory and applications

It is clear that the learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be: 1) the slow gradient-based learning algorithms are extensively used to train neural networks, and 2) all the parameters of the networks are tuned iteratively by using such learning algorithms. Unlike these traditional implementations, this paper proposes a new learning algorithm called extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses the input weights and analytically determines the output weights of SLFNs. In theory, this algorithm tends to provide the best generalization performance at extremely fast learning speed. The experimental results based on real-world benchmarking function approximation and classification problems including large complex applications show that the new algorithm can produce best generalization performance in some cases and can learn much faster than traditional popular learning algorithms for feedforward neural networks.

Extreme learning machine: a new learning scheme of feedforward neural networks

Deep learning and its applications to machine health monitoring

Broad Learning System (BLS) that aims to offer an alternative way of learning in deep structure is proposed in this paper. Deep structure and learning suffer from a time-consuming training process because of a large number of connecting parameters in filters and layers. Moreover, it encounters a complete retraining process if the structure is not sufficient to model the system. The BLS is established in the form of a flat network, where the original inputs are transferred and placed as “mapped features” in feature nodes and the structure is expanded in wide sense in the “enhancement nodes.” The incremental learning algorithms are developed for fast remodeling in broad expansion without a retraining process if the network deems to be expanded. Two incremental learning algorithms are given for both the increment of the feature nodes (or filters in deep structure) and the increment of the enhancement nodes. The designed model and algorithms are very versatile for selecting a model rapidly. In addition, another incremental learning is developed for a system that has been modeled encounters a new incoming input. Specifically, the system can be remodeled in an incremental way without the entire retraining from the beginning. Satisfactory result for model reduction using singular value decomposition is conducted to simplify the final structure. Compared with existing deep neural networks, experimental results on the Modified National Institute of Standards and Technology database and NYU NORB object recognition dataset benchmark data demonstrate the effectiveness of the proposed BLS.

Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture

The Web holds valuable, vast, and unstructured information about public opinion. Here, the history, current use, and future of opinion mining and sentiment analysis are discussed, along with relevant techniques and tools.

/pdf/new-avenues-in-opinion-mining-and-sentiment-analysis-1lp3r9s7xa.pdf

New Avenues in Opinion Mining and Sentiment Analysis

This special issue includes eight original works that detail the further developments of ELMs in theories, applications, and hardware implementation. In "Representational Learning with ELMs for Big Data," Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong propose using the ELM as an auto-encoder for learning feature representations using singular values. In "A Secure and Practical Mechanism for Outsourcing ELMs in Cloud Computing," Jiarun Lin, Jianping Yin, Zhiping Cai, Qiang Liu, Kuan Li, and Victor C.M. Leung propose a method for handling large data applications by outsourcing to the cloud that would dramatically reduce ELM training time. In "ELM-Guided Memetic Computation for Vehicle Routing," Liang Feng, Yew-Soon Ong, and Meng-Hiot Lim consider the ELM as an engine for automating the encapsulation of knowledge memes from past problem-solving experiences. In "ELMVIS: A Nonlinear Visualization Technique Using Random Permutations and ELMs," Anton Akusok, Amaury Lendasse, Rui Nian, and Yoan Miche propose an ELM method for data visualization based on random permutations to map original data and their corresponding visualization points. In "Combining ELMs with Random Projections," Paolo Gastaldo, Rodolfo Zunino, Erik Cambria, and Sergio Decherchi analyze the relationships between ELM feature-mapping schemas and the paradigm of random projections. In "Reduced ELMs for Causal Relation Extraction from Unstructured Text," Xuefeng Yang and Kezhi Mao propose combining ELMs with neuron selection to optimize the neural network architecture and improve the ELM ensemble's computational efficiency. In "A System for Signature Verification Based on Horizontal and Vertical Components in Hand Gestures," Beom-Seok Oh, Jehyoung Jeon, Kar-Ann Toh, Andrew Beng Jin Teoh, and Jaihie Kim propose a novel paradigm for hand signature biometry for touchless applications without the need for handheld devices. Finally, in "An Adaptive and Iterative Online Sequential ELM-Based Multi-Degree-of-Freedom Gesture Recognition System," Hanchao Yu, Yiqiang Chen, Junfa Liu, and Guang-Bin Huang propose an online sequential ELM-based efficient gesture recognition algorithm for touchless human-machine interaction.

Extreme Learning Machine

The worldwide spread of COVID-19 dramatically influences the world economic landscape. In this paper, we have quantitatively investigated the time-frequency co-movement impact of COVID-19 on U.S. and China stock market since early 2020 in terms of daily observation from National Association of Securities Dealers Automated Quotations Index (NDX), Dow Jones Industrial Average (DJIA), Standard & Poor's 500 Index (SPX), Shanghai Securities Composite Index (SSEC), Shenzhen Securities Component Index (SZI), in favor of spatiotemporal interactions over investor sentiment index, and propose to explore the divisibility and the predictability to the volatility of stock market during the development of COVID-19. We integrate evidence yielded from wavelet coherence and phase difference to suggest the responses of stock market indexes to the COVID-19 epidemic in a long-term band, which could be roughly divided into three distinguished phases, namely, 30-75, 110-150, and 220-280 business days for China, and 80-125 and 160-175 after 290 business days for the U.S. At the first phase, the reason for the extreme volatility of stock market mainly attributed to the sudden emergence of the COVID-19 epidemic due to the pessimistic expectations from investors; China and U.S. stock market shared strongly negative correlation with the growing number of COVID-19 cases. At the second phase, the revitalization of stock market shared strong simultaneous moves but exhibited opposite responses to the COVID-19 impact on China and U.S. stock market; the former retained a significant negative correlation, while the latter turned to positively correlated throughout the period. At the third phase, the progress in vaccine development and economic stimulus began to impose forces to stock market; the vulnerability to COVID-19 diminished to some extent as the investor sentiment indexes rebounded. Finally, we attempted to initially establish a coarse-grained representation to stock market indexes and investor sentiment indexes, which demonstrated the homogenous spacial distribution in the vectorgraph after normalization and quantization, implying the strong consistency when filtering the frequent small fluctuations during the evolution of the COVID-19 pandemic, which might help insights into the prediction of possible status transition in stock market performance under the public health issues, potentially performing as the quantitative references in reasonably deducing the economic influences.

/pdf/quantifying-time-frequency-co-movement-impact-of-covid-19-on-rzsz708qyz.pdf

Quantifying Time-Frequency Co-movement Impact of COVID-19 on U.S. and China Stock Market Toward Investor Sentiment Index.

The population of Atlantic cod significantly contributes to the prosperity of fishery production in the world. In this paper, we quantitatively investigate the global abundance variation in Atlantic cod from 1919 to 2016, in favor of spatio-temporal interactions over manifold impact factors at local observation sites, and propose to explore the predictive mechanism with the help of its periodicity, time-frequency co-movement and lead-lag effects, via Long Short Term Memory (LSTM). We first integrate evidences yielded from wavelet coefficients, to suggest that the abundance variation potentially follows a 36-year major cycle and 24-year secondary cycle at the time scales of 55 years and 37 years. We further evaluate the responses of Atlantic cod abundance to the external impact factors, including SST, catches, prey biomass, SSS, in aid of the wavelet coherence and phase difference, which allows to identify the dominantly correlative factors and capture the leading roles along the time domain, and then divide the responses around recent 60 years into three stages: before 1985, 1985-1995, after 1995. At the first stage, the reason for the decline in abundance could be mainly attributed to the rapid rise of fish catches. At the second stage, the impact of SST and SSS also constitute significant indices, besides over-fishing, meanwhile, the mortality of primary producers and forced migration of fish species, indirectly cause the decline. At the third stage, warming SST and growing SSS, directly led to the decrease of abundance. Finally, we establish one ensemble of LSTM-SAE architecture, to comprehensively reflect the predictive patterns at each stage. It has been demonstrated from experiment results that the models behaved better when intentionally feeding with the dominantly correlative multivariate inputs, instead of either all factors or only the abundance. The proposed scheme provides opportunities to symmetrically identify the underlying predictive attributes of Atlantic cod abundance, potentially perform as the quantitative references in reasonably making fishing decision. With the rapid development in deep learning capabilities, it is hopeful to expect better predictions of the responses to global changes, not only for Atlantic cod but also for other fish species and the ecosystem as a whole.

/pdf/the-identification-and-prediction-in-abundance-variation-of-45ol0a6kz3.pdf

The Identification and Prediction in Abundance Variation of Atlantic Cod via Long Short-Term Memory With Periodicity, Time–Frequency Co-movement, and Lead-Lag Effect Across Sea Surface Temperature, Sea Surface Salinity, Catches, and Prey Biomass From 1919 to 2016

Rui Nian

Papers

Extreme Learning Machine

Quantifying Time-Frequency Co-movement Impact of COVID-19 on U.S. and China Stock Market Toward Investor Sentiment Index.

The Identification and Prediction in Abundance Variation of Atlantic Cod via Long Short-Term Memory With Periodicity, Time–Frequency Co-movement, and Lead-Lag Effect Across Sea Surface Temperature, Sea Surface Salinity, Catches, and Prey Biomass From 1919 to 2016