This paper proposes a new and efficient technique to regularize the neural network in the context of deep learning using correlations among features. Previous studies have shown that oversized deep neural network models tend to produce a lot of redundant features that are either the shifted version of one another or are very similar and show little or no variations, thus resulting in redundant filtering. We propose a way to address this problem and show that such redundancy can be avoided using regularization and adaptive feature dropout mechanism. We show that regularizing both negative and positive correlated features according to their differentiation and based on their relative cosine distances yields network extracting dissimilar features with less overfitting and better generalization. This concept is illustrated with deep multilayer perceptron, convolutional neural network, sparse autoencoder, gated recurrent unit, and long short-term memory on MNIST digits recognition, CIFAR-10, ImageNet, and Stanford Natural Language Inference data sets.

Regularizing Deep Neural Networks by Enhancing Diversity in Feature Extraction

It is critical to evaluate the quality of clusters for most cluster analysis. A number of cluster validity indexes have been proposed, such as the Silhouette and Davies–Bouldin indexes. However, these validity indexes cannot be used to process clusters with arbitrary shapes. Some researchers employ graph-based distance to cluster nonspherical data sets, but the computation of graph-based distances between all pairs of points in a data set is time-consuming. A potential solution is to select some representative points. Inspired by this idea, we propose a novel Local Cores-based Cluster Validity (LCCV) index to improve the performance of Silhouette index. Local cores, with local maximum density, are selected as representative points. Since graph-based distance is used to evaluate the dissimilarity between local cores, the LCCV index is effective for obtaining the optimal cluster number for data sets containing clusters with arbitrary shapes. Moreover, a hierarchical clustering algorithm based on the LCCV index is proposed. The experimental results on synthetic and real data sets indicate that the new index outperforms existing ones.

A Novel Cluster Validity Index Based on Local Cores

Wireless sensor networks (WSNs) are composed with a set of sensor nodes, and its primary task is to sense and relay the data to the base station or sink. Sensor nodes are equipped with a limited battery, and more energy is consumed due to the relay of the data. While relaying on more data, nodes near to the base station will die soon and result in energy-hole or hotspot problem. A mobile sink has been introduced to collect the data from the nodes to avoid the hotspot problem. Several data collection algorithms have been proposed for 2D WSNs; however, these algorithms do not meet the requirements of the 3D WSNs due to its variations in topology, connectivity, coverage, scalability, etc. Therefore, the selection of mobile sink visiting points called rendezvous points (RPs), and path determination between then in 3D WSNs is a challenging task. In this paper, we determine the optimal RPs in 3D WSNs using unsupervised learning-based hierarchical agglomerative clustering. Further, we propose an algorithm for path determination between the RPs with minimal computational overhead. The simulation runs show that the proposed method finds the near-optimal set of RPs and path of the mobile sink while enhancing the network lifetime.

Data Collection and Path Determination Strategies for Mobile Sink in 3D WSNs

New imbalanced fault diagnosis framework based on Cluster-MWMOTE and MFO-optimized LS-SVM using limited and complex bearing data

Clustering validation and identifying the optimal number of clusters are of great importance in expert and intelligent systems. However, the commonly used similarity measures for validating are not versatile to measure the complex data structure, in reality, some of which are not as effective as that of the used clustering algorithm which gives the clustering results. This paper studies the validity indexes for the hierarchical clustering algorithm and proposes a unified validity index framework. For the single-linkage agglomerative hierarchical clustering we propose two efficient synthetical clustering validity (SCV) indexes using the minimum spanning tree to calculate the intra-cluster compactness to overcome the deficiencies of the measurements in the existing validity indexes. For the general hierarchical clustering, a self-adaptive similarity measure strategy and two generalized synthetical clustering validity (GSCV) indexes, which are the extension of the proposed SCV indexes, are developed. The proposed SCV and GSCV indexes constitute a unified validity index framework, where SCV index is a special case of GSCV index, can avoid the incompatibility of the similarity measure between the clustering and validation. The experimental comparisons with the state-of-the-art validity indexes on artificial and real-world data sets demonstrate the efficiency of the proposed validity indexes in discovering the true number of clusters and dealing with various sorts of data sets, including imbalanced data sets.

Efficient synthetical clustering validity indexes for hierarchical clustering

It is crucial to determine the optimal number of clusters for the clustering quality in cluster analysis. From the standpoint of sample geometry, two concepts, i.e., the sample clustering dispersion degree and the sample clustering synthesis degree, are defined, and a new clustering validity index is designed. Moreover, a method for determining the optimal number of clusters based on an agglomerative hierarchical clustering (AHC) algorithm is proposed. The new index and the method can evaluate the clustering results produced by the AHC and determine the optimal number of clusters for multiple types of datasets, such as linear, manifold, annular, and convex structures. Theoretical research and experimental results indicate the validity and good performance of the proposed index and the method.

Method for Determining the Optimal Number of Clusters Based on Agglomerative Hierarchical Clustering

This paper aims to explore teachers' support and students' academic achievements. Because students' academic achievements are related to the quality of higher education, however, not all students' academic achievements are invested in learning activities independently, and a big factor in learning gains is students' social support. With reference to previous studies and existing literature, This study selects teacher support as the independent variable, vocational students' academic achievements as the dependent variable, students' academic self-efficacy and learning engagement as the mediation variables to build a research model to explore the chain mediation effect of teacher support on academic achievement through academic self-efficacy and learning engagement. In this study, 414 vocational college students in Chongqing were investigated by questionnaire. The reliability and validity of the questionnaire were consistent with the academic level, and were statistically tested by SPSS 25.0; The results show that: 1. Teacher support positively predicts vocational students' academic achievements; 2.Teacher support can indirectly predict the academic achievements of higher vocational students through academic self-efficacy; 3. Teacher support can indirectly predict the academic achievements of higher vocational students through learning engagement; 4. Academic self-efficacy - learning engagement plays a chain mediation role between teacher support and academic achievement. Finally, based on the conclusions of the study, this study puts forward specific suggestions for future research.

Shibing Zhou

Papers

Method for Determining the Optimal Number of Clusters Based on Agglomerative Hierarchical Clustering

A Research on the Relationship between Teachers' Support and Students' Academic Achievement in Higher Vocational Colleges