Auto-Encoder (AE)-based deep subspace clustering (DSC) methods have achieved impressive performance due to the powerful representation extracted using deep neural networks while prioritizing categorical separability. However, self-reconstruction loss of an AE ignores rich useful relation information and might lead to indiscriminative representation, which inevitably degrades the clustering performance. It is also challenging to learn high-level similarity without feeding semantic labels. Another unsolved problem facing DSC is the huge memory cost due to $n\times n$ similarity matrix, which is incurred by the self-expression layer between an encoder and decoder. To tackle these problems, we use pairwise similarity to weigh the reconstruction loss to capture local structure information, while a similarity is learned by the self-expression layer. Pseudo-graphs and pseudo-labels, which allow benefiting from uncertain knowledge acquired during network training, are further employed to supervise similarity learning. Joint learning and iterative training facilitate to obtain an overall optimal solution. Extensive experiments on benchmark datasets demonstrate the superiority of our approach. By combining with the $k$ -nearest neighbors algorithm, we further show that our method can address the large-scale and out-of-sample problems. The source code of our method is available: https://github.com/sckangz/SelfsupervisedSC .

Pseudo-Supervised Deep Subspace Clustering

A survey on conic relaxations of optimal power flow problem

Morphological reconstruction (MR) is often employed by seeded image segmentation algorithms such as watershed transform and power watershed, as it is able to filter out seeds (regional minima) to reduce over-segmentation. However, the MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed. In this paper, a novel adaptive morphological reconstruction (AMR) operation is proposed that has three advantages. First, AMR can adaptively filter out useless seeds while preserving meaningful ones. Second, AMR is insensitive to the scale of structuring elements because multiscale structuring elements are employed. Finally, the AMR has two attractive properties: monotonic increasingness and convergence that help seeded segmentation algorithms to achieve a hierarchical segmentation. Experiments clearly demonstrate that the AMR is useful for improving performance of algorithms of seeded image segmentation and seed-based spectral segmentation. Compared to several state-of-the-art algorithms, the proposed algorithms provide better segmentation results requiring less computing time.

/pdf/adaptive-morphological-reconstruction-for-seeded-image-4qb5vlqrcz.pdf

Adaptive Morphological Reconstruction for Seeded Image Segmentation

A multitude of optimization tasks ensue in the context of the smart grid, often exhibiting undesirable characteristics like non-convexity, mixed types of design variables and multiple - and often conflicting - objectives. These tasks can be broadly categorized into three classes of problems, namely optimal power flow (OPF), scheduling and planning. Metaheuristic search methods form a generic class of optimization techniques, that have been shown to work successfully for complex problems. Not surprisingly, they have been widely applied in the smart grid, their use spanning almost every smart grid-related optimization task. In this work, we review the use of metaheuristic search for OPF, scheduling and planning through a unified approach, keeping in mind that these problems share many common challenges and objectives. The use of different metaheuristic methods is discussed extensively with regard to problem handling, multi-objective optimization performance and method accuracy in relation to computational complexity. An attempt to arrive at quantitative conclusions is also being made, by compiling tables which present collective results on common test grids. Lastly, the paper identifies promising directions for future research, concerning metaheuristic search application practices, method development and new challenges that we believe will shape the future of smart grid optimization.

Metaheuristic search in smart grid: A review with emphasis on planning, scheduling and power flow optimization applications

Auto-Encoder (AE)-based deep subspace clustering (DSC) methods have achieved impressive performance due to the powerful representation extracted using deep neural networks while prioritizing categorical separability. However, self-reconstruction loss of an AE ignores rich useful relation information and might lead to indiscriminative representation, which inevitably degrades the clustering performance. It is also challenging to learn high-level similarity without feeding semantic labels. Another unsolved problem facing DSC is the huge memory cost due to $n\times n$ similarity matrix, which is incurred by the self-expression layer between an encoder and decoder. To tackle these problems, we use pairwise similarity to weigh the reconstruction loss to capture local structure information, while a similarity is learned by the self-expression layer. Pseudo-graphs and pseudo-labels, which allow benefiting from uncertain knowledge acquired during network training, are further employed to supervise similarity learning. Joint learning and iterative training facilitate to obtain an overall optimal solution. Extensive experiments on benchmark datasets demonstrate the superiority of our approach. By combining with the $k$-nearest neighbors algorithm, we further show that our method can address the large-scale and out-of-sample problems.

Pseudo-supervised Deep Subspace Clustering

This paper proposes a novel deep subspace clustering approach which uses convolutional autoencoders to transform input images into new representations lying on a union of linear subspaces. The first contribution of our work is to insert multiple fully-connected linear layers between the encoder layers and their corresponding decoder layers to promote learning more favorable representations for subspace clustering. These connection layers facilitate the feature learning procedure by combining low-level and high-level information for generating multiple sets of self-expressive and informative representations at different levels of the encoder. Moreover, we introduce a novel loss minimization problem which leverages an initial clustering of the samples to effectively fuse the multi-level representations and recover the underlying subspaces more accurately. The loss function is then minimized through an iterative scheme which alternatively updates the network parameters and produces new clusterings of the samples. Experiments on four real-world datasets demonstrate that our approach exhibits superior performance compared to the state-of-the-art methods on most of the subspace clustering problems.

/pdf/multi-level-representation-learning-for-deep-subspace-2taoiklefr.pdf

Multi-Level Representation Learning for Deep Subspace Clustering

This two-part paper is concerned with the problem of minimizing a linear objective function subject to a bilinear matrix inequality (BMI) constraint. In this part, we first consider a family of convex relaxations which transform BMI optimization problems into polynomial-time solvable surrogates. As an alternative to the state-of-the-art semidefinite programming (SDP) and second-order cone programming (SOCP) relaxations, a computationally efficient parabolic relaxation is developed, which relies on convex quadratic constraints only. Next, we developed a family of penalty functions, which can be incorporated into the objective of SDP, SOCP, and parabolic relaxations to facilitate the recovery of feasible points for the original non-convex BMI optimization. Penalty terms can be constructed using any arbitrary initial point. We prove that if the initial point is sufficiently close to the feasible set, then the penalized relaxations are guaranteed to produce feasible points for the original BMI. In Part II of the paper, the efficacy of the proposed penalized convex relaxations is demonstrated on benchmark instances of $\mathcal{H}_{2}$ and $\mathcal{H}_{\infty}$ optimal control synthesis problems.

/pdf/convex-relaxation-of-bilinear-matrix-inequalities-part-i-2pn46iu4g4.pdf

Convex Relaxation of Bilinear Matrix Inequalities Part I: Theoretical Results

This paper is concerned with developing a novel approach to tackle the problem of subspace clustering. The approach introduces a convolutional autoencoder-based architecture to generate low-rank representations (LRR) of input data which are proven to be very suitable for subspace clustering. We propose to insert a fully-connected linear layer and its transpose between the encoder and decoder to implicitly impose a rank constraint on the learned representations. We train this architecture by minimizing a standard deep subspace clustering loss function and then recover underlying subspaces by applying a variant of spectral clustering technique. Extensive experiments on benchmark datasets demonstrate that the proposed model can not only achieve very competitive clustering results using a relatively small network architecture but also can maintain its high level of performance across a wide range of LRRs. This implies that the model can be appropriately combined with the state-of-the-art subspace clustering architectures to produce more accurate results.

/pdf/deep-low-rank-subspace-clustering-81qqigffjm.pdf

Fariba Zohrizadeh

Papers

A survey on conic relaxations of optimal power flow problem

Multi-Level Representation Learning for Deep Subspace Clustering

Convex Relaxation of Bilinear Matrix Inequalities Part I: Theoretical Results

Deep Low-Rank Subspace Clustering

Multi-Level Representation Learning for Deep Subspace Clustering