scispace - formally typeset
Search or ask a question
Journal ArticleDOI

Compressive parameter estimation via K-median clustering

01 Jan 2018-Signal Processing (Elsevier)-Vol. 142, pp 36-52
TL;DR: The use of earth mover’s distance (EMD), as applied to a pair of true and estimated PD coefficient vectors, to measure the parameter estimation error and it is shown that the EMD provides a better-suited metric for parameter estimation performance than the Euclidean distance.
About: This article is published in Signal Processing.The article was published on 2018-01-01 and is currently open access. It has received 6 citations till now. The article focuses on the topics: Estimation theory & Sparse approximation.
Citations
More filters
Journal ArticleDOI
TL;DR: This paper proposes the use of a machine learning method known as overlap aware learning along with CSP that generates a smoother decision boundary and hence improves the classification accuracy at higher undersampling factors and simulation results show the trend of improved classification accuracy using the proposed method.
Abstract: Compressed signal processing (CSP) is a branch of compressive sensing (CS), which gives a direction to solve a class of signal processing problems directly from the compressive measurements of a signal. CSP utilizes the information preserved in the compressive measurements of a signal to solve certain inference problems like: classification, detection, and estimation, without reconstructing the original signal. It further simplifies the signal processing compared to conventional CS by omitting their complex reconstruction stage. This, in turn, reduces the implementation complexity of signal processing systems. This paper investigates the performance of CSP for classification application. After extracting the features from compressive measurements, these features or the data instances are used for classification purpose. Through experimental analysis, it has been found that as the CS undersampling factor is increased, the overlapping among the data instances predominates. This results in a complex decision boundary, which in turn degrades the classification accuracy at higher undersampling factors. To overcome the above issue, this paper proposes the use of a machine learning method known as overlap aware learning along with CSP. This generates a smoother decision boundary and hence improves the classification accuracy at higher undersampling factors. The simulation results show the trend of improved classification accuracy using the proposed method. An analysis of the proposed method has been done on different datasets and based on run-time complexity and complexity vs gain analysis to verify the effectiveness of proposed method.

4 citations


Cites background from "Compressive parameter estimation vi..."

  • ...For this, a threshold needs be derived and then the detection rule can be compared against this threshold to detect the presence of desired signal [9]–[16]....

    [...]

Proceedings ArticleDOI
02 Jul 2021
TL;DR: In this article, the authors provide a global view of ML methods for multi-objective optimization problems and a reference for applying ML methods to solve a specific type of MOPs.
Abstract: In the real world, it is challenging to calculate a trade-off alternative with traditional classical methods for complex non-linear systems, which always involve multiple conflicting objectives. Such complicated systems urgently desire advanced methods to conquer the multi-objective optimization problems (MOPs). As a promising AI method, the development and application of Machine Learning (ML) attract increasingly more attention from researchers. The natures of ML methods, such as parallel computation possibility, no need for any priori assumptions, etc., ensure the effectiveness and efficiency for solving MOPs. However, as we know, there is no literature related to the comprehensive review of ML in multi-objective optimization domain until now. This literature review aims to provide researchers a global view of mainstream ML methods for MOO in a general domain and a reference for applying ML methods to solve a specific type of MOPs. In this paper, the general ML mainstream methods are summarized, based on which the literature relating to ML on MOPs are retrieved in comprehensive domains. The relevant literature is categorized according to the emphasis of object types, purposes and methods, and the categorization results are finally analyzed and discussed.

3 citations

Dissertation
03 Jul 2018
TL;DR: The ways the application of a compressive measurement kernel impacts the signal recovery performance are looked into and methods to infer the current signal complexity from the compressive observations are investigated.
Abstract: Since the advent of the first digital processing units, the importance of digital signal processing has been steadily rising. Today, most signal processing happens in the digital domain, requiring that analog signals be first sampled and digitized before any relevant data can be extracted from them. The recent explosion of the demands for data acquisition, storage and processing, however, has pushed the capabilities of conventional acquisition systems to their limits in many application areas. By offering an alternative view on the signal acquisition process, ideas from sparse signal processing and one of its main beneficiaries compressed sensing (CS), aim at alleviating some of these problems. In this thesis, we look into the ways the application of a compressive measurement kernel impacts the signal recovery performance and investigate methods to infer the current signal complexity from the compressive observations. We then study a particular application, namely that of sub-Nyquist sampling and processing of sparse analog multiband signals in spectral, angular and spatial domains.

1 citations


Cites background from "Compressive parameter estimation vi..."

  • ...Another important sparsity model that we consider here is the parametric model [34], [50]–[52]....

    [...]

Journal ArticleDOI
TL;DR: It is proved that the (1+1) EA can obtain a performance guarantee of 5 for k -median problem in polynomial expected runtime O ( m n 2 ⋅ d m a x ) if all distances between data points and cluster centers are integers.
Abstract: Evolutionary algorithms (EAs) have been widely studied in numerical experiments and successfully applied in practice. However, most existing EAs are designed without a theoretical analysis for their performance guarantee. In this paper, we define a fitness function to guide the well-known (1+1) evolutionary algorithm, called (1+1) EA, to optimize the NP-hard k -median problem. We prove that the (1+1) EA can obtain a performance guarantee of 5 for k -median problem in polynomial expected runtime O ( m n 2 ⋅ d m a x ) if all distances between data points and cluster centers are integers, where m and n are respectively the cardinalities of the data set and cluster center set, and d m a x denotes the largest distance between data points and cluster centers. To tackle the general case that the distances between data points and cluster centers are real number, we propose an improved (1+1) EA, which employs a distance scaling scheme during the evolutionary process. Our rigorous theoretical analysis shows that the improved (1+1) EA obtains a performance guarantee of 5 + e in expected runtime O ( e − 1 m n 2 log ( m ⋅ d m a x ) ) , where e > 0 is a constant. This study reveals that an appropriate scaling scheme can help EAs to obtain a constant performance guarantee in polynomial expected runtime and provides insights into design EAs for some NP-hard problems. In addition, we conduct experiment to compare the efficiency of the improved (1+1) EA with three clustering algorithms on optimizing three UCI datasets.

1 citations

Journal ArticleDOI
TL;DR: By expanding the scalar innovation of each subsystem model to the innovation vector, a partially coupled multi-innovation generalized stochastic gradient algorithm is proposed and indicates that the proposed algorithms are effective and have good parameter estimation performances.
Abstract: This paper researches the identification problem for the unknown parameters of the multivariate equation-error autoregressive systems. Firstly, the original identification model is decomposed into several sub-identification models according to the number of system outputs. Then, based on the characteristic that the information vector and the parameter vector are common among the sub-identification models, the coupling identification concept is used to propose a partially coupled generalized stochastic gradient algorithm. Furthermore, by expanding the scalar innovation of each subsystem model to the innovation vector, a partially coupled multi-innovation generalized stochastic gradient algorithm is proposed. Finally, the numerical simulations indicate that the proposed algorithms are effective and have good parameter estimation performances.
References
More filters
Book
D.L. Donoho1
01 Jan 2004
TL;DR: It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients, and a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing.
Abstract: Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0

18,609 citations

Journal ArticleDOI
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
Abstract: The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB). Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising. BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.

9,950 citations


"Compressive parameter estimation vi..." refers methods in this paper

  • ...In this paper, we propose the use of the earth mover’s distance (EMD), as applied to a pair of true and estimate PD coefficient vectors, to measure the error in sparsity-based parameter estimation....

    [...]

Journal ArticleDOI
TL;DR: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.
Abstract: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions. These waveforms are chosen in order to best match the signal structures. Matching pursuits are general procedures to compute adaptive signal representations. With a dictionary of Gabor functions a matching pursuit defines an adaptive time-frequency transform. They derive a signal energy distribution in the time-frequency plane, which does not include interference terms, unlike Wigner and Cohen class distributions. A matching pursuit isolates the signal structures that are coherent with respect to a given dictionary. An application to pattern extraction from noisy signals is described. They compare a matching pursuit decomposition with a signal expansion over an optimized wavepacket orthonormal basis, selected with the algorithm of Coifman and Wickerhauser see (IEEE Trans. Informat. Theory, vol. 38, Mar. 1992). >

9,380 citations


"Compressive parameter estimation vi..." refers methods in this paper

  • ...Additionally, we leverage the relationship between K-median clustering and EMD-based sparse approximation to develop improved PD-based parameter estimation algorithms....

    [...]

Journal ArticleDOI
TL;DR: It is demonstrated theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal.
Abstract: This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

8,604 citations


"Compressive parameter estimation vi..." refers methods in this paper

  • ...Additionally, we leverage the relationship between K-median clustering and EMD-based sparse approximation to develop improved PD-based parameter estimation algorithms....

    [...]

Book
01 Jan 2013
TL;DR: This book discusses data mining through the lens of cluster analysis, which examines the relationships between data, clusters, and algorithms, and some of the techniques used to solve these problems.
Abstract: 1 Introduction 1.1 What is Data Mining? 1.2 Motivating Challenges 1.3 The Origins of Data Mining 1.4 Data Mining Tasks 1.5 Scope and Organization of the Book 1.6 Bibliographic Notes 1.7 Exercises 2 Data 2.1 Types of Data 2.2 Data Quality 2.3 Data Preprocessing 2.4 Measures of Similarity and Dissimilarity 2.5 Bibliographic Notes 2.6 Exercises 3 Exploring Data 3.1 The Iris Data Set 3.2 Summary Statistics 3.3 Visualization 3.4 OLAP and Multidimensional Data Analysis 3.5 Bibliographic Notes 3.6 Exercises 4 Classification: Basic Concepts, Decision Trees, and Model Evaluation 4.1 Preliminaries 4.2 General Approach to Solving a Classification Problem 4.3 Decision Tree Induction 4.4 Model Overfitting 4.5 Evaluating the Performance of a Classifier 4.6 Methods for Comparing Classifiers 4.7 Bibliographic Notes 4.8 Exercises 5 Classification: Alternative Techniques 5.1 Rule-Based Classifier 5.2 Nearest-Neighbor Classifiers 5.3 Bayesian Classifiers 5.4 Artificial Neural Network (ANN) 5.5 Support Vector Machine (SVM) 5.6 Ensemble Methods 5.7 Class Imbalance Problem 5.8 Multiclass Problem 5.9 Bibliographic Notes 5.10 Exercises 6 Association Analysis: Basic Concepts and Algorithms 6.1 Problem Definition 6.2 Frequent Itemset Generation 6.3 Rule Generation 6.4 Compact Representation of Frequent Itemsets 6.5 Alternative Methods for Generating Frequent Itemsets 6.6 FP-Growth Algorithm 6.7 Evaluation of Association Patterns 6.8 Effect of Skewed Support Distribution 6.9 Bibliographic Notes 6.10 Exercises 7 Association Analysis: Advanced Concepts 7.1 Handling Categorical Attributes 7.2 Handling Continuous Attributes 7.3 Handling a Concept Hierarchy 7.4 Sequential Patterns 7.5 Subgraph Patterns 7.6 Infrequent Patterns 7.7 Bibliographic Notes 7.8 Exercises 8 Cluster Analysis: Basic Concepts and Algorithms 8.1 Overview 8.2 K-means 8.3 Agglomerative Hierarchical Clustering 8.4 DBSCAN 8.5 Cluster Evaluation 8.6 Bibliographic Notes 8.7 Exercises 9 Cluster Analysis: Additional Issues and Algorithms 9.1 Characteristics of Data, Clusters, and Clustering Algorithms 9.2 Prototype-Based Clustering 9.3 Density-Based Clustering 9.4 Graph-Based Clustering 9.5 Scalable Clustering Algorithms 9.6 Which Clustering Algorithm? 9.7 Bibliographic Notes 9.8 Exercises 10 Anomaly Detection 10.1 Preliminaries 10.2 Statistical Approaches 10.3 Proximity-Based Outlier Detection 10.4 Density-Based Outlier Detection 10.5 Clustering-Based Techniques 10.6 Bibliographic Notes 10.7 Exercises Appendix A Linear Algebra Appendix B Dimensionality Reduction Appendix C Probability and Statistics Appendix D Regression Appendix E Optimization Author Index Subject Index

7,356 citations