Richard J. Hathaway

Bio: Richard J. Hathaway is an academic researcher from Georgia Southern University. The author has contributed to research in topics: Cluster analysis & Fuzzy clustering. The author has an hindex of 34, co-authored 63 publications receiving 5725 citations. Previous affiliations of Richard J. Hathaway include University of South Carolina.

Switching regression models and fuzzy clustering

Abstract: A family of objective functions called fuzzy c-regression models, which can be used too fit switching regression models to certain types of mixed data, is presented. Minimization of particular objective functions in the family yields simultaneous estimates for the parameters of c regression models, together with a fuzzy c-partitioning of the data. A general optimization approach for the family of objective functions is given and corresponding theoretical convergence results are discussed. The approach is illustrated by two numerical examples that show how it can be used to fit mixed data to coupled linear and nonlinear models. >

Convergence theory for fuzzy c-means: Counterexamples and repairs

Abstract: A counterexample to the original incorrect convergence theorem for the fuzzy c-means (FCM) clustering algorithms (see J.C. Bezdak, IEEE Trans. Pattern Anal. and Math. Intell., vol.PAMI-2, no.1, pp.1-8, 1980) is provided. This counterexample establishes the existence of saddle points of the FCM objective function at locations other than the geometric centroid of fuzzy c-partition space. Counterexamples previously discussed by W.T. Tucker (1987) are summarized. The correct theorem is stated without proof: every FCM iterate sequence converges, at least along a subsequence, to either a local minimum or saddle point of the FCM objective function. Although Tucker's counterexamples and the corrected theory appear elsewhere, they are restated as a caution not to further propagate the original incorrect convergence statement.

Fuzzy c-means clustering of incomplete data

Abstract: The problem of clustering a real s-dimensional data set X={x/sub 1/,...,x/sub n/} /spl sub/ R/sup s/ is considered. Usually, each observation (or datum) consists of numerical values for all s features (such as height, length, etc.), but sometimes data sets can contain vectors that are missing one or more of the feature values. For example, a particular datum x/sub k/ might be incomplete, having the form x/sub k/=(254.3, ?, 333.2, 47.45, ?)/sup T/, where the second and fifth feature values are missing. The fuzzy c-means (FCM) algorithm is a useful tool for clustering real s-dimensional data, but it is not directly applicable to the case of incomplete data. Four strategies for doing FCM clustering of incomplete data sets are given, three of which involve modified versions of the FCM algorithm. Numerical convergence properties of the new algorithms are discussed, and all approaches are tested using real and artificially generated incomplete data sets.

Convergence of alternating optimization

Abstract: Let f : Rs → R be a real-valued function, and let x = (x1,...,xs)T ∈ Rs be partitioned into t subsets of non-overlapping variables as x = (X1,...,Xt)T, with Xi ∈ Rpi for i = 1,...,t, Σi=1tpi = s. Alternating optimization (AO) is an iterative procedure for minimizing f(x) = f(X1, X2,..., Xt) jointly over all variables by alternating restricted minimizations over the individual subsets of variables X1,...., Xt. Alternating optimization has been (more or less) studied and used in a wide variety of areas. Here a self-contained and general convergence theory is presented that is applicable to all partitionings of x. Under reasonable assumptions, the general AO approach is shown to be locally, q-linearly convergent, and to also exhibit a type of global convergence.

Some Notes on Alternating Optimization

Abstract: Let f : Rs ? R be a real-valued scalar field, and let x = (x1,..., xs)T ? Rs be partitioned into t subsets of non-overlapping variables as x = (X1,...,Xt)T, with Xi ? Rpi, for i = 1, ..., t, ?i=1tPi = s. Alternating optimization (AO) is an iterative procedure for minimizing (or maximizing) the function f(x) = f(X1,X2,...,Xt) jointly over all variables by alternating restricted minimizations over the individual subsets of variables X1,...,Xt. AO is the basis for the c-means clustering algorithms (t=2), many forms of vector quantization (t = 2, 3 and 4), and the expectation-maximization (EM) algorithm (t = 4) for normal mixture decomposition. First we review where and how AO fits into the overall optimization landscape. Then we discuss the important theoretical issues connected with the AO approach. Finally, we state (without proofs) two new theorems that give very general local and global convergence and rate of convergence results which hold for all partitionings of x.

Compressed sensing

Abstract: Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0

Pattern Recognition and Machine Learning

Abstract: Probability Distributions.- Linear Models for Regression.- Linear Models for Classification.- Neural Networks.- Kernel Methods.- Sparse Kernel Machines.- Graphical Models.- Mixture Models and EM.- Approximate Inference.- Sampling Methods.- Continuous Latent Variables.- Sequential Data.- Combining Models.

Survey of clustering algorithms

Abstract: Data analysis plays an indispensable role for understanding various phenomena. Cluster analysis, primitive exploration with little or no prior knowledge, consists of research developed across a wide variety of communities. The diversity, on one hand, equips us with many tools. On the other hand, the profusion of options causes confusion. We survey clustering algorithms for data sets appearing in statistics, computer science, and machine learning, and illustrate their applications in some benchmark data sets, the traveling salesman problem, and bioinformatics, a new field attracting intensive efforts. Several tightly related topics, proximity measure, and cluster validation, are also discussed.