Inter-domain Cluster Mapping and GMCV Based Transformation for Domain Adaptation
10 Dec 2013-pp 74-81
TL;DR: An algorithm for a direct solution of domain adaptation to transform data in source domain to match the distribution in the target domain by formulating a transformation matrix based on the Geometric Mean of Co-Variances (GMCV), estimated from the covariance matrices of the data from both the domains.
Abstract: This paper describes an algorithm for a direct solution of domain adaptation (DA) to transform data in source domain to match the distribution in the target domain. This is achieved by formulating a transformation matrix based on the Geometric Mean of Co-Variances (GMCV), estimated from the covariance matrices of the data from both the domains. As a pre-processing step, we propose an iterative framework for clustering over data from both the domains, to produce an inter-domain mapping function of clusters. A closed form solution for direct DA is obtained from the GMCV formulation. Experimental results on real world datasets confirms the importance of clustering prior to transformation using GMCV for better classification accuracy. Results show the superior result of the proposed method of DA, when compared with a few state of the art methods.
Citations
More filters
TL;DR: A method based on information theory and a kernel-based clustering algorithm is proposed to detect efficiently the set of land-cover classes which are common to both domains as well as the additional or missing classes in the target domain image.
Abstract: This paper addresses the problem of land-cover classification of remotely sensed image pairs in the context of domain adaptation. The primary assumption of the proposed method is that the training data are available only for one of the images (source domain), whereas for the other image (target domain), no labeled data are available. No assumption is made here on the number and the statistical properties of the land-cover classes that, in turn, may vary from one domain to the other. The only constraint is that at least one land-cover class is shared by the two domains. Under these assumptions, a novel graph theoretic cross-domain cluster mapping algorithm is proposed to detect efficiently the set of land-cover classes which are common to both domains as well as the additional or missing classes in the target domain image. An interdomain graph is introduced, which contains all of the class information of both images, and subsequently, an efficient subgraph-matching algorithm is proposed to highlight the changes between them. The proposed cluster mapping algorithm initially clusters the target domain data into an optimal number of groups given the available source domain training samples. To this end, a method based on information theory and a kernel-based clustering algorithm is proposed. Considering the fact that the spectral signature of land-cover classes may overlap significantly, a postprocessing step is applied to refine the classification map produced by the clustering algorithm. Two multispectral data sets with medium and very high geometrical resolution and one hyperspectral data set are considered to evaluate the robustness of the proposed technique. Two of the data sets consist of multitemporal image pairs, while the remaining one contains images of spatially disjoint geographical areas. The experiments confirm the effectiveness of the proposed framework in different complex scenarios.
36 citations
Cites background or methods or result from "Inter-domain Cluster Mapping and GM..."
...The cluster mapping is achieved in [29] by means of...
[...]
...It is interesting to see that [29] was unable to produce well-defined clusters when new classes were added to the target domain with high degree overlapping with the existing classes....
[...]
...For Hyper 1, [29] has performed a correct cluster mapping for the water class as indicated by the small JS divergence (0....
[...]
...It is to be noted that the parallel clustering technique of [29] produces a minor degraded result compared to the kernel k-means used in the proposed setup as indicated by the JS divergence measures in both cases....
[...]
...In particular, [29] failed to detect the floodplain grass-1, island interior, and firescar2 properly given that these classes are very much overlapping in the spectral domain....
[...]
References
More filters
01 Dec 2005
TL;DR: This paper proposes and analyzes parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences, and shows that there is a bijection between regular exponential families and a largeclass of BRegman diverGences, that is called regular Breg man divergence.
Abstract: A wide variety of distortion functions, such as squared Euclidean distance, Mahalanobis distance, Itakura-Saito distance and relative entropy, have been used for clustering. In this paper, we propose and analyze parametric hard and soft clustering algorithms based on a large class of distortion functions known as Bregman divergences. The proposed algorithms unify centroid-based parametric clustering approaches, such as classical kmeans , the Linde-Buzo-Gray (LBG) algorithm and information-theoretic clustering, which arise by special choices of the Bregman divergence. The algorithms maintain the simplicity and scalability of the classical kmeans algorithm, while generalizing the method to a large class of clustering loss functions. This is achieved by first posing the hard clustering problem in terms of minimizing the loss in Bregman information, a quantity motivated by rate distortion theory, and then deriving an iterative algorithm that monotonically decreases this loss. In addition, we show that there is a bijection between regular exponential families and a large class of Bregman divergences, that we call regular Bregman divergences. This result enables the development of an alternative interpretation of an efficient EM scheme for learning mixtures of exponential family distributions, and leads to a simple soft clustering algorithm for regular Bregman divergences. Finally, we discuss the connection between rate distortion theory and Bregman clustering and present an information theoretic analysis of Bregman clustering algorithms in terms of a trade-off between compression and loss in Bregman information.
1,723 citations
"Inter-domain Cluster Mapping and GM..." refers methods in this paper
...While re-clustering X̂ (step 8), α2 ensures that an instance x̂i is assigned to a cluster with minimum Bregman divergence [9] from a cluster in Y....
[...]
...7: Calculate Bregman divergence [9] of each instance of X̂ with respect to each of the clusters formed in Y, as: α2(i, k) = (x̂i − μt ) (E−1 t Es)(x̂i − μt ), ∀i = 1, 2, ....
[...]
...Once the clusters in Y are formed, the clusters in X̂ are reformed based on the Bregman divergence [9] from the means of clusters of Y....
[...]
20 Jun 2007
TL;DR: In this paper, the authors proposed a transfer learning framework called TrAdaBoost, which allows users to utilize a small amount of newly labeled data to leverage the old data to construct a high-quality classification model for the new data.
Abstract: Traditional machine learning makes a basic assumption: the training and test data should be under the same distribution. However, in many cases, this identical-distribution assumption does not hold. The assumption might be violated when a task from one new domain comes, while there are only labeled data from a similar old domain. Labeling the new data can be costly and it would also be a waste to throw away all the old data. In this paper, we present a novel transfer learning framework called TrAdaBoost, which extends boosting-based learning algorithms (Freund & Schapire, 1997). TrAdaBoost allows users to utilize a small amount of newly labeled data to leverage the old data to construct a high-quality classification model for the new data. We show that this method can allow us to learn an accurate model using only a tiny amount of new data and a large amount of old data, even when the new data are not sufficient to train a model alone. We show that TrAdaBoost allows knowledge to be effectively transferred from the old data to the new. The effectiveness of our algorithm is analyzed theoretically and empirically to show that our iterative algorithm can converge well to an accurate model.
1,509 citations
30 Aug 2009
TL;DR: The usefulness of multi-class subgroup discovery is demonstrated experimentally, using discovered subgroups as features for a decision tree learner, and significant improvements in accuracy and AUC are achieved with particular evaluation measures and settings.
Abstract: Subgroup discovery aims at finding subsets of a population whose class distribution is significantly different from the overall distribution. It has previously predominantly been investigated in a two-class context. This paper investigates multi-class subgroup discovery methods. We consider six evaluation measures for multi-class subgroups, four of them new, and study their theoretical properties. We extend the two-class subgroup discovery algorithm CN2-SD to incorporate the new evaluation measures and a new weighting scheme inspired by AdaBoost. We demonstrate the usefulness of multi-class subgroup discovery experimentally, using discovered subgroups as features for a decision tree learner. Not only is the number of leaves of the decision tree reduced with a factor between 8 and 16 on average, but significant improvements in accuracy and AUC are achieved with particular evaluation measures and settings. Similar performance improvements can be observed when using naive Bayes.
825 citations
Proceedings Article•
03 Dec 2007
TL;DR: This paper proposes a direct importance estimation method that does not involve density estimation and is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized.
Abstract: A situation where training and test samples follow different input distributions is called covariate shift. Under covariate shift, standard learning methods such as maximum likelihood estimation are no longer consistent—weighted variants according to the ratio of test and training input densities are consistent. Therefore, accurately estimating the density ratio, called the importance, is one of the key issues in covariate shift adaptation. A naive approach to this task is to first estimate training and test input densities separately and then estimate the importance by taking the ratio of the estimated densities. However, this naive approach tends to perform poorly since density estimation is a hard task particularly in high dimensional cases. In this paper, we propose a direct importance estimation method that does not involve density estimation. Our method is equipped with a natural cross validation procedure and hence tuning parameters such as the kernel width can be objectively optimized. Simulations illustrate the usefulness of our approach.
785 citations
"Inter-domain Cluster Mapping and GM..." refers background or methods in this paper
...The cyan, brown and magenta curves show the classification accuracy using different methods published for DA: KLIEP [1], ASVM [4] and CDSVM [3]....
[...]
...One solution to this problem is to weigh each instance in the source domain appropriately such that, the weighted instances of the source domain are used for training to minimize the expected loss [1], [2]....
[...]
...Comparative studies are done using ASVM [4], CD-SVM [3] and KLIEP [1]....
[...]