Unsupervised domain adaptation has caught appealing attentions as it facilitates the unlabeled target learning by borrowing existing well-established source domain knowledge. Recent practice on domain adaptation manages to extract effective features by incorporating the pseudo labels for the target domain to better solve cross-domain distribution divergences. However, existing approaches separate target label optimization and domain-invariant feature learning as different steps. To address that issue, we develop a novel Graph Adaptive Knowledge Transfer (GAKT) model to jointly optimize target labels and domain-free features in a unified framework. Specifically, semi-supervised knowledge adaptation and label propagation on target data are coupled to benefit each other, and hence the marginal and conditional disparities across different domains will be better alleviated. Experimental evaluation on two cross-domain visual datasets demonstrates the effectiveness of our designed approach on facilitating the unlabeled target task learning, compared to the state-of-the-art domain adaptation approaches.

Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation

It is widely acknowledged that the success of deep learning is built upon large-scale training data and tremendous computing power. However, the data and computing power are not always available for many real-world applications. In this paper, we address the machine learning problem where it lacks training data and limits computing power. Specifically, we investigate domain adaptation which is able to transfer knowledge from one labeled source domain to an unlabeled target domain, so that we do not need much training data from the target domain. At the same time, we consider the situation that the running environment is confined, e.g., in edge computing the end device has very limited running resources. Technically, we present the Faster Domain Adaptation (FDA) protocol and further report two paradigms of FDA: early stopping and amid skipping. The former accelerates domain adaptation by multiple early exit points. The latter speeds up the adaptation by wisely skip several amid neural network blocks. Extensive experiments on standard benchmarks verify that our method is able to achieve the comparable and even better accuracy but employ much less computing resources. To the best of our knowledge, there are very few works which investigated accelerating knowledge adaptation in the community. This work is expected to inspire the topic for more discussion. 

Faster Domain Adaptation Networks

Cross-domain object detection uses knowledge from source domain tasks to enhance the object detection in target domain. It can reduce the workload of data annotations in the new domain and significantly improve the adaptation ability of the network. We consider a more realistic transfer scenario, that is, our target domain samples cannot be obtained at the same time so that learners cannot forget the old domain knowledge while learning the new domain knowledge, which is consistent with the basic assumption of incremental learning. However, the existing methods do not consider the sequential learning process of the multiple target domains which may cause performance degradation. To address this issue, we propose a novel multi-domain adaptation method for object detection based on incremental learning. Specifically, the incremental learning network saves the knowledge of multiple domains and makes the model to fuse the knowledge of different domains during the training effectively. To make it work better, the progressive training strategy is proposed to make the model gradually adapt to multiple domains. Moreover, we use a multi-level feature alignment module to ensure that domain alignment is realized on features at various levels. We perform experiments on two sets of 6 datasets, which demonstrate that our model can effectively solve the problem of domain knowledge forgetting in multi-target domain adaptation and significantly improve the detection accuracy in each domain.

Incremental learning based multi-domain adaptation for object detection

A novel method for journal bearing degradation evaluation and remaining useful life prediction under different working conditions

Domain adaptation learning is proposed as an effective technology for leveraging rich supervision knowledge from the related domain(s) to learn a reliable classifier for a new domain. One popular kind of domain adaptation methods is based on feature representation. However, such methods fail to consider the within-class and between-class relations after obtaining the new representation. In addition, they do not consider the negative effects of features that might be redundant or irrelevant to the final classification. To this end, a novel domain-invariant feature learning method based on the maximum margin criterion and sparsity technique for unsupervised domain adaptation is proposed in this paper, referred to as discriminative and informative joint distribution adaptation (DIJDA). Specifically, DIJDA adopts the maximum margin criterion in the adaptation process such that the transformed samples are near to those in the same class but segregated from those in different classes. As a result, the discriminative knowledge referred from source labels can be transferred to target domain effectively. Moreover, DIJDA imposes a row-sparsity constraint on the transformation matrix, which enforces rows of the matrix corresponding to inessential feature attributes to be all zero. Therefore, the most informative feature attributes can be extracted. Compared with several state-of-the-art methods, DIJDA substantially improves the classification results on five widely used benchmark datasets, which demonstrates the effectiveness of the proposed method.

Discriminative and informative joint distribution adaptation for unsupervised domain adaptation

Domain adaptation algorithms leverage the knowledge from a well-labeled source domain to facilitate the learning of an unlabeled target domain, in which the source domain and the target domain are related but drawn from different data distributions. Existing domain adaptation approaches are either trying to explicitly mitigate the data distribution gaps by minimizing some distance metrics, or attempting to learn a new feature representation by revealing the shared factors and use the learned representation as a bridge of knowledge transfer. Recently, several researchers claim that jointly optimizing the distribution gaps and latent factors can learn a better transfer model. In this paper, therefore, we propose a novel approach which simultaneously mitigates the data distribution and learns a feature representation via a common objective. Specifically, we present joint metric and feature representation learning (JMFL) for unsupervised domain adaptation. JMFL, on the one hand, minimizes the domain discrepancy between the source domain and the target domain. On the other hand, JMFL reveals the shared underlying factors between the two domains to learn a new feature representation. We smoothly incorporate the two aspects into a unified objective and present a detailed optimization method. Extensive experiments on several open benchmarks verify that our approach achieves state-of-the-art results with significant improvements.

Joint metric and feature representation learning for unsupervised domain adaptation

Domain adaptation investigates the problem of leveraging knowledge from a well-labeled source domain to an unlabeled target domain, where the two domains are drawn from different data distributions. Because of the distribution shifts, different target samples have distinct degrees of difficulty in adaptation. However, existing domain adaptation approaches overwhelmingly neglect the degrees of difficulty and deploy exactly the same framework for all of the target samples. Generally, a simple or shadow framework is fast but rough. A sophisticated or deep framework, on the contrary, is accurate but slow. In this paper, we aim to challenge the fundamental contradiction between the accuracy and speed in domain adaptation tasks. We propose a novel approach, named agile domain adaptation, which agilely applies optimal frameworks to different target samples and classifies the target samples according to their adaptation difficulties. Specifically, we propose a paradigm which performs several early detections before the final classification. If a sample can be classified at one of the early stage with enough confidence, the sample would exit without the subsequent processes. Notably, the proposed method can significantly reduce the running cost of domain adaptation approaches, which can extend the application scenarios of domain adaptation to even mobile devices and real-time systems. Extensive experiments on two open benchmarks verify the effectiveness and efficiency of the proposed method.

Agile Domain Adaptation

In this paper, we formulate cross-domain multistream classification as a domain adaptation problem. Then we propose a novel algorithm that utilizes low-rank representation and graph embedding to preserve data structures, which benefits in dealing with concept drifts and concept revolution. In addition, we deploy MMD metric to minimize the distribution discrepancy between the source data stream and the target data stream. Experiment results on Office+Caltech dataset with DeCAF\(_6\) features verified the effectiveness of our algorithm.

A Domain Adaptation Approach for Multistream Classification

Domain adaptation investigates the problem of leveraging knowledge from a well-labeled source domain to an unlabeled target domain, where the two domains are drawn from different data distributions. Because of the distribution shifts, different target samples have distinct degrees of difficulty in adaptation. However, existing domain adaptation approaches overwhelmingly neglect the degrees of difficulty and deploy exactly the same framework for all of the target samples. Generally, a simple or shadow framework is fast but rough. A sophisticated or deep framework, on the contrary, is accurate but slow. In this paper, we aim to challenge the fundamental contradiction between the accuracy and speed in domain adaptation tasks. We propose a novel approach, named {\it agile domain adaptation}, which agilely applies optimal frameworks to different target samples and classifies the target samples according to their adaptation difficulties. Specifically, we propose a paradigm which performs several early detections before the final classification. If a sample can be classified at one of the early stage with enough confidence, the sample would exit without the subsequent processes. Notably, the proposed method can significantly reduce the running cost of domain adaptation approaches, which can extend the application scenarios of domain adaptation to even mobile devices and real-time systems. Extensive experiments on two open benchmarks verify the effectiveness and efficiency of the proposed method.

Yue Xie

Papers

Joint metric and feature representation learning for unsupervised domain adaptation

Agile Domain Adaptation

A Domain Adaptation Approach for Multistream Classification

Agile Domain Adaptation