“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

비만아 부모의 돌봄 경험: q 방법론

Artificial neural networks (ANNs) constitute a class of flexible nonlinear models designed to mimic biological neural systems. In this entry, we introduce ANN using familiar econometric terminology and provide an overview of ANN modeling approach and its implementation methods. † Correspondence: Chung-Ming Kuan, Institute of Economics, Academia Sinica, 128 Academia Road, Sec. 2, Taipei 115, Taiwan; ckuan@econ.sinica.edu.tw. †† I would like to express my sincere gratitude to the editor, Professor Steven Durlauf, for his patience and constructive comments on early drafts of this entry. I also thank Shih-Hsun Hsu and Yu-Lieh Huang for very helpful suggestions. The remaining errors are all mine.

Artificial neural networks

Ensemble methods are considered the state‐of‐the art solution for many machine learning challenges. Such methods improve the predictive performance of a single model by training multiple models and combining their predictions. This paper introduce the concept of ensemble learning, reviews traditional, novel and state‐of‐the‐art ensemble methods and discusses current challenges and trends in the field.

Ensemble learning: A survey

The need for appropriate ways to measure the distance or similarity between data is ubiquitous in machine learning, pattern recognition and data mining, but handcrafting such good metrics for specific problems is generally difficult. This has led to the emergence of metric learning, which aims at automatically learning a metric from data and has attracted a lot of interest in machine learning and related fields for the past ten years. This survey paper proposes a systematic review of the metric learning literature, highlighting the pros and cons of each approach. We pay particular attention to Mahalanobis distance metric learning, a well-studied and successful framework, but additionally present a wide range of methods that have recently emerged as powerful alternatives, including nonlinear metric learning, similarity learning and local metric learning. Recent trends and extensions, such as semi-supervised metric learning, metric learning for histogram data and the derivation of generalization guarantees, are also covered. Finally, this survey addresses metric learning for structured data, in particular edit distance learning, and attempts to give an overview of the remaining challenges in metric learning for the years to come.

A Survey on Metric Learning for Feature Vectors and Structured Data

In recent years, deep learning models have achieved remarkable successes in various applications, such as pattern recognition, computer vision, and signal processing. However, high-performance deep architectures are often accompanied by a large storage space and long computational time, which make it difficult to fully exploit many deep neural networks (DNNs), especially in scenarios in which computing resources are limited. In this paper, to tackle this problem, we introduce a method for compressing the structure and parameters of DNNs based on neuron agglomerative clustering (NAC). Specifically, we utilize the agglomerative clustering algorithm to find similar neurons, while these similar neurons and the connections linked to them are then agglomerated together. Using NAC, the number of parameters and the storage space of DNNs are greatly reduced, without the support of an extra library or hardware. Extensive experiments demonstrate that NAC is very effective for the neuron agglomeration of both the fully connected and convolutional layers, which are common building blocks of DNNs, delivering similar or even higher network accuracy. Specifically, on the benchmark CIFAR-10 and CIFAR-100 datasets, using NAC to compress the parameters of the original VGGNet by 92.96% and 81.10%, respectively, the compact network obtained still outperforms the original networks.

https://www.mdpi.com/1424-8220/20/21/6033/pdf

Compressing Deep Networks by Neuron Agglomerative Clustering.

With the sea-level rising, the measurement of sea surface height (SSH) is attracting more and more attention in the area of oceanography. Satellite radar altimeter is usually used to measure the SSH. However, deviation between the measured value and the actual one always exists. Among others, the sea state bias (SSB) is a main reason to cause the deviation. In general, one needs to estimate SSB first to correct the measured SSH. Currently, existing SSB estimation methods more or less have shortcomings, such as with many parameters, high prediction error and long training time. In this paper, we introduce an effective and efficient linear model called LASSO to the SSB estimation. The LASSO algorithm minimizes the residual sum of squares under the condition that the sum of the absolute values of each coefficient is less than a certain constant. In the implementation of LASSO, we use the significant wave height and wind speed to fit the LASSO model. Hence, the applied model has only 3 parameters, corresponding to the two inputs and a bias. Experimental results on the data of JASON-2, JASON-3, T/P and HY-2 radar altimetry show that LASSO performs better than geophysical data records (GDR) and ordinary least squares (OLS) estimator. Moreover, from the running time, we can see that LASSO is very efficient.

Sea State Bias Estimation with Least Absolute Shrinkage and Selection Operator (LASSO)

Sea state bias (SSB) is an important component of errors for the radar altimeter measurements of sea surface height (SSH). However, existing SSB estimation methods are almost all based on single-task learning (STL), where one model is built on the data from only one radar altimeter. In this paper, taking account of the data from multiple radar altimeters available, we introduced a multi-task learning method, called trace-norm regularized multi-task learning (TNR-MTL), for SSB estimation. Corresponding to each individual task, TNR-MLT involves only three parameters. Hence, it is easy to implement. More importantly, the convergence of TNR-MLT is theoretically guaranteed. Compared with the commonly used STL models, TNR-MTL can effectively utilize the shared information between data from multiple altimeters. During the training of TNR-MTL, we used the JASON-2 and JASON-3 cycle data to solve two correlated SSB estimation tasks. Then the optimal model was selected to estimate SSB on the JASON-2 and the HY-2 70–71 cycle intersection data. For the JSAON-2 cycle intersection data, the corrected variance (M) has been reduced by 0.60 cm2 compared to the geophysical data records (GDR); while for the HY-2 cycle intersection data, M has been reduced by 1.30 cm2 compared to GDR. Therefore, TNR-MTL is proved to be effective for the SSB estimation tasks.

Trace-Norm Regularized Multi-Task Learning for Sea State Bias Estimation

Few-shot classification aims to learn novel categories by giving few labeled samples. How to make best use of the limited data to obtain a learner with fast learning ability has become a challenging problem. In this paper, we propose a feature redirection network (FRNet) for few-shot classification to make the features more discriminative. The proposed FRNet not only highlights relevant category features of support samples, but also learns how to generate task-relevant features of query samples. Experiments conducted on three datasets have demonstrate its superiority over the state-of-the-art methods.

Feature Redirection Network for Few-Shot Classification

In recent years, explaining convolutional neural networks (CNN) has received increasing attention since it helps to build humans’ trust on CNNs by exposing their inference basis. In this field, generating intuitive saliency maps that highlight the input regions most related to model decisions is one of the popular approaches. Building on a state-of-the-art saliency method named Score-CAM, we propose Score-CAM++ to generate better saliency maps with higher efficiency (when compared to Score-CAM). Different from Score-CAM, which adopts all the feature maps in target layer to produce saliency maps, we propose the “feature map selection” operation to select the feature maps that capture the “positive” information (i.e., the features whose increment in intensity leads to an increase in target score). Then the selected feature maps are up-sampled as masks to perturb the input and compute the weights of the corresponding feature maps. Finally, the linear combination of the weighted feature maps forms the saliency map. Compared with Score-CAM, the experiments based on VGG-16 show that our method saves 25%-30% of the time when generating saliency maps. Meanwhile, our conducted evaluations, both subjective and objective, show that our method provides better visual explanations when compared to the previous saliency methods.

Guoqiang Zhong

Papers

Compressing Deep Networks by Neuron Agglomerative Clustering.

Sea State Bias Estimation with Least Absolute Shrinkage and Selection Operator (LASSO)

Trace-Norm Regularized Multi-Task Learning for Sea State Bias Estimation

Feature Redirection Network for Few-Shot Classification

Score-CAM++: Class Discriminative Localization with Feature Map Selection