“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

The identification of grape leaf diseases based on deep learning is critical to controlling the spread of diseases and ensuring the healthy development of the grape industry. Focusing on the lack of training images of grape leaf diseases, this paper proposes a novel model named Leaf GAN, which is based on generative adversarial networks (GANs), to generate images of four different grape leaf diseases for training identification models. A generator model with degressive channels is first designed to generate grape leaf disease images; then, the dense connectivity strategy and instance normalization are fused into an efficient discriminator to identify real and fake disease images by utilizing their excellent feature extraction capability on grape leaf lesions. Finally, the deep regret gradient penalty method is applied to stabilize the training process of the model. Using a total of 4,062 grape leaf disease images, the Leaf GAN model ultimately generates 8,124 grape leaf disease images. The generated grape leaf disease images based on Leaf GAN model can obtain better performance than DCGAN and WGAN in terms of the Frechet inception distance. The experimental results show that the proposed Leaf GAN model generates sufficient grape leaf disease images with prominent lesions, providing a feasible solution for the data augmentation of grape leaf disease images. For the eight prevailing classification models with the expanded dataset, the identification performance based on CNNs indicated higher accuracies, whereby all the accuracies were better than those of the initial dataset with other data augmentation methods. Among them, Xception achieves a recognition accuracy of 98.70% on the testing set. The results demonstrate that the proposed data augmentation method represents a new approach to overcoming the overfitting problem in disease identification and can effectively improve the identification accuracy.

/pdf/a-data-augmentation-method-based-on-generative-adversarial-2057w7jgyh.pdf

A Data Augmentation Method Based on Generative Adversarial Networks for Grape Leaf Disease Identification

Face detection in low light scenarios is challenging but vital to many practical applications, e.g., surveillance video, autonomous driving at night. Most existing face detectors heavily rely on extensive annotations, while collecting data is time-consuming and laborious. To reduce the burden of building new datasets for low light conditions, we make full use of existing normal light data and explore how to adapt face detectors from normal light to low light. The challenge of this task is that the gap between normal and low light is too huge and complex for both pixel-level and object-level. Therefore, most existing low-light enhancement and adaptation methods do not achieve desirable performance. To address the issue, we propose a joint High-Low Adaptation (HLA) framework. Through a bidirectional low-level adaptation and multi-task high-level adaptation scheme, our HLA-Face outperforms state-of-the-art methods even without using dark face labels for training. Our project is publicly available at: https://daooshee.github.io/HLA-Face-Website/.

/pdf/hla-face-joint-high-low-adaptation-for-low-light-face-1yr4xgnfyy.pdf

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection

Tactic analysis is a major issue in badminton as the effective usage of tactics is the key to win. The tactic in badminton is defined as a sequence of consecutive strokes. Most existing methods use statistical models to find sequential patterns of strokes and apply 2D visualizations such as glyphs and statistical charts to explore and analyze the discovered patterns. However, in badminton, spatial information like the shuttle trajectory, which is inherently 3D, is the core of a tactic. The lack of sufficient spatial awareness in 2D visualizations largely limited the tactic analysis of badminton. In this work, we collaborate with domain experts to study the tactic analysis of badminton in a 3D environment and propose an immersive visual analytics system, TIVEE, to assist users in exploring and explaining badminton tactics from multi-levels. Users can first explore various tactics from the third-person perspective using an unfolded visual presentation of stroke sequences. By selecting a tactic of interest, users can turn to the first-person perspective to perceive the detailed kinematic characteristics and explain its effects on the game result. The effectiveness and usefulness of TIVEE are demonstrated by case studies and an expert interview. 

TIVEE: Visual Exploration and Explanation of Badminton Tactics in Immersive Visualizations

Existing night vehicle detection methods mainly detect vehicles by detecting headlights or taillights. However, these features are adversely affected by the complex road lighting environment. In this paper, a cascade detection network framework FteGanOd is proposed with a feature translate-enhancement (FTE) module and the object detection (OD) module. First, the FTE module is built based on CycleGAN and multi-scale feature fusion is proposed to enhance the detection of vehicle features at night. The features of night and day are combined by fusing different convolutional layers to produce enhanced feature (EF) maps. Second, the OD module, based on the existing object detection network, is improved by cascading with the FTE module to detect vehicles on the EF maps. The proposed FteGanOd method recognizes vehicles at night with greater accuracy by improving the contrast between vehicles and the background and by suppressing interference from ambient light. The proposed FteGanOd is validated on the Berkeley Deep Drive (BDD) dataset and our private dataset. The experimental results show that our proposed method can effectively enhance vehicle features and improve the accuracy of vehicle detection at night.

/pdf/feature-enhancement-based-on-cyclegan-for-nighttime-vehicle-1k0u0agwsz.pdf

Feature Enhancement Based on CycleGAN for Nighttime Vehicle Detection

Camera-based blind-spot detection systems improve the shortcomings of radar-based systems for accurately detecting the position of a vehicle. However, as with many camera-based applications, the detection performance is insufficient in a low-illumination environment such as at night. This problem can be solved with augmented nighttime images in the training data but acquiring them and annotating the additional images are cumbersome tasks. Therefore, we propose a framework that converts daytime images into synthetic nighttime images using a generative adversarial network and that augments the synthetic images for the training process of the vehicle detector. A public dataset comprising different viewpoints of target images was used to easily obtain the images required for training the generative adversarial network. Experiments on a real nighttime dataset demonstrate that the proposed framework improved the detection performance considerably in comparison with using daytime images only.

Nighttime Data Augmentation Using GAN for Improving Blind-Spot Detection

We developed a method for the precise estimation of the 3D trajectory of a baseball by modeling the movement of the baseball and estimating the capture delay, using multiple unsynchronized cameras. To develop the proposed algorithm, we mimicked the real-world process of capturing a baseball in simulation space, and analyzed the capture process using a multiple unsynchronized camera system. We represented the movement of the baseball using a piece-wise spline model, and predicted the position of the baseball in the subframes in a manner which is robust to position error and change in direction of movement of the baseball. This method accurately predicts the baseball position over time by modeling the movement of the baseball in a real baseball game environment, and improves the accuracy of the reconstructed 3D baseball trajectories. We defined an objective function to estimate the capture delay, and estimate the optimal capture delay parameter using non-linear optimization method. In addition, we evaluated the performance of the proposed method in simulation space and in a real-world situation. The experimental results show that the proposed method can estimate a 3D baseball trajectory precisely using a multiple unsynchronized camera system and is robust to variations in capture delay, both in the simulation space and in real-world situations.

/pdf/precise-3d-baseball-pitching-trajectory-estimation-using-4kgfupglhv.pdf

Precise 3D Baseball Pitching Trajectory Estimation Using Multiple Unsynchronized Cameras

Semantic segmentation is essential for autonomous driving, which classifies roads and other objects in the image and provides pixel-level information. For high quality autonomous driving, it is necessary to consider the driving environment of the vehicle, and the vehicle speed should be controlled according to types of road. For this purpose, the semantic segmentation module has to classify types of road. However, current public datasets do not provide annotation data for these road types. In this paper, we propose a method to train the semantic segmentation model for classifying road types. We analyzed the problems that can occur when using a public dataset like KITTI or Cityscapes for training, and used Mapillary Vistas data as training data to get generalized performance. In addition, we use focal loss and over-sampling techniques to alleviate the class imbalance problem caused by relatively small class data.

Paved and Unpaved Road Segmentation Using Deep Neural Network

A baseball pitching trajectory analysis system is used to train pitchers and provide baseball content for viewers. Previous systems needed a system manager because the trajectory was detected manually, thus system automation is essential for popularizing baseball pitching trajectory analysis systems, and detecting the pitching start and end times comprise the key point toward achieving this. In this paper, we define the pitching start and end times according to the situations in baseball games and accurately detect these points in time based on a physical model. Experiments on image data captured at a professional baseball game indicate that the proposed method could detect the start and end times of a pitch as accurately as a human can.

Start-End Time Detection in Baseball Videos for Automatic Pitching Trajectory Analysis

In this paper, we propose a ball position-measurement method to evaluate the performance of a baseball pitching trajectory analysis system. To evaluate the performance of the trajectory analysis system, a method of accurately measuring the actual ball position is needed. Existing methods cannot perform a highly accurate, continuous performance evaluation of the pitching trajectory analysis system. The proposed method accurately measures ball position using measuring equipment by passing the pitch through a structure in which two cameras are installed. The results demonstrated that the error was smaller than the radius of the ball at 40 points in the structure, and the system can be used for performance evaluation of the pitching trajectory analysis system.

Hongjun Lee

Papers

Nighttime Data Augmentation Using GAN for Improving Blind-Spot Detection

Precise 3D Baseball Pitching Trajectory Estimation Using Multiple Unsynchronized Cameras

Paved and Unpaved Road Segmentation Using Deep Neural Network

Start-End Time Detection in Baseball Videos for Automatic Pitching Trajectory Analysis

A Method of Measuring Baseball Position at the Strike Zone