“Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告

我的台灣, 看見心靈的故鄉 =2009林磐聳藝術與設計展

This study proposes a novel unified and unsupervised end-to-end image fusion network, termed as U2Fusion, which is capable of solving different fusion problems, including multi-modal, multi-exposure, and multi-focus cases. Using feature extraction and information measurement, U2Fusion automatically estimates the importance of corresponding source images and comes up with adaptive information preservation degrees. Hence, different fusion tasks are unified in the same framework. Based on the adaptive degrees, a network is trained to preserve the adaptive similarity between the fusion result and source images. Therefore, the stumbling blocks in applying deep learning for image fusion, e.g., the requirement of ground-truth and specifically designed metrics, are greatly mitigated. By avoiding the loss of previous fusion capabilities when training a single model for different tasks sequentially, we obtain a unified model that is applicable to multiple fusion tasks. Moreover, a new aligned infrared and visible image dataset, RoadScene (available at https://github.com/hanna-xu/RoadScene), is released to provide a new option for benchmark evaluation. Qualitative and quantitative experimental results on three typical image fusion tasks validate the effectiveness and universality of U2Fusion. Our code is publicly available at https://github.com/hanna-xu/U2Fusion.

U2Fusion: A Unified Unsupervised Image Fusion Network.

We introduce an unsupervised machine learning (ML) based technique for the identification and characterization of microstructures in three-dimensional (3D) samples obtained from molecular dynamics simulations, particle tracking data, or experiments. Our technique combines topology classification, image processing, and clustering algorithms, and can handle a wide range of microstructure types including grains in polycrystalline materials, voids in porous systems, and structures from self/directed assembly in soft-matter complex solutions. Our technique does not require a priori microstructure description of the target system and is insensitive to disorder such as extended defects in polycrystals arising from line and plane defects. We demonstrate quantitively that our technique provides unbiased microstructural information such as precise quantification of grains and their size distributions in 3D polycrystalline samples, characterizes features such as voids and porosity in 3D polymeric samples and micellar size distribution in 3D complex fluids. To demonstrate the efficacy of our ML approach, we benchmark it against a diverse set of synthetic data samples representing nanocrystalline metals, polymers and complex fluids as well as experimentally published characterization data. Our technique is computationally efficient and provides a way to quickly identify, track, and quantify complex microstructural features that impact the observed material behavior.

/pdf/machine-learning-enabled-autonomous-microstructural-58yy2bofk1.pdf

Machine learning enabled autonomous microstructural characterization in 3D samples

In this paper, we propose a fast unified image fusion network based on proportional maintenance of gradient and intensity (PMGI), which can end-to-end realize a variety of image fusion tasks, including infrared and visible image fusion, multi-exposure image fusion, medical image fusion, multi-focus image fusion and pan-sharpening. We unify the image fusion problem into the texture and intensity proportional maintenance problem of the source images. On the one hand, the network is divided into gradient path and intensity path for information extraction. We perform feature reuse in the same path to avoid loss of information due to convolution. At the same time, we introduce the pathwise transfer block to exchange information between different paths, which can not only pre-fuse the gradient information and intensity information, but also enhance the information to be processed later. On the other hand, we define a uniform form of loss function based on these two kinds of information, which can adapt to different fusion tasks. Experiments on publicly available datasets demonstrate the superiority of our PMGI over the state-of-the-art in terms of both visual effect and quantitative metric in a variety of fusion tasks. In addition, our method is faster compared with the state-of-the-art.

/pdf/rethinking-the-image-fusion-a-fast-unified-image-fusion-4hg5lvqf7e.pdf

Rethinking the Image Fusion: A Fast Unified Image Fusion Network based on Proportional Maintenance of Gradient and Intensity

Muti-focus image fusion is the extraction of focused regions from different images to create one all-in-focus fused image. The key point is that only objects within the depth-of-field have a sharp appearance in the photograph, while other objects are likely to be blurred. We propose an unsupervised deep learning model for multi-focus image fusion. We train an encoder–decoder network in an unsupervised manner to acquire deep features of input images. Then, we utilize spatial frequency, a gradient-based method to measure sharp variation from these deep features, to reflect activity levels. We apply some consistency verification methods to adjust the decision map and draw out the fused result. Our method analyzes sharp appearances in deep features instead of original images, which can be seen as another success story of unsupervised learning in image processing. Experimental results demonstrate that the proposed method achieves state-of-the-art fusion performance compared to 16 fusion methods in objective and subjective assessments, especially in gradient-based fusion metrics.

SESF-Fuse: an unsupervised deep model for multi-focus image fusion

The roles of grain orientation and grain boundary characteristics in the enhanced superelasticity of Cu71.8Al17.8Mn10.4 shape memory alloys

Superelastic anisotropy characteristics of columnar-grained Cu–Al–Mn shape memory alloys and its potential applications

Quantitative analysis through image processing is a key step to gain information regarding the microstructure of materials. In this paper, we develop a deep learning-based method to address the task of image segmentation for microscopic images using an Al–La alloy. Our work makes three key contributions. (1) We train a deep convolutional neural network based on DeepLab to achieve image segmentation and have significant results. (2) We adopt a local processing method based on symmetric overlap-tile strategy which makes it possible to analyze the microscopic images with high resolution. Additionally, it achieves seamless segmentation. (3) We apply symmetric rectification to enhance the accuracy of results with 3D information. Experimental results showed that our method outperforms existing segmentation methods.

Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images

Designing a material with multiple desired properties is a great challenge, especially in a complex material system. Here, we propose a material design strategy to simultaneously optimize multiple targeted properties of multi-component Co-base superalloys via machine learning. The microstructural stability, γ′ solvus temperature, γ′ volume fraction, density, processing window, freezing range, and oxidation resistance were simultaneously optimized. A series of novel Co-base superalloys were successfully selected and experimentally synthesized from >210,000 candidates. The best performer, Co-36Ni-12Al-2Ti-4Ta-1W-2Cr, possesses the highest γ′ solvus temperature of 1266.5 °C without the precipitation of any deleterious phases, a γ′ volume fraction of 74.5% after aging for 1000 h at 1000 °C, a density of 8.68 g cm−3 and good high-temperature oxidation resistance at 1000 °C due to the formation of a protective alumina layer. Our approach paves a new way to rapidly design multi-component materials with desired multi-performance functionality.

/pdf/machine-learning-assisted-design-of-g-strengthened-co-base-5b9f0rky3w.pdf

Haiyou Huang

Papers

SESF-Fuse: an unsupervised deep model for multi-focus image fusion

The roles of grain orientation and grain boundary characteristics in the enhanced superelasticity of Cu71.8Al17.8Mn10.4 shape memory alloys

Superelastic anisotropy characteristics of columnar-grained Cu–Al–Mn shape memory alloys and its potential applications

Deep Learning-Based Image Segmentation for Al-La Alloy Microscopic Images

Machine learning assisted design of γ′-strengthened Co-base superalloys with multi-performance optimization