Pilar Rosado

Bio: Pilar Rosado is an academic researcher from University of Barcelona. The author has contributed to research in topics: Probabilistic latent semantic analysis & Generative model. The author has an hindex of 2, co-authored 5 publications receiving 11 citations.

Medical image editing in the latent space of Generative Adversarial Networks

Abstract: We consider a set of arithmetic operations in the latent space of Generative Adversarial Networks (GANs) to edit histopathological images. We analyze thousands of image patches from whole-slide images of breast cancer metastases in histological lymph node sections. Image files were downloaded from the pathology contests CAMELYON 16 and 17. We show that widely known architectures, such as: Deep Convolutional Generative Adversarial Networks (DCGAN) and Conditional Deep Convolutional Generative Adversarial Networks (cDCGAN), allow image editing using semantic concepts that represent underlying visual patterns in histopathological images, expanding GAN's well-known capabilities in medical image editing. We computed the Grad-cam heatmap of real positive images and of generated positive images, validating that the highlighted features both in the real and synthetic images match. We also show that GANs can be used to generate quality images, making GANs a valuable resource for augmenting small medical imaging datasets.

Artistic ideation based on computer vision methods

Abstract: This paper analyzes the automatic classification of scenes that are the basis of the ideation and the designing of the sculptural production of an artist. The main purpose is to evaluate the per- formance of the Bag-of-Features methods, in the challenging task of categorizing scenes when scenes differ in semantics rather than the objects they contain. We have employed a kernel-based recognition method that works by computing rough geometric correspondence on a global scale using the pyramid matching scheme introduced by Lazebnik (7). Results are promising, on average the score is about 70%. Experiments suggest that the automatic categorization of images based on computer vision methods can provide objective principles in cataloging images. Image representation is a very important element for image classification, annotation, seg- mentation or retrieval. Nearly all the methods in computer vision which deals with image content representation resort to features capable of representing image content in a compact way. Local features based representation can produce a versatile and robust image representa- tion capable of representing global and local content at the same time. Describing an object or scene using local features computed at interest locations makes the description robust to par- tial occlusion and image transformation. This results from the local character of the features and their invariance to image transformations. The bag-of-visterms (BOV) is an image representation built from automatically extracted and quantized local descriptors referred to as visterms in the remainder of this paper. The BOV representation, which is derived from these local features, has been shown to be one of the best image representations in several tasks. The main objective of this study is assessing the performance of SIFT descriptors, BOV representation and spatial pyramid matching for automatic analysis of images that are the basis of the ideation and designing of art work. Additionally, we explore the capability of this kind of modelization to become useful for the production of software based art.

Semantic-Based Image Analysis with the Goal of Assisting Artistic Creation

Abstract: We have approached the difficulties of automatic cataloguing of images on which the conception and design of sculptor M. Planas artistic production are based. In order to build up a visual vocabulary for basing image description on, we followed a procedure similar to the method Bag-of-Words (BOW). We have implemented a probabilistic latent semantic analysis (PLSA) that detects underlying topics in images. Whole image collection was clustered into different types that describe aesthetic preferences of the artist. The outcomes are promising, the described cataloguing method may provide new viewpoints for the artist in future works.

Computer Vision Models to Categorize Art Collections According to the Visual Content: A New Approach to the Abstract Art of Antoni Tàpies

Abstract: This study uses computer vision models, which to some extent simulate the initial stages of human visual perception, to help categorize data in large sets of images of artworks by the artist Antoni...

GANs and Artificial Facial Expressions in Synthetic Portraits

Abstract: Generative adversarial networks (GANs) provide powerful architectures for deep generative learning. GANs have enabled us to achieve an unprecedented degree of realism in the creation of synthetic images of human faces, landscapes, and buildings, among others. Not only image generation, but also image manipulation is possible with GANs. Generative deep learning models are inherently limited in their creative abilities because of a focus on learning for perfection. We investigated the potential of GAN’s latent spaces to encode human expressions, highlighting creative interest for suboptimal solutions rather than perfect reproductions, in pursuit of the artistic concept. We have trained Deep Convolutional GAN (DCGAN) and StyleGAN using a collection of portraits of detained persons, portraits of dead people who died of violent causes, and people whose portraits were taken during an orgasm. We present results which diverge from standard usage of GANs with the specific intention of producing portraits that may assist us in the representation and recognition of otherness in contemporary identity construction.

Automated Visual Content Analysis (AVCA) in Communication Research: A Protocol for Large Scale Image Classification with Pre-Trained Computer Vision Models

Abstract: The increasing volume of images published online in a wide variety of contexts requires communication researchers to address this reality by analyzing visual content at a large scale. Ongoing advan...

Improving Skin Cancer Classification Using Heavy-Tailed Student T-Distribution in Generative Adversarial Networks (TED-GAN).

Abstract: Deep learning has gained immense attention from researchers in medicine, especially in medical imaging. The main bottleneck is the unavailability of sufficiently large medical datasets required for the good performance of deep learning models. This paper proposes a new framework consisting of one variational autoencoder (VAE), two generative adversarial networks, and one auxiliary classifier to artificially generate realistic-looking skin lesion images and improve classification performance. We first train the encoder-decoder network to obtain the latent noise vector with the image manifold’s information and let the generative adversarial network sample the input from this informative noise vector in order to generate the skin lesion images. The use of informative noise allows the GAN to avoid mode collapse and creates faster convergence. To improve the diversity in the generated images, we use another GAN with an auxiliary classifier, which samples the noise vector from a heavy-tailed student t-distribution instead of a random noise Gaussian distribution. The proposed framework was named TED-GAN, with T from the t-distribution and ED from the encoder-decoder network which is part of the solution. The proposed framework could be used in a broad range of areas in medical imaging. We used it here to generate skin lesion images and have obtained an improved classification performance on the skin lesion classification task, rising from 66% average accuracy to 92.5%. The results show that TED-GAN has a better impact on the classification task because of its diverse range of generated images due to the use of a heavy-tailed t-distribution.

Brain Programming is Immune to Adversarial Attacks: Towards Accurate and Robust Image Classification using Symbolic Learning.

Abstract: In recent years, the security concerns about the vulnerability of Deep Convolutional Neural Networks (DCNN) to Adversarial Attacks (AA) in the form of small modifications to the input image almost invisible to human vision make their predictions untrustworthy. Therefore, it is necessary to provide robustness to adversarial examples in addition to an accurate score when developing a new classifier. In this work, we perform a comparative study of the effects of AA on the complex problem of art media categorization, which involves a sophisticated analysis of features to classify a fine collection of artworks. We tested a prevailing bag of visual words approach from computer vision, four state-of-the-art DCNN models (AlexNet, VGG, ResNet, ResNet101), and the Brain Programming (BP) algorithm. In this study, we analyze the algorithms' performance using accuracy. Besides, we use the accuracy ratio between adversarial examples and clean images to measure robustness. Moreover, we propose a statistical analysis of each classifier's predictions' confidence to corroborate the results. We confirm that BP predictions' change was below 2\% using adversarial examples computed with the fast gradient sign method. Also, considering the multiple pixel attack, BP obtained four out of seven classes without changes and the rest with a maximum error of 4\% in the predictions. Finally, BP also gets four categories using adversarial patches without changes and for the remaining three classes with a variation of 1\%. Additionally, the statistical analysis showed that the predictions' confidence of BP were not significantly different for each pair of clean and perturbed images in every experiment. These results prove BP's robustness against adversarial examples compared to DCNN and handcrafted features methods, whose performance on the art media classification was compromised with the proposed perturbations.

Brain programming is immune to adversarial attacks: Towards accurate and robust image classification using symbolic learning

Abstract: In recent years, the security concerns about the vulnerability of Deep Convolutional Neural Networks (DCNN) to Adversarial Attacks (AA) in the form of small modifications to the input image almost invisible to human vision make their predictions untrustworthy. Therefore, it is necessary to provide robustness to adversarial examples in addition to an accurate score when developing a new classifier. In this work, we perform a comparative study of the effects of AA on the complex problem of art media categorization, which involves a sophisticated analysis of features to classify a fine collection of artworks. We tested a prevailing bag of visual words approach from computer vision, four state-of-the-art DCNN models (AlexNet, VGG, ResNet, ResNet101), and the Brain Programming (BP) algorithm. In this study, we analyze the algorithms' performance using accuracy. Besides, we use the accuracy ratio between adversarial examples and clean images to measure robustness. Moreover, we propose a statistical analysis of each classifier's predictions' confidence to corroborate the results. We confirm that BP predictions' change was below 2\% using adversarial examples computed with the fast gradient sign method. Also, considering the multiple pixel attack, BP obtained four out of seven classes without changes and the rest with a maximum error of 4\% in the predictions. Finally, BP also gets four categories using adversarial patches without changes and for the remaining three classes with a variation of 1\%. Additionally, the statistical analysis showed that the predictions' confidence of BP were not significantly different for each pair of clean and perturbed images in every experiment. These results prove BP's robustness against adversarial examples compared to DCNN and handcrafted features methods, whose performance on the art media classification was compromised with the proposed perturbations.

Semantic-Based Image Analysis with the Goal of Assisting Artistic Creation

Abstract: We have approached the difficulties of automatic cataloguing of images on which the conception and design of sculptor M. Planas artistic production are based. In order to build up a visual vocabulary for basing image description on, we followed a procedure similar to the method Bag-of-Words (BOW). We have implemented a probabilistic latent semantic analysis (PLSA) that detects underlying topics in images. Whole image collection was clustered into different types that describe aesthetic preferences of the artist. The outcomes are promising, the described cataloguing method may provide new viewpoints for the artist in future works.