Showing papers by "Andrew C. Gallagher published in 2015"

A mixed bag of emotions: Model, predict, and transfer emotion distributions

Abstract: This paper explores two new aspects of photos and human emotions. First, we show through psychovisual studies that different people have different emotional reactions to the same image, which is a strong and novel departure from previous work that only records and predicts a single dominant emotion for each image. Our studies also show that the same person may have multiple emotional reactions to one image. Predicting emotions in “distributions” instead of a single dominant emotion is important for many applications. Second, we show not only that we can often change the evoked emotion of an image by adjusting color tone and texture related features but also that we can choose in which “emotional direction” this change occurs by selecting a target image. In addition, we present a new database, Emotion6, containing distributions of emotions.

3D Reasoning from Blocks to Stability

Abstract: Objects occupy physical space and obey physical laws. To truly understand a scene, we must reason about the space that objects in it occupy, and how each objects is supported stably by each other. In other words, we seek to understand which objects would, if moved, cause other objects to fall. This 3D volumetric reasoning is important for many scene understanding tasks, ranging from segmentation of objects to perception of a rich 3D, physically well-founded, interpretations of the scene. In this paper, we propose a new algorithm to parse a single RGB-D image with 3D block units while jointly reasoning about the segments, volumes, supporting relationships, and object stability. Our algorithm is based on the intuition that a good 3D representation of the scene is one that fits the depth data well, and is a stable, self-supporting arrangement of objects (i.e., one that does not topple). We design an energy function for representing the quality of the block representation based on these properties. Our algorithm fits 3D blocks to the depth values corresponding to image segments, and iteratively optimizes the energy function. Our proposed algorithm is the first to consider stability of objects in complex arrangements for reasoning about the underlying structure of the scene. Experimental results show that our stability-reasoning framework improves RGB-D segmentation and scene volumetric representation.

VIP: Finding important people in images

Abstract: People preserve memories of events such as birthdays, weddings, or vacations by capturing photos, often depicting groups of people. Invariably, some individuals in the image are more important than others given the context of the event. This paper analyzes the concept of the importance of individuals in group photographs. We address two specific questions - Given an image, who are the most important individuals in it? Given multiple images of a person, which image depicts the person in the most important role? We introduce a measure of importance of people in images and investigate the correlation between importance and visual saliency. We find that not only can we automatically predict the importance of people from purely visual cues, incorporating this predicted importance results in significant improvement in applications such as im2text (generating sentences that describe images of groups of people).

VIP: Finding Important People in Images

Abstract: People preserve memories of events such as birthdays, weddings, or vacations by capturing photos, often depicting groups of people. Invariably, some individuals in the image are more important than others given the context of the event. This paper analyzes the concept of the importance of individuals in group photographs. We address two specific questions -- Given an image, who are the most important individuals in it? Given multiple images of a person, which image depicts the person in the most important role? We introduce a measure of importance of people in images and investigate the correlation between importance and visual saliency. We find that not only can we automatically predict the importance of people from purely visual cues, incorporating this predicted importance results in significant improvement in applications such as im2text (generating sentences that describe images of groups of people).

Sharing digital media assets for presentation within an online social network

Abstract: A method for reducing the number of images or the length of a video from a digital image collection using a social network, includes receiving a digital image collection captured by a user to be viewed by a viewer; wherein the viewer and the user are members of the same social network and using a processor to access the social network to determine a relationship between a viewer and the user. The method further includes using the processor to determine a set of summarization parameters based on the relationship between the viewer and the user and using the processor to reduce the number of images or the length of the video from the digital image collection using the determined set of summarization parameters to be viewed by the viewer.

Personalized Travel Navigation and Photo-Shooting Navigation Using Large-Scale Geotags

Abstract: In this chapter, a geotag-based inter/intra-city travel and photo-shooting navigation system that considers both the personal preference and the seasonal/temporal popularity is presented. For the inter-city travel navigation, similarity among users is efficiently calculated by combining our visit pattern similarity and photo shooting pattern similarity. Accurate intra-city travel navigation is achieved by incorporating the seasonal and temporal information into a Markov model. Photo-shooting navigation by using large-scale geotagged photos is also presented. The effectiveness of the proposed algorithms have been experimentally demonstrated by using more than millions of geo-tags and photos downloaded from Flickr.