The computation represented by a sensory neuron's response to stimuli is constructed from an array of physiological processes both belonging to that neuron and inherited from its inputs. Although many of these physiological processes are known to be nonlinear, linear approximations are commonly used to describe the stimulus selectivity of sensory neurons (i.e., linear receptive fields). Here we present an approach for modeling sensory processing, termed the Nonlinear Input Model (NIM), which is based on the hypothesis that the dominant nonlinearities imposed by physiological mechanisms arise from rectification of a neuron's inputs. Incorporating such ‘upstream nonlinearities’ within the standard linear-nonlinear (LN) cascade modeling structure implicitly allows for the identification of multiple stimulus features driving a neuron's response, which become directly interpretable as either excitatory or inhibitory. Because its form is analogous to an integrate-and-fire neuron receiving excitatory and inhibitory inputs, model fitting can be guided by prior knowledge about the inputs to a given neuron, and elements of the resulting model can often result in specific physiological predictions. Furthermore, by providing an explicit probabilistic model with a relatively simple nonlinear structure, its parameters can be efficiently optimized and appropriately regularized. Parameter estimation is robust and efficient even with large numbers of model components and in the context of high-dimensional stimuli with complex statistical structure (e.g. natural stimuli). We describe detailed methods for estimating the model parameters, and illustrate the advantages of the NIM using a range of example sensory neurons in the visual and auditory systems. We thus present a modeling framework that can capture a broad range of nonlinear response functions while providing physiologically interpretable descriptions of neural computation.

/pdf/inferring-nonlinear-neuronal-computation-based-on-1i0m9km2pw.pdf

Inferring nonlinear neuronal computation based on physiologically plausible inputs.

When remembering a real-world scene, people encode both detailed information about specific objects and higher order information like the overall gist of the scene. However, formal models of change detection, like those used to estimate visual working memory capacity, assume observers encode only a simple memory representation that includes no higher order structure and treats items independently from one another. We present a probabilistic model of change detection that attempts to bridge this gap by formalizing the role of perceptual organization and allowing for richer, more structured memory representations. Using either standard visual working memory displays or displays in which the items are purposefully arranged in patterns, we find that models that take into account perceptual grouping between items and the encoding of higher order summary information are necessary to account for human change detection performance. Considering the higher order structure of items in visual working memory will be critical for models to make useful predictions about observers' memory capacity and change detection abilities in simple displays as well as in more natural scenes.

/pdf/a-probabilistic-model-of-visual-working-memory-incorporating-1kkmnybyn6.pdf

A probabilistic model of visual working memory: Incorporating higher order regularities into working memory capacity estimates.

This paper presents an approach for automatically creating graphic design layouts using a new energy-based model derived from design principles. The model includes several new algorithms for analyzing graphic designs, including the prediction of perceived importance, alignment detection, and hierarchical segmentation. Given the model, we use optimization to synthesize new layouts for a variety of single-page graphic designs. Model parameters are learned with Nonlinear Inverse Optimization (NIO) from a small number of example layouts. To demonstrate our approach, we show results for applications including generating design layouts in various styles, retargeting designs to new sizes, and improving existing designs. We also compare our automatic results with designs created using crowdsourcing and show that our approach performs slightly better than novice designers.

Learning Layouts for Single-PageGraphic Designs

Feedback on designs is critical for helping users iterate toward effective solutions. This paper presents Voyant, a novel system giving users access to a non-expert crowd to receive perception-oriented feedback on their designs from a selected audience. Based on a formative study, the system generates the elements seen in a design, the order in which elements are noticed, impressions formed when the design is first viewed, and interpretation of the design relative to guidelines in the domain and the user's stated goals. An evaluation of the system was conducted with users and their designs. Users reported the feedback about impressions and interpretation of their goals was most helpful, though the other feedback types were also valued. Users found the coordinated views in Voyant useful for analyzing relations between the crowd's perception of a design and the visual elements within it. The cost of generating the feedback was considered a reasonable tradeoff for not having to organize critiques or interrupt peers.

Voyant: generating structured feedback on visual designs using a crowd of non-experts

Methods and systems for placing an overlay in video content are provided. A method receives video content and input indicating an overlay to be placed in the video. The method determines, based on overlay and video properties, locations where the overlay can be placed. The method presents suggested locations for the overlay and receives a selection of a suggested location. The overlay is placed in the selected location. A system includes memory with instructions for inserting an overlay into video content. The system receives an indication of an overlay to be placed in the video, performs attention modeling on the video to identify zones likely to be of interest to a viewer. The system presents locations within the identified zones where the overlay can be inserted and receives a selection of a location. The system inserts the overlay into the selected location and renders the video with the inserted overlay.

Placing unobtrusive overlays in video content

Understanding and exploiting the abilities of the human visual system is an important part of the design of usable user interfaces and information visualizations. Good design enables quick, easy and veridical perception of key components of that design. An important facet of human vision is its ability to seemingly effortlessly perform "perceptual organization; it transforms individual feature estimates into perception of coherent regions, structures, and objects. We perceive regions grouped by proximity and feature similarity, grouping of curves by good continuation, and grouping of regions of coherent texture. In this paper, we discuss a simple model for a broad range of perceptual grouping phenomena. It takes as input an arbitrary image, and returns a structure describing the predicted visual organization of the image. We demonstrate that this model can capture aspects of traditional design rules, and predicts visual percepts in classic perceptual grouping displays.

/pdf/an-intuitive-model-of-perceptual-grouping-for-hci-design-5bt1mblwjc.pdf

An intuitive model of perceptual grouping for HCI design

Intracellular studies have revealed the importance of cotuned excitatory and inhibitory inputs to neurons in auditory cortex, but typical spectrotemporal receptive field models of neuronal processing cannot account for this overlapping tuning. Here, we apply a new nonlinear modeling framework to extracellular data recorded from primary auditory cortex (A1) that enables us to explore how the interplay of excitation and inhibition contributes to the processing of complex natural sounds. The resulting description produces more accurate predictions of observed spike trains than the linear spectrotemporal model, and the properties of excitation and inhibition inferred by the model are furthermore consistent with previous intracellular observations. It can also describe several nonlinear properties of A1 that are not captured by linear models, including intensity tuning and selectivity to sound onsets and offsets. These results thus offer a broader picture of the computational role of excitation and inhibition in A1 and support the hypothesis that their interactions play an important role in the processing of natural auditory stimuli.

Inferring the role of inhibition in auditory processing of complex natural stimuli.

Basic perceptual quality of coded audio material is commonly evaluated using ITU-R BS-1534 MUSHRA (Multi Stimulus with Hidden Reference and Anchors) listening tests. MUSHRA guidelines call for experienced listeners. However, the majority of consumers using the final product are no expert-listeners. Also the degree of expertise in a listening test may vary amongst listeners in the same laboratory. It would be useful to know how the audio quality evaluation differs between trained and untrained listeners and how training and actual tests should be designed in order to be as reliable as possible. To investigate the rating differences between experts and non-experts, we performed MUSHRA listening tests with 13 experienced and 11 inexperienced listeners using 5 speech and audio codecs delivering a wide range of basic audio quality. Except for the hidden reference, absolute ratings of non-experts were consistently higher than those of experts. However, rank order only rarely changed between experts and non-experts. For lower quality values, confidence intervals were significantly larger for non-experts than for experts. Experienced listeners set more than twice as many loops as non-experts, compared more often between codecs and listened to high quality codecs for a longer duration than non-experts.

Audio quality evaluation by experienced and inexperienced listeners

Background Ecological momentary assessment (EMA) is a method to evaluate hearing aids in everyday life that uses repeated smartphone-based questionnaires to assess a situation as it happens. Although being ecologically valid and avoiding memory bias, this method may be prone to selection biases due to questionnaires being skipped or the phone not being carried along in certain situations. Purpose This investigation analyzed which situations are underrepresented in questionnaire responses and physically measured objective EMA data (e.g., sound level), and how such underrepresentation may depend on different triggers. Method In an EMA study, 20 subjects with hearing impairment provided daily information on reasons for missed data, that is, skipped questionnaires or missing connections between their phone and hearing aids. Results Participants often deliberately did not bring the study phone to social situations or skipped questionnaires because they considered it inappropriate, for example, during church service or when engaging in conversation. They answered fewer questions in conversations with multiple partners and were more likely to postpone questionnaires when not in quiet environments. Conclusion Data for social situations will likely be underrepresented in EMA. However, these situations are particularly important for the evaluation of hearing aids, as individuals with hearing impairment often have difficulties communicating in noisy situations. Thus, it is vital to optimize the design of the study to find a balance between avoiding memory bias and enabling subjects to report retrospectively on situations where phone usage may be difficult. The implications for several applications of EMA are discussed. Supplemental Material https://doi.org/10.23641/asha.12746849.

https://pubs.asha.org/doi/pdf/10.1044/2020_AJA-19-00075

Nadja Schinkel-Bielefeld

Papers

An intuitive model of perceptual grouping for HCI design

Inferring the role of inhibition in auditory processing of complex natural stimuli.

Audio quality evaluation by experienced and inexperienced listeners

Evaluation of Hearing Aids in Everyday Life Using Ecological Momentary Assessment: What Situations Are We Missing?

Generation and Evaluation of Isolated Audio Coding Artifacts