Showing papers in "Visual Communication in 2022"

Drone views: a multimodal ethnographic perspective

Abstract: Drone visuals are rapidly becoming part of our sociocultural imaginaries, generating distinct images that differ from traditional visual conventions and producing unexpected perspectives of the world that reveal hidden aspects of our surroundings. Despite the growing use of camera-laden drones in a range of commercial and non-commercial activities, to date, little scholarly attention has been paid to the semiotics of drone visuals. This article is the first to draw specific attention to the compositional structure of drone visuals, combining social semiotic analysis with ethnographic insights to assess how they are changing the way we think about the world. Exploring drone hobbyists’ and developers’ perspectives on drone usage and the visuals they generate, the authors identify and examine three frequently occurring characteristics of drone visuals: top-down views, 360-degree panoramic views and ‘classic’ landscape perspectives. The critical analysis of these peculiarities leads them to argue for the potential of these innovative visions to reshape our visual culture. In their conclusion, the authors aim to open a conversation about the way technological advancements mark important sociocultural changes in sense-making processes, geographical imaginations and everyday life experiences.

Walking through the city soundscape: an audio-visual analysis of sensory experience for people with psychosis

Abstract: This article discusses how an aspect of urban environments – sound and noise – is experienced by people walking in the city; it particularly focuses on atypical populations such as people diagnosed with psychosis, who are reported to be particularly sensitive to noisy environments. Through an analysis of video-recordings of naturalistic activities in an urban context and of video-elicitations based on these recordings, the study details the way participants orient to sound and noise in naturalistic settings, and how sound and noise are reported and reexperienced during interviews. By bringing together urban context, psychosis and social interaction, this study shows that, thanks to video recordings and conversation analysis, it is possible to analyse in detail the multimodal organization of action (talk, gesture, gaze, walking bodies) and of the sensory experience(s) of aural factors, as well as the way this organization is affected by the ecology of the situation.

The rhythms of cancer survivorship

Abstract: This visual essay incorporates photographs from a research project on the changing landscapes of cancer survivorship in Australia. Study participants were asked to tell the story of what cancer looks and feels like and what it means to them. The photographs were captioned and discussed during a follow-up interview. Employing Henri Lefebvre’s rhythmanalysis, these photographs, captions and narrations show how the routines and expectations arising in cancer survivorship reorient life around movement in time and place. They provide insights into the unfolding of cancer survivorship in the ebb and flow of lived experience.

Evading Big Brother: Using visual methods to understand children’s perception of sensors and interest in subverting digital surveillance

Abstract: In relation to this Special Issue’s focus on ugly information, this article examines children’s perception of the often invisible interactions they have with sensor-enabled digital devices and, when prompted, their interest in subverting or blocking these sensors to evade surveillance. The authors report on a study of 12 children, aged 8–12 years, that investigated their knowledge of the sensing abilities of commonly used digital devices (smart phones, smart watches, smart speakers and games consoles), and their attitudes towards having active agency over sensors. In line with this journal’s readership, visual methods used for data collection and analysis are described. Specifically, within semi-structured focus groups, drawing was used to understand what children thought was inside digital devices and the extent of their awareness of digital sensors. Child participants were invited to model speculative tools for deceiving digital sensors in order to explore their interest in having agency over digital surveillance. Data in the form of drawings, photographs of models and video recordings were analysed using experimental visual methods that included 3D rendering and comics, as well as visual content and thematic analysis. These drew out four key themes: (1) the role of inference in sensor awareness; (2) misunderstanding of device components and sensing capabilities; (3) attitudes to surveillance; and (4) children’s interest in subverting rather than blocking sensors. We discuss how technology companies’ desire to create ‘magical experiences’ may contribute to incorrect inferences about information gathering systems, how this reduces children’s agency over the information they share and how it puts them at greater risk from digital surveillance. The article makes an original contribution to knowledge in this area by calling for a two-pronged approach from technology companies and educators to address these issues by making sensor presence more visible, educating children about the full extent of sensor capability and bringing critical discussion of them into curricula.

‘Foodstagramming’ in Early 20th-Century Postcards: A Transhistorical Perspective

Abstract: Introduced in 1907, the ‘real photo’ postcard destabilized the boundaries between private and public life, enabling people to perform identity in ways that anticipate contemporary social media practices. In this visual essay, the author explores one particular phenomenon – the sharing of food – drawing comparisons with ‘foodstagramming’ in terms of its compositional structure, social objectives and communicative functions. In doing so, she challenges the supposed novelty of modes of self-presentation on social media, embedding them in a broader historical trajectory.

The politics of typographic placemaking: the cases of TilburgsAns and Dubai Font

Abstract: This article explores typographic placemaking by comparing the design and public launch of two city fonts: TilburgsAns (2016) and Dubai Font (2017). Building on recent work on semiotic technology and graphic ideology, the authors examine how these fonts’ visual features and the promotional discourses surrounding their launch are utilized for placemaking, and how this is facilitated and constrained by technology and ideology. The results show that the two projects of typographic placemaking build on a similar repertoire of semiotic technology, but make different use of it. The authors sustain that this difference is explained by the political aims of the two projects, on the one hand, and their economic and organizational scale, on the other. A postcolonial perspective further underlines their geopolitically and historically different preconditions.

Analyzing picturebooks: semiotic, literary, and artistic frameworks

Abstract: The multimodal and visual nature of children’s picturebooks has been documented in research emanating from multiple fields of inquiry. In this article, the authors present three types of analytical frameworks that are useful for conducting research on contemporary picturebooks as multimodal entities. Each framework draws upon different aspects of visual images, design features, and written language, and uses different theoretical lenses to call forth particular aspects of contemporary picturebooks. The three analytical frameworks are: (1) social semiotic frameworks, (2) literary frameworks, and (3) artistic frameworks. This article suggests that only through an orchestration of a range of analytical frameworks can scholars and educators begin to understand the complexity of contemporary picturebooks and their role in educational settings.

Design timescapes: futuring through visual thinking

Abstract: Encouraging designers to think about the precedents and consequences of their designs is integral to generating a design ethic that respects both our past and future generations (see Escobar, ‘Sustainability: Design for the pluriverse’, 2011, and Fry, Design Futuring, 2009). Situating designs as interventions in time also clearly acknowledges our growing responsibilities as designers in the age of the Anthropocene. The visualization of these relationships serves not only the designer engaged in the research, but those from other disciplines seeking to understand what historical or sociopolitical contexts may have informed a particular design innovation at a particular moment in time. In this reflective practitioner piece, the author presents design timescapes, a novel visual thinking tool that not only challenges designers to visualize the relationships between design and societal shifts but encourages the development of visual argumentation for design proposals. This approach is also useful in introducing the concept of design futuring to students/designers unfamiliar with this emergent field. To illustrate the various manifestations of this tool, she shares examples of where she has applied design timescapes as part of her futuring practice, and as a pedagogical tool. She concludes by offering suggestions for how this tool, in combination with emerging design futuring practice, contributes to the expansion of the resources of visual communication, design practice, research, and education.

Visual narratives of environmental change: collective memory and identity at New Zealand heritage sites

Abstract: This article interrogates historical photographs exhibited at public heritage sites in Aotearoa New Zealand. The analysis reveals that – by portraying 19th-century environmental change as a ‘heroic’ narrative of ‘progress’ – the photographs construct New Zealand national identity in opposition to nature, rather than promote a sense of connectedness with the natural environment. The article thus makes three important contributions to the literature on the visualization of environmental and climate change. First, the empirical case study demonstrates that visual narratives shape our social identities in relation to nature. Second, the article adds a rare socio-semiotic analysis to the environmental communication literature, highlighting that photographs have to be examined through multimodal methods and in relation to wider discursive processes of meaning making. Third, by borrowing ideas from the literature on collective memory, the article shows that, even though they depict scenes that are set in the distant past, historical photographs can still influence environmental attitudes and behaviours in the present.

From researchers to academic entrepreneurs: a diachronic analysis of the visual representation of academics in university annual reports

Abstract: Influenced by the global neoliberalization of higher education, academic entrepreneurialism has become a new paradigm of university development and has brought about profound changes in various types of university discourse. Against this backdrop, this study investigates the transformations in the visual depiction of academics in the annual reports of six major universities in Hong Kong during the past two decades. Drawing on critical visual analysis, the study shows that the communicative purposes of the images have shifted from reporting the research process to promoting research outcomes. The visual identities of academics have shown clear transformations of becoming increasingly individualized, entrepreneurial and self-promotional. With a higher degree of social interaction and closer social distance with viewers, they are playing an increasingly important role in building public relations. The study enriches the social analysis of neoliberalization as a process through the quantitative and diachronic lens. It demonstrates how a visual analytical method applied to the critical analysis of identity construction and university discourse can provide an explicit understanding of the visual manifestations of neoliberalism in higher education and its diachronic change.

Rhythm in literary apps

Abstract: This article addresses how rhythm may function in literary apps. The article has two aims: increasing the knowledge of how literary apps work as texts, by exploring their aspects of rhythm, and developing the understanding of the theoretical term of rhythm. The authors propose a rhythmanalysis in which two different types of rhythm – reading rhythm and narrative rhythm – are taken into account. The two types of rhythm may both occur at different structural levels in the text. This approach is applied to the analysis of rhythm in the popular literary app, Florence (Wong et al., 2018, Florence Tablet application software), drawing on concepts from multimodal social semiotics (Van Leeuwen, Introducing Social Semiotics, 2005), although leaning towards a more reception-oriented approach than the traditional text-oriented analysis in social semiotics. Literary apps are defined in this context as multimodal fictional narratives that can lead to an aesthetic experience for the reader (Iser, 1984, Der Akt des Lesens); however, non-narrative apps, such as poetry, may also be defined as literary apps. These apps may be read on a tablet or a smartphone. This article elucidates some of the many facets of rhythm related to the multimodal design of a literary app, which invites different forms of interactivity than the linear reading and page-turning of print-based picture books. The findings of the analysis show how rhythm not only contributes to the multimodal cohesional aspects of literary apps, but is fundamental to the meaning potential of the literary app.

Drawn into the future: The epistemic work of visual scenarios in the configuration of human–robot encounters

Abstract: This article looks at artistic impressions of future robotics and considers how they inspire research into human–machine interaction. Our analysis of visual scientific practices and the epistemic ramifications of these speculative drawings emerges from a long-term participant observation study in a multi-disciplinary project on smart and autonomous technologies in public spaces. We discuss the design, appropriation and modulation of visual scenarios, and scrutinize how these diegetic futurescapes are imaginatively engaging and suggestive of scientific progress and experimentation. The article argues that the future-oriented scenes defy common notions of post hoc scientific representations. Instead, they are ex ante presentations of the ambition to imagine human–machine relations in the future and to draw the large-scale research venture together. The register of evaluation thereby shifts from aesthetic criteria to scientific parameters. More than just visual tokens, the scenarios became a catalyst for collaboration.

Opening up semiotic spaces for gender expression: a case study of the construction of gender in Australian award-winning early childhood picture books

Abstract: This article examines the visual and verbal expressions of gender in Australian award-winning early childhood picture books. It brings together social semiotic analysis and the narratological concepts of narration and focalization to examine the extent to which one community of practice (authors, illustrators, publishers and awards council) reproduces symbolic manifestations of gender, or offers readers space to engage with alternatives. The authors’ findings suggest that, while the literary works produced by this community of practice mostly serve to reinforce hegemonic cultural attitudes of what constitutes desirable femininity and masculinity in Australia, there is ample opportunity for change.

Visual communication and mental health

Abstract: This article conceptualizes the field of ‘visual communication and mental health’, prioritizing the question of how visual imagery is experienced. Taking as its starting point the challenge of overcoming stigma and the limitations of visual clichés of depression and mental illness, the author argues for a dynamic, relational model of communications, foregrounding lived experience. To illuminate the psychosocial impacts and potential benefits of creative engagement with visual media, she draws on understandings of symbolic communication, derived in particular from the work of DW Winnicott and the British Independent tradition of psychoanalysis. The imagery discussed includes stock and campaign imagery, conceptual/expressive artwork and a virtual reality (VR) production that extends an innovative approach to mental health literacy by First Nations artists.

Typicality effect in data graphs

Abstract: Some types of data graphs are more easily understood than others. Following the suggestion that typically encountered graphs may activate individuals’ cognitive schema quickly, this study investigated prior exposure to and typicality of three common graph types: vertical bar graphs, horizontal bar graphs and line graphs; and three common data patterns: rising, neutral and falling. Results from two samples ( N = 57 and N = 30) suggest that vertical bar graphs are encountered more frequently, are rated as more typical and are identified more quickly than horizontal bar graphs and line graphs; also that rising data patterns are more typical than falling and neutral data patterns. The findings contribute new knowledge about the hierarchical structure of graph schema and can inform design choices about which graph types might best facilitate viewers’ understanding of data visualizations.

Learning through mess: Sensemaking visual communication practices in a UK multidisciplinary applied health study

Abstract: This article addresses the challenges and opportunities associated with the development of new visual communication practices and outputs, using an example of such work conducted in a UK interdisciplinary applied health project. Reflecting on his role as co-researcher and practice as a visual ethnographer in the study, the author argues that new visual communication practices may emerge from ‘mess’ and even ugliness. In the case discussed, the author comes to terms with mess and elements of failure as potential phenomena of learning through a process of Sensemaking (see Weick’s Sensemaking in Organizations, 1995), by applying innovative visual methods to the approach. Through his version of visual Sensemaking, the author identifies a set of principles to inform innovation in collaborative, interdisciplinary visual communication.

Sculpting the interpersonal: towards a social semiotic framework for analysing interpersonal meaning in statues

Abstract: The underlying question of this article is ‘how do statues convey interpersonal meaning?’ To answer this question, the authors briefly critically examine the current social semiotic analytical framework for statues and develop a revised framework for analysing interpersonal meaning in which features from Kress and van Leeuwen’s Reading Images: The Grammar of Visual Design (2006) and selected features from O’Toole’s framework from The Language of Displayed Art (2011) for analysing sculpture are integrated. These features are also extended and/or complemented by incorporating features obtained from research into the fields of gesture, body language and facial expression. Further, in keeping with Systemic Functional Linguistic-inspired research, system networks are used to map out the potential material and semantic (interpersonal) features for figurative statues and to present possible configurations among these features. Although the focus of this article is on interpersonal meaning, it is acknowledged that within a social semiotic approach, there is an interdependency among interpersonal, experiential (representational) and textual (compositional) meanings, and that these configure within a specific context of situation. This interdependency is only briefly attended to in the article itself, but the proposed framework provides a starting point for developing an account of the way that interpersonal features and their realizations in statues may configure with representational and compositional features and their realizations. With the current world focus on statues and their sometimes controversial social meanings, this article offers a timely opportunity for a range of users such as social semioticians or art educators and students to consider, through a systematic analytical framework, the way in which statues may relate interpersonally with viewers, and provides a key step towards accounting for the way that configurations of interpersonal, representational and compositional features may construe contextual tensions in relation to the overall message conveyed by a statue.

Michel Pastoureau and the history of visual communication

Abstract: Despite early and ongoing calls for a systematic engagement with history, social semiotics has largely emphasized research on the synchronic rather than diachronic dimensions of meaning-making. And while the ‘instability’ of semiotic practices (see Kress’s Multimodality: A Social Semiotic Approach to Contemporary Communication, 2010) and the importance of semiotic change (see Van Leeuwen’s Introducing Social Semiotics, 2005) have become key themes in semiotics, there is still a need for a dynamic approach to the study of visual and multimodal communication, focusing not only on describing how meaning-making resources and their uses are changing, but also on why they are changing. In this article, the authors focus on the importance of the work of medieval historian Michel Pastoureau for the social semiotic study of visual communication, highlighting that this work can help us further refine and even rethink key social semiotic concepts such as modes and media, provenance, and context. Pastoureau’s work shows how we can make theoretical statements about instability, change and innovation more concrete and, ultimately, empirically based. His approach can also help us understand semiotic change and its relation to social and cultural (and also economic and technological) change more broadly, often with the aid of the (crucial) normative discourses that shape semiotic practices over time.

Computing colorism: skin tone in online retail imagery

Abstract: A long legacy of media imagery persistently distorts, stereotypes, and ignores marginalized racial and ethnic groups despite widespread calls to diversify media representations. In particular, fashion and beauty media continue to feature light-skinned models and celebrities over dark-skinned individuals, even lightening dark skin with photo editing to achieve ideals of whiteness and lightness. This practice aligns with colorism, or the privileging of light skin tones for access to economic and social capital. This study examines colorism in a particular genre of digital photography, online retail images, as a problem of visual representation. The novel method of visual computational analysis is used to quantitatively compare how mainstream clothing retail brands represent model skin tones across still and video media modes. The findings suggest that analyzed retailers tended to favor light-skinned models on their websites and that model skin tones in product videos were significantly darker than in product photos. These findings are considered through research on race and technology, photographic manipulation, and media misinformation. Ultimately, the study suggests that visual (in)consistencies can reveal the role of structural biases in shaping media representations. The article also provides a methodological tool for conducting this work.

Editorial: Information Is Ugly

Abstract: Design frameworks that outline the benefits of thinking in terms of binaries suggest that, as designers, we can situate ourselves and our work in relation to opposite extremes. Doing so is more likely to bring about innovation and imagine ideological possibilities. This Special Issue creates a binary between ugly and beautiful with a specific focus on the former. The standard dictionary definitions of ugly are in relation to an unpleasant or repulsive appearance or a topic that is likely to involve violence or other unpleasantries. We draw both definitions into our discussion. We have collected articles that have at their heart ugly topics, including the climate crisis, racism and digital surveillance, which reflect contemporary times. As we write, a war has begun in Ukraine. The news shifts the global pandemic to second position and shunts Black Lives Matter, the rising cost of living in the UK, widening social inequality and the climate crisis out of the news headline. Each news package is filled to the brim with infinite amounts of ugly information. We push for a rebellion against the convention of beautifying data, such as became mainstream following the popularity of Information Is Beautiful (McCandless, 2009) and other works emphasizing the sleek graphic design of data visualization. Instead, we seek to engage people in exploring how to research, analyse and present ugly information in other ways. 1093555 VCJ Visual CommunicationFass et al.: Editorial

The Riverine Archive: Nausea and information loss on the neoliberal ship of fools

Abstract: Within academia, as in other corporatized environments, there are irreconcilable tensions embedded in the managerial data imaginary, in contrast to the messy reality of lived experience and increasingly precarious working conditions for those at the coalface of Higher Education. Combined with student debt and escalating surveillance via so-called ‘artificially intelligent’ transactional data analysis, fantasies of control and domination converge on platitudes about Big Data and computational information, often presented as unambiguously neutral via idealized visualizations and ‘dashboards’ such as those commonly provided by Tableau and Google. As a structure, the digital archive is no different, open to fantasies of secure representation but, in fact, always unstable, subject to the materiality and flux of electronic, symbolic and social processes. As academics exposed to intensified metricization within increasingly data driven institutions, how can we counter the reduction of our qualitative research and experience to dashboards and scores? Drawing upon the work of Fleming’s Dark Academia: How Universities Die (2021), Poster’s The Mode of Information (1990), Cascella’s En Abime: Listening, Reading, Writing: An Archival Fiction (2012) and Bayne et al.’s The Manifesto for Teaching Online (2020) among others, and focusing on work developed by the authors from 2008 onwards, they discuss their contingent, often nauseating virtual reality information repository, the Riverine Archive, developed to hold and withhold information about their writing and art within the ugly hulk of what they define as a ‘neoliberal ship of fools’. The work seeks to offer a counter to prevailing platitudes and fantasies about the neutrality and realism of data to directly represent a stable, singular reality. The authors concur that information is not beautiful, but rather a fallacy of stability, servicing a rapacious, anti-academic neoliberal ideology that needs to be brought to the surface and subjected to honest discourse, away from hype and managerial wishful thinking. Truth and beauty cannot be framed as stable, monolithic or universal; to do so is to replicate a colonial projection of knowledge, a mono-logic, centring all truth upon the Global North.

Generational differences in viewing behaviors: an eye-tracking study

Abstract: Many scholars over the past two decades have contended that constant exposure to visually-oriented technologies makes younger individuals inherently more visually skilled than previous generations. The study presented here investigates this claim by using eye tracking to examine patterns in the ways in which individuals interact with visual stimuli, specifically journalistic photographs. Study participants included 29 college students aged 18–22 (mean = 19), and 20 non-student members of the university and surrounding community, aged 40–63 (mean = 50). Eye movements were recorded using a TobiiPro x2-60 eye tracker connected to a 17-inch gaming laptop. If younger individuals and older individuals have different levels of visual ability, there should be observable differences between the eye movements of the two groups. However, the differences observed between the two groups of participants were very limited and did not point to any consistent patterns that would suggest differing levels of skill at reading images.

Imaging cancer: image-based diagnostic communication in radiologists’ embodied cognition

Abstract: This article thematizes the specific process of cancer detection in radiology, which presupposes a delicate synthesis of the specifics of oncoradiology images and the skilful actions performed by the radiologist. The enactment of cancer via meaningful action rather than recognizing static depiction puts the structures of image consciousness into the wider context along with memory, free imagination and amodal completion, among others. Hence, by way of reinterpreting phenomenological projects via enactivism and incorporating them into the radiologist’s work (cases, radiograms), medical diagnostics in general and oncoradiology in particular presuppose a multimodal categorial structuring (of meaning) that goes far beyond direct sensory givens. In most branches of radiology, we cannot tell what the cancer is without attending to the multitude of its appearance and the perceptual and imaginative strategies of those who make it appear. As such, this article also considers the wider problem of how knowledge is related to the (embodied) subjectivity in a particular social setting.

Alien domesticity: representing home during a pandemic

Abstract: Cultural geography and the related disciplines of urban sociology and anthropology have long focused their theoretical lenses on the city as a space of lived multiplicity. This photographic essay focuses its lens on the home as such a space. During the pandemic lockdown of 2020–2021, many of us spent more time in our homes than we ever had before, working, teaching, schooling, shopping, and barricading ourselves from the outside world. This essay borrows from the often ambiguous and anonymizing aesthetics of street photography to depict the multiple, overlapping worlds of home during a pandemic. Home, as depicted here, is an always unfinished process of affective assemblage and dissolution. The images featured seek to capture that lack of resolution, the messy emotional texture of home life under lockdown.