Book•
The Routledge handbook of multimodal analysis
01 Jan 2009-
TL;DR: Theoretical and Methodological Tools for Multimodal Analysis as mentioned in this paper is a toolkit for multimodal analysis with a focus on the analysis of the transmodal moment.
Abstract: Introduction: Handbook Rationale, Scope and Structure Part 1 Theoretical And Methodological Tools For Multimodal Analysis 1.An Introduction to multimodalit 2. Different approaches to multimodality 3.What are multimodal data and transcription? 4.What is mode? 5.Parametric systems: the case of voice quality Theo van Leeuwen 6. Modal density and modal configurations: multimodal actions 7. Transformation, transduction and the transmodal moment Part 1 readings Par 2 Key themes for multimodality 8. Historical Changes in the Semiotic Landscape From Calculation to Computation 9. Technology and Sites of Display 10. Multimodality and Mobile Culture 11. Multimodality, Identity, and Time 12. Multimodality and reading: the construction of meaning through image-text interaction 13. Power, social justice and multimodal pedagogies Part 3 Multimodality across different theoretical perspectives 14. Multimodality and language: A retrospective and prospective view 15. Multimodality and theories of the visual 16. Multimodality and New Literacy Studies 17. Using Multimodal Corpora for Empirical Research 18. Critical Discourse Analysis and multimodality 19. Semiotic paradigms and multimodality 20. Reception of multimodality: Applying eye-tracking methodology in multimodal research 21. Representations in practices: A socio-cultural approach to multimodality in reasoning 22. Indefinite precision: artefacts and interaction in design 23. Anthropology and Multimodality: The Conjugation of the Senses Part 4 Multimoda Case Studies 24. Practical function and meaning: a case study of Ikea tables 2 The use of gesture in operations 26. Gesture and Movement in Tourist Spaces 2 The kineikonic mode: towards a multimodal aproach to moving image media 28. Multimodal Analytics: Software and Visualization Techniques for Analyzing and Interpreting Multimodal Data 29. Colour: code, mode, modality -- the case of.
Citations
More filters
01 Jan 2014
TL;DR: This chapter explains how visual and multimodal metaphors in advertising work, and discusses some cases to show how metaphor analysis can be a critical tool in the evaluation of advertising.
Abstract: Metaphors present one kind of thing (a "target") in terms of another (a "source"), and are therefore ideal instruments for advertisers to make claims about products (the metaphors’ targets) efficiently and creatively. Since the intended interpretation of metaphors is often not spelled out, advertisers often get away with suggesting meanings without taking responsibility for them by making skillful use of visuals as part of metaphors. This chapter explains how visual and multimodal metaphors in advertising work, and discusses some cases to show how metaphor analysis can be a critical tool in the evaluation of advertising.
16 citations
01 Jan 2013
TL;DR: This article explored how HIV and AIDS discourse is differently constituted and evaluated across different modes by different communities of speakers in Malawi and explored how different languages and other social semiotics are used as resources across the different modes.
Abstract: This study explores how HIV and AIDS discourse is differently constituted and evaluated across different modes by different communities of speakers in Malawi. Particularly, the study explores how different languages and other social semiotics are used as resources across the different modes. Among other things, it further investigates the implications of the unequal social distribution of modes of communication and semiotic resources in Malawi (eastern region in particular) for the fight against HIV and AIDS. The study employed the Multimodal Discourse Analysis (MDA) approach, the notions of Resemiotisation (Kress and van Leeuwen 2006, Norris 2004, Martin and Rose 2004) and semiotic remediation (Prior and Hengst 2010) in analysing the data. This approach is necessary as the study focuses on HIV and AIDS communication which is essentially multimodal in nature.
The study used both quantitative and qualitative methodologies involving questionnaires, interviews, focus group discussions, document analysis and observation of television programs, traditional dances and other modes such as music videos. The study found that different practices have been semiotically remediated and reformulated for health palatability. As a result, taboos have been de-tabooed and technical terms have been ‗untechnicalised‘ so that even ordinary people are able to use health technical terms. The study also shows how cultural practices (such as having ‗live‘ sexual contact with the widow) have been semiotically remediated with the usage of condoms or herbs for cleansing
rituals. The study further finds that literacy is not a major challenge for the consumption of HIV and AIDS messages. However the study also shows that wrongly presented messages such as textual overcrowding, usage of proverbs and depiction of western culture in HIV and AIDS messages obscure consumption. In addition the study reveals that proverbs can hardly iv be understood by all consumers and in turn led to division between mostly the older generation and rural who understand and the younger and urban people who have difficulty
comprehending the proverbs. Lastly the study finds that some modes of communication did not prove effective, for instance, SMS, television and radio as these do not benefit all consumers as they are socioeconomically determined.
16 citations
TL;DR: Analysis of video recordings from an ethnographic study of primary school children working with problem‐centred assignments reveals how the activity of seeking pictures is shaped by the assumption that pictures are different from facts and information.
Abstract: Purpose – The aim of this study is to further understanding of the situated activity of seeking pictures. It relates to an ongoing discussion on how multimodal information literacies are enacted in different social practices.Design/methodology/approach – In order to understand the characteristics of the communication and interactions in the activity of seeking pictures, video recordings from an ethnographic study of primary school children working with problem‐centred assignments have been analysed.Findings – The analysis reveals how the activity of seeking pictures is shaped by the assumption that pictures are different from facts and information; pictures are seen primarily as having decorative functions. The activity is also characterised by playful, yet efficient cooperation between the children; they make the activity meaningful by transforming it into a play and game activity where pictures become important as physical objects, but not as a semiotic means of learning.Research limitations/implication...
16 citations
TL;DR: The authors conducted a survey of faculty at a research-intensive public university and found that faculty across disciplines use multiple modes of communication in their professional work, their scholarly communication, and their pedagogy.
Abstract: While a strong case has been made for addressing multimodality in composition, the case has been less clear for WAC/WID and CxC programs and research. Studies of disciplinary communication have documented the use of multiple modes in a number of fields, but few engage directly with theories of multimodality or with multimodality in context of changes related to networked, digital media. This study presents a snapshot of multimodal communication practices and assignments across disciplines developed through a survey of faculty at a research-intensive public university. Quantitative results indicate that, with some disciplinary variation, faculty across disciplines use multiple modes of communication in their professional work, their scholarly communication, and their pedagogy. Qualitative analysis of faculty responses complicates this picture with diverse conceptualizations of the relationships between modes. Themes related to faculty experiences of genre change and to the challenges of communicating about multimodality across disciplines are also addressed. These results justify the need for professional development efforts focused on multimodality in the context of WAC/WID and CxC programs and for continued research on multimodality in university contexts, even as they point to the challenges of communicating across disciplines that lack shared vocabulary. [1] The conversations seemed to be going well. A couple of groups had already started sketching rubrics on their whiteboards and were earnestly discussing what criteria to include. This faculty workshop on evaluating multimodal assignments had been prompted by questions raised about assessment at an earlier workshop on multimodal assignments. Vanessa was pleased to see the depth of engagement between lecturers and graduate teaching assistants. Both had much to offer each other on this topic. A lecturer in a group that had yet to start drawing or writing raised his hand. Vanessa squeezed past another group to join them. \"How's it going?\" Vanessa tried to sound upbeat, but she had misgivings. At the earlier workshop, a substantial amount of time had been spent addressing concerns about the appropriateness of multimodal assignments for a WID-based first-year composition curriculum. Vanessa hoped those
16 citations
Additional excerpts
...…last few years, the MODE research center at the University College London has regularly published work on communication in surgical operating theaters, yielding insights into the use of touch and gesture for communication and learning (e.g. Bezemer, 2014; Bezemer, Cope, Kress, & Kneebone, 2014)....
[...]
TL;DR: For instance, the authors found that while polarized news outlets made their positionality on the candidates obvious, more neutral or central news outlets revealed their preferences through subtle multimodal design choices.
Abstract: In online and video/television spaces, news media discourses incorporate multimodal design as a discursive move capable of steering meaning toward desirable implications. Around the 2016 U.S. presidential elections, while polarized news outlets made their positionality on the candidates obvious, more neutral or central news outlets revealed their preferences through subtle multimodal design choices. One of these design choices is using a quantitative visual rhetoric: persuasive multimodal moves that draw on quantification through visual, spatial, and textual manipulation—involving the choice of data representation, visual images, and illustrations, (im)balance between numeric and alphabetic texts, and general quantitative narrative. This quantitative visual rhetoric helps news outlets manipulate facts without lying with words, leaving the onus of misinterpretation on the readers/viewers. In this article, using examples from five types of media outlets—far-left, left-leaning, central, right-leaning, and far-right— we share examples of design choices we found through multimodal analysis of their quantitative visual rhetoric during the 2016 elections. We share implications for media literacy education and civic engagement.
16 citations
Cites background from "The Routledge handbook of multimoda..."
...We finalized two articles per outlet, adding up to a total of ten articles that we further analyzed taking a multimodal approach to tease affordances of inherent design choices (Gibson, 2014; Kress, 2004; Jewitt, 2014)....
[...]
...Generating a large dataset for such niche topics is not only difficult, but also not appropriate given the importance of teasing apart the modal affordances of a smaller set of examples (Jewitt, 2014)....
[...]
...Therefore, when questioning the veracity of information, one needs to identify and tease apart the rhetoric of the journalistic media, which begins with the dissection of the design choices made with multimodal texts and discursive moves (Jewitt, 2014)....
[...]
...In multimodal analysis, Kress (2003, 2004) and Jewitt (2014) have shown that identifying modal affordances of key examples are more important than the size of the sample....
[...]