Book•
The Routledge handbook of multimodal analysis
01 Jan 2009-
TL;DR: Theoretical and Methodological Tools for Multimodal Analysis as mentioned in this paper is a toolkit for multimodal analysis with a focus on the analysis of the transmodal moment.
Abstract: Introduction: Handbook Rationale, Scope and Structure Part 1 Theoretical And Methodological Tools For Multimodal Analysis 1.An Introduction to multimodalit 2. Different approaches to multimodality 3.What are multimodal data and transcription? 4.What is mode? 5.Parametric systems: the case of voice quality Theo van Leeuwen 6. Modal density and modal configurations: multimodal actions 7. Transformation, transduction and the transmodal moment Part 1 readings Par 2 Key themes for multimodality 8. Historical Changes in the Semiotic Landscape From Calculation to Computation 9. Technology and Sites of Display 10. Multimodality and Mobile Culture 11. Multimodality, Identity, and Time 12. Multimodality and reading: the construction of meaning through image-text interaction 13. Power, social justice and multimodal pedagogies Part 3 Multimodality across different theoretical perspectives 14. Multimodality and language: A retrospective and prospective view 15. Multimodality and theories of the visual 16. Multimodality and New Literacy Studies 17. Using Multimodal Corpora for Empirical Research 18. Critical Discourse Analysis and multimodality 19. Semiotic paradigms and multimodality 20. Reception of multimodality: Applying eye-tracking methodology in multimodal research 21. Representations in practices: A socio-cultural approach to multimodality in reasoning 22. Indefinite precision: artefacts and interaction in design 23. Anthropology and Multimodality: The Conjugation of the Senses Part 4 Multimoda Case Studies 24. Practical function and meaning: a case study of Ikea tables 2 The use of gesture in operations 26. Gesture and Movement in Tourist Spaces 2 The kineikonic mode: towards a multimodal aproach to moving image media 28. Multimodal Analytics: Software and Visualization Techniques for Analyzing and Interpreting Multimodal Data 29. Colour: code, mode, modality -- the case of.
Citations
More filters
01 Jan 2014
TL;DR: In this paper, the authors present two classroom portraits of a 5th and 9th grade classrooms as activity systems where teachers and learners are engaged in multimodal composing, and they suggest that multimodality must be carefully understood as part of an activity system.
Abstract: In this chapter, the authors present two classroom portraits of a 5th and 9th grade classrooms as activity systems where teachers and learners are engaged in multimodal composing. In their analysis, they are most interested in how principles of design, affordances of modes, and multimodality become internalized as psychological tools that shape learning in the context of activity. The authors ask two research questions: What are the mediational artifacts (both ideal and material) in these activity settings? What does this reveal about multimodality as a socially situated process? Conclusions drawn from the two different cases lead the authors to suggest that multimodality must be carefully understood as part of an activity system.
9 citations
TL;DR: In this article, a shift from verbal to visual modes of expression that take shape in emerging genres and media formats is discussed, while the importance of vis-a-vation is emphasized.
Abstract: Today’s communicative practices are increasingly characterized by a shift from verbal to visual modes of expression that take shape in emerging genres and media formats. While the importance of vis...
9 citations
Cites background from "The Routledge handbook of multimoda..."
...In the same vein, Kress (2009) highlights the jointly-constructed understanding of the perceived action-opportunities afforded by modes....
[...]
...…on: The different meaning-making potentials of different modes; e.g. language depicting events in time, images representing objects in space (Kress, 2009; St€ockl, 2004; St€ockl et al., 2019); The intersemiotic relations that hold between different modes; e.g. language and image extending…...
[...]
TL;DR: This paper used an extract from an interview concerning gravitational wave physics to show that the meaning of hesitancies within speech are different when spoken and when read from the corresponding transcript, and they proposed a new subject called language code analysis which would systematically examine how meanings change when the'same' words are communicated via different media and symbol systems.
Abstract: We use an extract from an interview concerning gravitational wave physics to show that the meaning of hesitancies within speech are different when spoken and when read from the corresponding transcript. When used in speech, hesitancies can indicate a pause for thought, when read in a transcript they indicate uncertainty. In a series of experiments the perceived uncertainty of the transcript was shown to be higher than the perceived uncertainty of the spoken version with almost no overlap for any respondent. We propose that finding and the method could be the beginning of a new subject we call 'Language Code Analysis' which would systematically examine how meanings change when the 'same' words are communicated via different media and symbol systems.
9 citations
Dissertation•
01 Apr 2015
TL;DR: In this paper, a touch-sensitive, sound-generating rug and its application in an experimental context are examined through the development of a touch sensitive, sound generating rug, which allows an insight into how they make sense of multisensory information via observation of how they physically respond to it.
Abstract: This research takes a practice-based approach to exploring perceptual matters that often go unnoticed in the context of everyday lived experience. My approach focuses on the experiential possibilities of knowledge emerging through artistic enquiry, and uses a variety of modes (like textiles, sound, physical computing, programming, video and text) to be conducted and communicated. It examines scholarship in line with the ecological theory of perception, and is particularly informed by neurobiological research on sensory integration as well as by cultural theories that examine the role of sensory appreciation in perception.
Different processes contributing to our perceptual experience are examined through the development of a touch-sensitive, sound-generating rug and its application in an experimental context. Participants’ interaction with the rug and its sonic output allows an insight into how they make sense of multisensory information via observation of how they physically respond to it. In creating possibilities for observing the two ends of the perceptual process (sensory input and behavioural output), the rug provides a platform for the study of what is intangible to the observer (perceptual activity) through what can actually be observed (physical activity).
My analysis focuses on video recordings of the experimental process and data reports obtained from the software used for the sound generating performance of the rug. Its findings suggest that attentional focus, active exploration, and past experience actively affect the ability to integrate multisensory information and are crucial parameters for the formation of a meaningful percept upon which to act. Although relational to the set experimental conditions and the specificities of the experimental group, these findings are in resonance with current cross-disciplinary discourse on perception, and indicate that art research can be incorporated into the wider arena of neurophysiological and behavioural research to expand its span of resources and methods.
9 citations
Cites background from "The Routledge handbook of multimoda..."
...…the breaking down of boundaries between local and global; the new possibilities offered by technological advances in our everyday life; and the impact of interactivity and virtuality across many domains, particularly within the fields of education, communication, and design (see 14 Jewitt 2009)....
[...]
TL;DR: In this paper, the authors explore what is understood by hybrid communities and how their practices might be researched using hybrid methods to investigate the experiences of participants in them, and consider what might be an ethical framework for researching activity in these entities, giving examples from several projects that have tried carefully to embed this framework in their practices.
Abstract: Hybrid communities using online and face-to-face communications to construct their practices are increasingly part of everyday life amongst people who have easy access to the internet. Researching these communities raises a number of challenges for researchers in the pursuit of ethical research. The paper begins by exploring what is understood by hybrid communities and how their practices might be researched using hybrid methods to investigate the experiences of participants in them. The discussion then considers what might be an ethical framework for researching activity in these entities, giving examples from several projects that have tried carefully to embed this framework in their practices. In exploring these studies, the paper highlights the ethical possibilities and challenges that online and offline spaces offer for researchers in the conduct of their qualitative educational research.
9 citations
Cites background from "The Routledge handbook of multimoda..."
...Collecting multimodal material signals the central part played by visual, gestural, and kinaesthetic modes in interpersonal communications, alongside language, in the development and expression of meaning (Jewitt, 2009; Kress, 2010)....
[...]