Book•
The Routledge handbook of multimodal analysis
01 Jan 2009-
TL;DR: Theoretical and Methodological Tools for Multimodal Analysis as mentioned in this paper is a toolkit for multimodal analysis with a focus on the analysis of the transmodal moment.
Abstract: Introduction: Handbook Rationale, Scope and Structure Part 1 Theoretical And Methodological Tools For Multimodal Analysis 1.An Introduction to multimodalit 2. Different approaches to multimodality 3.What are multimodal data and transcription? 4.What is mode? 5.Parametric systems: the case of voice quality Theo van Leeuwen 6. Modal density and modal configurations: multimodal actions 7. Transformation, transduction and the transmodal moment Part 1 readings Par 2 Key themes for multimodality 8. Historical Changes in the Semiotic Landscape From Calculation to Computation 9. Technology and Sites of Display 10. Multimodality and Mobile Culture 11. Multimodality, Identity, and Time 12. Multimodality and reading: the construction of meaning through image-text interaction 13. Power, social justice and multimodal pedagogies Part 3 Multimodality across different theoretical perspectives 14. Multimodality and language: A retrospective and prospective view 15. Multimodality and theories of the visual 16. Multimodality and New Literacy Studies 17. Using Multimodal Corpora for Empirical Research 18. Critical Discourse Analysis and multimodality 19. Semiotic paradigms and multimodality 20. Reception of multimodality: Applying eye-tracking methodology in multimodal research 21. Representations in practices: A socio-cultural approach to multimodality in reasoning 22. Indefinite precision: artefacts and interaction in design 23. Anthropology and Multimodality: The Conjugation of the Senses Part 4 Multimoda Case Studies 24. Practical function and meaning: a case study of Ikea tables 2 The use of gesture in operations 26. Gesture and Movement in Tourist Spaces 2 The kineikonic mode: towards a multimodal aproach to moving image media 28. Multimodal Analytics: Software and Visualization Techniques for Analyzing and Interpreting Multimodal Data 29. Colour: code, mode, modality -- the case of.
Citations
More filters
01 Jan 2012
TL;DR: The litteratie mediatique, complementaire de la litteraties classique, peine toutefois a investir l'ecole as discussed by the authors, peine etre en mesure de proposer des activites didactiques integrant, par exemple, les blogues ou les reseaux sociaux, les bandes dessinees, les productions video, les romans-photos, les jeux video (serious games).
Abstract: L'univers de l'ecrit s'est radicalement metamorphose avec l'avenement du numerique, les modes semantiques (texte, image, son) et les supports technologiques etant toujours plus diversifies. La litteratie mediatique, complementaire de la litteratie classique, peine toutefois a investir l'ecole. Elle fraie sa voie difficilement mais patiemment entre le cours de francais, l'initiation a la recherche documentaire et le cours d'histoire ou d'education civique. Afin de soutenir les eleves dans l'apprentissage de la lecture et de l'ecriture multimodale, les enseignants devraient etre en mesure de proposer des activites didactiques integrant, par exemple, les blogues ou les reseaux sociaux, les bandes dessinees, les productions video, les romans-photos, les jeux video (serious games). Cependant, il manque d'assises theoriques et empiriques sur lesquelles fonder ces changements sur le plan pedagogique.
37 citations
Cites background from "The Routledge handbook of multimoda..."
...…ce que nous pourrions croire ; la nouveauté à son égard est qu’il propose aujourd’hui une conception fortement remaniée de la nature même des messages et de leur diffusion (Jewitt, 2009b ; Kress, 2009, 2010) grâce à des médias de communication en constante évolution (Bearne et Wolstencroft, 2007)....
[...]
...Socialement construit et culturellement transmis, le mode est en fait un moyen de « faire du sens » (Kress, 2009)....
[...]
...Kress (2009) soutient d’ailleurs que la mobilisation d’un mode plutôt que d’un autre découle, dans notre civilisation actuelle, de la domination historique et culturelle de la vue et de l’ouïe sur les autres sens, par exemple l’odorat ou le toucher....
[...]
TL;DR: A review of literature examines relevant research that supports new ways of viewing children as active transmitters of culture in situated learning contexts, where case studies explore children's redesign of semiotic modes of music and verbal linguistics as mentioned in this paper.
Abstract: This review of literature examines relevant research that supports new ways of viewing children as active transmitters of culture in situated learning contexts, where case studies explore children’s redesign of semiotic modes of music and verbal linguistics. Some recent research discussed in this article supports the premise that cognitive abilities of children in early learning settings may be transformed through embodied ways of representing prior knowledge. Young children have been observed enriching prior knowledge during interactions in music invention, using the gestural mode to interpret rhythmic and melodic motifs, structure and phrasing through movement to music, or extending these elements of music (audio mode) in invented song or instrumental play. In engaging literacy tasks, they co-construct texts by drawing on semiotic resources of visual symbols and spatial design elements in written linguistic modes. This cognitive structuring is also revealed in the underlying patterns found in their embodied music invention. How knowledge is represented is crucial to children’s apprehension of knowledge through co-construction. It enables their selection of media and mode for redesign, to promote their understanding of concepts and facilitate problem solving. Multimodal redesign in young children’s music and verbal linguistics is explored as a rich source for communicating meaning and developing higher thinking.
37 citations
Cites background from "The Routledge handbook of multimoda..."
...Children negotiate their own identities and pathways by remaking texts and representations (Jewitt, 2009)....
[...]
...“Representational meaning (experiential meaning) is meaning concerned with the construal of material or mental experience—the processes, participants and circumstances involved” (Jewitt, 2009, p. 303)....
[...]
...Representations, or modes, are the outcome of a cultural shaping of a material (Jewitt, 2009)....
[...]
...Literacy as a social and cultural activity empowers children to negotiate their own pathways by remaking texts (Jewitt, 2009)....
[...]
TL;DR: In this article, the authors adopt a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions and examine how semiotic resources are deployed in each medium through elements of mise-en-scene, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences.
Abstract: This paper adopts a multimodal social semiotic approach for exploring the semiotic changes involved in the transformation of a novel into stage and screen productions. It examines how semiotic resources are deployed in each medium through elements of mise-en-scene, such as speech, music, sound, lighting, props, staging, and cinematographic techniques, and the viewing perspectives that are thus established for audiences. The genre of Gothic horror is selected for this purpose, given how this form of performance has transfixed audiences for centuries and has been adapted for both the stage and the screen. In order to demonstrate how each performance medium has produced its own unique set of foregrounding devices to enthral and captivate audiences, a comparative analysis of excerpts from the novel The Woman in Black by Susan Hill, a videotaped theatrical performance, and the 1989 British television film of the same name is undertaken. The paper discusses the implications of the multimodal semiotic ap...
37 citations
23 Apr 2020
TL;DR: The authors used a Free Associative Narrative Interview (FNIHI) with eight experienced psychoanalytic psychotherapists to investigate the meaning of autism spectrum disorders in clinical and social terrains, revealing four main interpretive repertoires that organized the rhetorical agenda's of participants.
Abstract: This research investigated the construction of autism in clinical and social terrains. Study one drew from Critical Discursive Psychology (CDP) to examine the language of psychoanalytic psychotherapists in constructing the phenomenon of autism spectrum disorders. This study relied on interview data with eight experienced psychoanalytic psychotherapists using a Free Associative Narrative Interview design. The investigation of the therapists’ discourses revealed four main interpretive repertoires that organised the rhetorical agenda’s of participants. The analytic notions of interpretive repertoires, ideological dilemmas and subject positions demonstrated how neo-liberal political frameworks influenced the therapists' negotiation of the meaning of autism. The implications of this discursive framework were subjected to a critical analysis revealing the limitations that they impose on the possible ways of being for autistic people. The second study used multimodal analysis to investigate an activist’s momentary identities on a “viral” YouTube video entitled: “In My Language” (see appendix 4). It focused on the verbal and non-verbal elements of the video material. The analytic attention predominantly settled on the interplay between the various semiotic resources that the activist utilised to negotiate a multiplicity of meanings. A wide range of identities produced by the participant’s social actions, exploring a political manifesto against the social oppression exerted on people with autism. The findings suggested that meaning-making inside this video was intricately related to the pathological language that saturates autistic lives from their beginning. This study also considered how multimodal designs of research could add to the investigations of disability and autism studies, pointing to the need to employ more autism lead research in the clinical and non-clinical sites. The findings from both studies highlighted two critical factors in autism as a discursive and multimodal phenomenon occupying a socio-cultural niche. A) Autism evolves through a conflictual and irreconcilable discursive framework. This conflict reflects profound issues of power that were taken as residing in a micro-fascism political dynamic. B) A need to break from the dichotomous deployment of autism in the current political setting is becoming apparent. The current clinical and social arrangement needs to change; a negotiation in which psychoanalytically and relationally inspired disability politics may become central. Part of this new “diplomacy” lies in engineering new discursive research designs that could offer the opportunity for the two realms to inter-relate in unforeseen and unpredictable ways.
37 citations
Cites background from "The Routledge handbook of multimoda..."
...The limitations of multimodal research often refer to the ‘impressionistic’ (p. 26) character of the approach (Jewitt, 2009)....
[...]
...What could also be happening here is that the author is trying to mediate or change the inevitability of the visual first impression which is a classical neurotypical sway of 12 The term frozen action is used to describe latent actions that are entailed in material objects (Jewitt, 2009)....
[...]
...A brief review of the different multimodal schools (Adami, 2017: Jewitt, 2009), shows that apart from limited exceptions (Zidjaly, 2015; Doak, 2019), the endeavours undertaken inside this field, assume a relatively unproblematic view of the relational events among people....
[...]
...What could also be happening here is that the author is trying to mediate or change the inevitability of the visual first impression which is a classical neurotypical sway of 12 The term frozen action is used to describe latent actions that are entailed in material objects (Jewitt, 2009)....
[...]
01 Jan 2017
TL;DR: In this article, some pitfalls in analysing visual and multimodal metaphors are discussed, and some ideas are put forward to make these insights productive in educational contexts. But they do not consider how metaphors involving visuals may misfire when they are interpreted by members from another culture than the one for which they were designed.
Abstract: It is often claimed that a picture tells us more than a thousand words, but studying pictorial metaphors reveals how much background knowledge is needed to understand and evaluate visuals. Commercial print advertising and billboards make for good case studies, because their goal is unambiguous: to sell consumer products and services. In this chapter some of the pitfalls in analysing visual and multimodal metaphors are discussed, Consideration of a number of examples suggests how metaphors involving visuals may misfire when they are interpreted by members from another culture than the one for which they were designed. In the conclusion some ideas are put forward to make these insights productive in educational contexts.
36 citations
Cites background from "The Routledge handbook of multimoda..."
...In the first 1 The question of what constitutes a mode is a much-debated and hitherto unresolved issue (Forceville 2010)....
[...]