Book•
The Routledge handbook of multimodal analysis
01 Jan 2009-
TL;DR: Theoretical and Methodological Tools for Multimodal Analysis as mentioned in this paper is a toolkit for multimodal analysis with a focus on the analysis of the transmodal moment.
Abstract: Introduction: Handbook Rationale, Scope and Structure Part 1 Theoretical And Methodological Tools For Multimodal Analysis 1.An Introduction to multimodalit 2. Different approaches to multimodality 3.What are multimodal data and transcription? 4.What is mode? 5.Parametric systems: the case of voice quality Theo van Leeuwen 6. Modal density and modal configurations: multimodal actions 7. Transformation, transduction and the transmodal moment Part 1 readings Par 2 Key themes for multimodality 8. Historical Changes in the Semiotic Landscape From Calculation to Computation 9. Technology and Sites of Display 10. Multimodality and Mobile Culture 11. Multimodality, Identity, and Time 12. Multimodality and reading: the construction of meaning through image-text interaction 13. Power, social justice and multimodal pedagogies Part 3 Multimodality across different theoretical perspectives 14. Multimodality and language: A retrospective and prospective view 15. Multimodality and theories of the visual 16. Multimodality and New Literacy Studies 17. Using Multimodal Corpora for Empirical Research 18. Critical Discourse Analysis and multimodality 19. Semiotic paradigms and multimodality 20. Reception of multimodality: Applying eye-tracking methodology in multimodal research 21. Representations in practices: A socio-cultural approach to multimodality in reasoning 22. Indefinite precision: artefacts and interaction in design 23. Anthropology and Multimodality: The Conjugation of the Senses Part 4 Multimoda Case Studies 24. Practical function and meaning: a case study of Ikea tables 2 The use of gesture in operations 26. Gesture and Movement in Tourist Spaces 2 The kineikonic mode: towards a multimodal aproach to moving image media 28. Multimodal Analytics: Software and Visualization Techniques for Analyzing and Interpreting Multimodal Data 29. Colour: code, mode, modality -- the case of.
Citations
More filters
Dissertation•
26 Sep 2018
TL;DR: In this paper, a cognitive framework for character impression formation is proposed based on the cognitive concept of schema, which is inscribed within the cognitive stylistics theories and analytical frameworks, and explores the question of how narrative film viewers presumably understand characters and make impressions about them as a result of the interplay of different types of schema.
Abstract: This thesis deals with the cognitive process of cinematic character construal and explores the question of how narrative film viewers presumably understand characters and make impressions about them as a result of the interplay of different types of schema.
Based on the cognitive concept of schema (Barlett [1932] 1995), Speech Act Theory (Austin, 1962; Searle, 1969) and cognitivist film theory (Bordwell,1985), a cognitive framework, which is inscribed within the cognitive stylistics theories and analytical frameworks, is proposed for character impression formation. The model posits that viewers understand characters in terms of the interactions between three planes of background knowledge: 1. Social knowledge (knowledge of real-life people, social roles, and interpersonal relations), 2. Narrative film knowledge (knowledge of film narrative, style and techniques), 3. Pragmalinguistic knowledge (knowledge of linguistic and pragmatic norms. The focus of the linguistic plane is on Speech Act Theory).
This research argues that film draws on medium-specific, multimodal devices to tell the story and create characters. With regard to the multimodality of film discourse, this thesis suggests a toolkit for character creation and comprehension. The three planes of social, film and pragmalinguistic schemas and also the marked aspects of the suggested cinematic character creation toolkit are all applied to three art auteur films: The Piano Teacher (2001), Autumn Sonata (1987), and Ten (2001), whose comprehension is challenging for viewers Pragmalinguistic because of their deviations from the conventional norms of Hollywood cinema in terms of narrative, characterization, cinematic style and techniques. Methodologically, this analysis is informed by multimodal analysis (Kress and van Leeuwen, 1996), and multimodal transcription (Baldry and Thibault, 2006) in particular, which consider text as an ensemble of different communication modes all of which contribute to meaningmaking. Thus, this analysis presents a detailed account of viewers’ plausible understanding trajectory, and an explanation of linguistic and visual/cinematic strategies to narrative (as the context in which characters are created and developed), and particularly character creation within the scope of the films mentioned above.
24 citations
Cites background from "The Routledge handbook of multimoda..."
...In fact, the meanings in any mode are interwoven with the meanings constructed by those of all the other co-present and cooperated modes in a given communicative event (Jewitt, 2009)....
[...]
TL;DR: It is argued for re-imagining what texts, multimodality and identity are and do in the age of AI, and some of the forms of critical consciousness that media education might provide in this new mixed landscape are sketched.
Abstract: Computational objects (eg, algorithms, bots, surveillance technology and data) have become increasingly present in our daily lives and are consequential for our changing relations to texts, multimodality and identity. Yet, our current theories of literacy, and especially the prevalence of mediational and representational perspectives, are inadequate to account for these changing relations. What are the implications for critical literacy education when it takes seriously computational agents that interact, produce and process texts? While such work is only beginning in education, scholars in other fields are increasingly writing about how AI and algorithmic mediation are changing the landscape of online intra-action, and business strategies and tactics for working with AI are advancing far ahead of critical literacy education. Drawing on our own and others? research into non-human actors online, and building on posthuman theories of networks, heterogeneous actants and the assemblage, in this conceptual paper, we sketch some of the forms of critical consciousness that media education might provide in this new mixed landscape. Practitioner Notes What is already known about this topic AI is a hot topic in education and in public discourse, but critical literacy theories have not sufficiently accounted for how AI and computational agents change what it means to be ?critically literate.? Technology is an important force in shaping (and is also shaped by) literacy practices and identity. Corporate actors have an enormous influence on the texts we read and write, but this influence is often hidden. What this paper adds We bridge between critical literacy studies and posthumanist theory to conceptualize critical posthuman literacy. We argue for re-imagining what texts, multimodality and identity are and do in the age of AI. We pose new questions of our texts and ourselves, informed by posthuman critical literacy. Implications for practice and/or policy Today?s readers and composers must be able to identify and interrogate networks of computational and human agents that permeate literacy practices. Beyond identifying and understanding computational agents, posthuman critical literacy necessitates that people can actively build more ethical assemblages with computational agents.
24 citations
Cites background from "The Routledge handbook of multimoda..."
...…and identity Media educators have developed robust ways of critically analyzing and understanding the multimodal qualities of texts (eg, Jewitt, 2009; Kress, 2009; Manovich, 2001), for understanding the expansion and pluralization of literacy (eg, Cope & Kalantzis, 2000) and for…...
[...]
Dissertation•
01 Jan 2015
TL;DR: In this paper, a study of an engage, study, activate (ESA) lesson of teaching modals of present deduction is presented in a published English language teaching course book and is typical of the way modal forms are presented to teach epistemic modality in many commercially produced English Language teaching course books.
Abstract: This thesis is a study of an engage, study, activate (ESA) lesson of teaching modals of present deduction. The lesson has been taken from a published English language teaching course book and is typical of the way modal forms are presented to teach epistemic modality in many commercially produced English language teaching course books. I argue that for cognitive, social, linguistic and procedural reasons the linguistic forms and structures presented in the lesson are not straightforwardly transferred to the activate stage of the lesson.
Using insights from spoken language corpora I carry out a comparative analysis with the modal forms presented in the course book. I then explore the notion of ‘context’ and drawing on systemic functional grammar discuss how modal forms function in discourse to realise interpersonal relations. Moving my research to the English language classroom I collect ethnographic classroom data and using social semiotic multimodality as an analytical framework I explore learner interaction to uncover the communicative resources learners use to express epistemic modality in a discussion activity from the same lesson.
My analysis reveals that the modal structures in the course book differ to some extent from spoken language corpora. It shows that the course book offers no instruction on the interpersonal dimension of modality and thus how speakers use signals of modality to position themselves interpersonally vis-a-vis their interlocutors. The data collected from the English language class reveals that during the lesson learners communicate modality through modes of communication such as eye gaze, gesture and posture in addition to spoken language. Again drawing from systemic functional grammar I explain how these modes have the potential to express interpersonal meaning and thus highlight that meaning is communicated through modal ensembles.
Based on these findings I propose a number of teaching strategies to raise awareness of the interpersonal function of modality in multimodal discourse, and for the use of language corpora to better inform teaching materials on selections of modality.
24 citations
TL;DR: In this paper, final-year chemistry students studying a pharmacology subject created a "blended media" digital product as an assignment to summarize an independently prepared technical literature review on a current research topic in pharmacology for a non-expert audience.
Abstract: Students can now digitally construct their own representations of scientific concepts using a variety of modes including writing, diagrams, 2-D and 3-D models, images or speech, all of which communicate meaning. In this study, final-year chemistry students studying a pharmacology subject created a “blended media” digital product as an assignment to summarize an independently prepared technical literature review on a current research topic in pharmacology for a non-expert audience. A blended media is a simplified way for students to combine a variety of modes to complement a narration to explain a concept to others. In this study, the students learned how to create a blended media during a one-hour workshop, and used the technique to create the representation as an assessment task. The research question that guided the study was, “What are the students’ perceptions of making a digital product such as blended media and how did these shape their multimodal awareness?” We draw from theoretical perspectives in multimodalities, representations and meaning making. Data included interviews at three points of the semester, the literature review and the digital media product. We present three case studies with volunteering students, who demonstrated a strong awareness of effective communications techniques as they attended to the audience. Making a blended media is a creative way for chemistry students to summarize complex scientific information and as a task may help to focus their multimodal awareness and developing communications skills.
23 citations
Cites background from "The Routledge handbook of multimoda..."
...Representations that include more than one mode are ‘multimodal’ and student awareness of modal choices helps to create a coherent product, as noted by Jewitt (2009): “semiotic mode[s] combine as resources for meaning making…as the resources of different modes are combined, meanings are…...
[...]
01 Jan 2013-Review of Cognitive Linguistics. Published under the auspices of the Spanish Cognitive Linguistics Association
TL;DR: In this paper, the metaphor searching for one's identity is looked for a home in animation films and it is shown that analysing this metaphor presupposes understanding "home" as a symbol.
Abstract: The quickly growing discipline of multimodality has hitherto primarily found its inspirational models in semiotics and in Systemic Functional Linguistics. However, Cognitive Linguistics, and specifically its Conceptual Metaphor Theory branch, has over the past years proved a store of knowledge and methods of analysis that can benefit the further advance of the young discipline. In this paper the metaphor searching for one’s identity is looking for a home in animation films is examined. It is shown that (a) analysing this metaphor presupposes understanding “home” as a symbol; (b) animation has medium-specific affordances to implement the metaphor; (c) the metaphor combines embodied and cultural dimensions.
23 citations