Book•
The Routledge handbook of multimodal analysis
01 Jan 2009-
TL;DR: Theoretical and Methodological Tools for Multimodal Analysis as mentioned in this paper is a toolkit for multimodal analysis with a focus on the analysis of the transmodal moment.
Abstract: Introduction: Handbook Rationale, Scope and Structure Part 1 Theoretical And Methodological Tools For Multimodal Analysis 1.An Introduction to multimodalit 2. Different approaches to multimodality 3.What are multimodal data and transcription? 4.What is mode? 5.Parametric systems: the case of voice quality Theo van Leeuwen 6. Modal density and modal configurations: multimodal actions 7. Transformation, transduction and the transmodal moment Part 1 readings Par 2 Key themes for multimodality 8. Historical Changes in the Semiotic Landscape From Calculation to Computation 9. Technology and Sites of Display 10. Multimodality and Mobile Culture 11. Multimodality, Identity, and Time 12. Multimodality and reading: the construction of meaning through image-text interaction 13. Power, social justice and multimodal pedagogies Part 3 Multimodality across different theoretical perspectives 14. Multimodality and language: A retrospective and prospective view 15. Multimodality and theories of the visual 16. Multimodality and New Literacy Studies 17. Using Multimodal Corpora for Empirical Research 18. Critical Discourse Analysis and multimodality 19. Semiotic paradigms and multimodality 20. Reception of multimodality: Applying eye-tracking methodology in multimodal research 21. Representations in practices: A socio-cultural approach to multimodality in reasoning 22. Indefinite precision: artefacts and interaction in design 23. Anthropology and Multimodality: The Conjugation of the Senses Part 4 Multimoda Case Studies 24. Practical function and meaning: a case study of Ikea tables 2 The use of gesture in operations 26. Gesture and Movement in Tourist Spaces 2 The kineikonic mode: towards a multimodal aproach to moving image media 28. Multimodal Analytics: Software and Visualization Techniques for Analyzing and Interpreting Multimodal Data 29. Colour: code, mode, modality -- the case of.
Citations
More filters
TL;DR: The analysis focuses on how semiotic resources available in DVC are used for meaning-making, drawing on semiotics, interactional sociolinguistics, nonverbal communication, multimodal interaction analysis and conversation analysis.
Abstract: Online language learning and teaching in multimodal contexts has been identified as one of the key research areas in computer-aided learning (CALL) (Lamy, 2013; White, 2014). 1 This paper aims to explore meaning-making in online language learner interactions via desktop videoconferencing (DVC) and in doing so illustrate multimodal transcription and analysis as well as the application of theoretical frameworks from other fields. Recordings of learner DVC interactions and interviews are qualitatively analysed within a case study methodology. The analysis focuses on how semiotic resources available in DVC are used for meaning-making, drawing on semiotics, interactional sociolinguistics, nonverbal communication, multimodal interaction analysis and conversation analysis. The findings demonstrate the use of contextualization cues, five codes of the body, paralinguistic elements for emotional expression, gestures and overlapping speech in meaning-making. The paper concludes with recommendations for teachers and researchers using and investigating language learning and teaching in multimodal contexts.
46 citations
TL;DR: This article framed within social semiotic perspectives on multimodality, and analyzed the ways in which teachers draw on different semiotic resources for multi-modal learning, and the primary aim of the study was to analyze the ways teachers drew on different resources for different tasks.
45 citations
45 citations
DOI•
01 Jan 2017
TL;DR: In this paper, the authors use cartography and ethnographic methods to capture and analyse both the momentary seizing of space and the latent creation of new sovereign urban spaces that defy current segregative urbanism.
Abstract: Cities are much more than just the background of our lives, or complex artefacts. They are the social field and the content over which we quarrel; the reason and the tool by which we co-operate. Cities are shaped in the form of our passions. Those passions do not just give meaning to our world and to our lives, they prompt us to act upon them by enacting new spatialities to come. This thesis deals with the enactment of new forms of space and city through emotions. The recently identified âNew Urban Questionâ defines the combination of increasing social inequalities, growing spatial segregation, and climate change vulnerabilities that crystallize in our cities. It also signals a crisis in design and planning disciplines, urging researchers and designers to renew their methods for studying and designing the city. In response to this call, my research considers the way emotions shape both our way of understanding and making cities. Despite often disregarded as either contagious and irrational hysteria or a subjective question of taste, I claim emotions to be a constitutive dimension of the urban condition. In urban conflict specifically, both the urban condition and affective phenomena become salient and their relevance is highlighted. The recent activities of a group of activists and social movements in Madrid - emerged around the mobilizations of the 15M in Spain - constitute an ideal context for this research, for they are spatially constituted and are spatially performative. Their contentious actions take the form of a series of spatial practices, called in this thesis New Urban Practices, where I find a potential answer to both the crisis of the contemporary city and its mirroring crisis in architecture and urbanism. I approach the analysis of New Urban Practices in Madrid through three complementary empirical studies that tackle space, action and emotion, making use of a transdisciplinary mixed-methods approach. The first study tackles the spatial conditions of porosity that have accompanied the emergence and development of these social movements. It uses methods developed by Space Syntax to analyse spatial configurationâand cartography and demographics to analyse the spatial content. The second study elaborates on the actions of these social movements, their spatial practices. In this study, I use cartography and ethnographic methods to capture and analyse both the momentary seizing of space and the latent creation of new sovereign urban spaces that defy current segregative urbanism. Finally, the third study explores the emotions felt and experienced by these activists. Through emotional dimensional analysis and discursive analysis of surveys and in-depth interviews, I find participants undergo a substantive transformation of their spatial perception accompanying a strong emotional transition. Using theories of Extended Emotions and the ideas behind SIRN, I conclude by identifying urbanity as an extended emotion. In addition, the modes of using and producing city space of the New Urban Practices are found to pose a new model of urban governance and action, the urban prototype, proposing a more inclusive, democratic and socially sustainable forms of enacting our cities, in Europe and worldwide.
45 citations
TL;DR: A social semiotic framework for the multimodal analysis of website interactivity that adapts Halliday’s three metafunctions to the analysis of the two-fold nature and two-dimensional functioning of interactive sites/signs, providing a fine-grained account of the interactive meaning potentials of digital texts.
Abstract: This article presents a social semiotic framework for the multimodal analysis of website interactivity. Distinguishing it from interaction, it defines interactivity as the affordance of a text of being acted (up)on, thus including hypertextuality. The author introduces the notion of ‘interactive sites/signs’ as the loci of interactivity in digital texts; these have a two-fold nature and a two-dimensional functioning. In their two-fold nature, they are both places enabling actions producing effects and forms endowed with meanings. Notwithstanding the non-direct correspondence between forms, actions and effects (which makes any specific association between the three significant within a webpage design), and in spite of their many possible forms (encompassing still and dynamic images, shapes and writing), a small range of actions can activate them (click/click+type/hover), producing a restricted set of textual effects (access/provide/transfer text). In their two-dimensional functioning, interactive sites/signs function both syntagmatically, on the page where they are displayed, in their relation with other co-occurring elements, and paradigmatically, opening to optional text realizations, hence in their relation with these. The framework adapts Halliday’s three metafunctions to the analysis of the two-fold nature and two-dimensional functioning of interactive sites/signs. It provides a fine-grained account of the interactive meaning potentials of digital texts, distinguishing between a text’s aesthetics of interactivity – as visually communicated before it is activated, performed and experienced – and its functionality, in the configuration of interactive possibilities offered by a page. Designed to complement the extant practices of text analysis of webpages, the framework can be used comparatively, as exemplified in its application to the analysis of two blog pages, and can provide a more refined assessment of the interactive meaning potential of a webpage than traditional methodologies such as content analysis.
45 citations
Cites background from "The Routledge handbook of multimoda..."
...Within studies in multimodality (Jewitt, 2009; Kress and Van Leeuwen, 2001, 2006[1996]; O’Halloran, 2004), face-to-face interaction has been studied extensively, specifically by Norris (2004, 2006), who has developed a multimodal framework for its analysis....
[...]