List of Figures List of Tables Foreword Introduction Robin P. Fawcett, M.A.K. Halliday, Sydney M. Lamb and Adam Makkai 1 Language as Code and Language as Behaviour: A Systemic-Functional Interpretation of the Nature and Ontogenesis of Dialogue M.A.K. Halliday 2 Metaphors of Information John Regan 3 How Universal is a Localist Hypothesis? A Linguistic Contribution to the Study of 'Semantic Styles' of Language Yoshihiko Ikegami 4 Some Speculations on Language Contact in a Wider Setting Jeffrey Ellis 5 Ways of Saying: Ways of Meaning Ruqaiya Hasan Index

Language as social semiotic

The systemic functional (SF) approach to multimodal discourse analysis (MDA) is concerned with the theory and practice of analysing meaning arising from the use of multiple semiotic resources in discourses which range from written, printed and electronic texts to material lived-in reality. The SF-MDA approach developed in this article explores the meaning arising through the use of language and visual imagery in printed texts. This involves investigation of linguistic and visual forms of semiosis, and formulation of cross-functional systems such as colour. An integrative platform based on the SF metafunctional principle is proposed, and intersemiotic mechanisms and systems (content and expression strata) are developed to capture the expansion of meaning which occurs when linguistic and visual forms combine. The SF-MDA approach is demonstrated through the analysis of ideational meaning in a print advertisement. The practical approach involves the use of digital technology in the form of image-editing software which gives rise to a more detailed semantic and ideological interpretation. The analysis reveals how metaphorical constructions of meaning (i.e. semiotic metaphors) take place across linguistic and visual elements.

/pdf/systemic-functional-multimodal-discourse-analysis-sf-mda-h5lt9igrjq.pdf

Systemic functional-multimodal discourse analysis (SF-MDA): constructing ideational meaning using language and visual imagery

1. Contributors 2. Acknowledgements 3. Introduction (by Cienki, Alan) 4. Why study metaphor and gesture? (by Cienki, Alan) 5. From left to right...: Coverbal gestures and their symbolic use of space (by Calbris, Genevieve) 6. Gesture as a conceptual mapping tool (by Williams, Robert F.) 7. A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor (by Nunez, Rafael) 8. Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar (by Mittelberg, Irene) 9. Unexpected metaphors (by McNeill, David) 10. Catchment, growth point and spatial metaphor: Analysing Derrida's oral discourse on deconstruction (by Montredon, Jacques) 11. Form, meaning, and convention: A comparison of a metaphoric gesture with an emblem (by Parrill, Fey) 12. What gestures reveal about the nature of metaphor (by Muller, Cornelia) 13. Commentaries on the value of studying metaphor and gesture from the perspectives of different disciplines 14. Metaphoric gesture and cognitive linguistics (by Langacker, Ronald W.) 15. Metaphoric gestures and cultural analysis (by Quinn, Naomi) 16. Metaphor and gesture: A view from the microanalysis of interaction (by Streeck, Jurgen) 17. Implications of cognitive metaphor and gesture studies for Ethnomethodology and Conversation Analysis and vice versa (by Hougaard, Anders) 18. Sign and gesture: Towards a new paradigm (by Wilcox, Sherman) 19. The study of metaphor and gesture: A critique from the perspective of semiotics (by Bouissac, Paul) 20. The neuroscience of metaphoric gestures: Why they exist (by Lakoff, George) 21. Metaphor and gesture: Some implications for psychology (by Gibbs, Jr., Raymond W.) 22. Index

Metaphor and Gesture

The increasingly integrative use of images with language in many different types of texts in electronic and paper media has created an urgent need to go beyond logocentric accounts of literacy and literacy pedagogy. Correspondingly there is a need to augment the genre, grammar and discourse descriptions of verbal text as resources for literacy pedagogy to include descriptions of the meaning-making resources of images. Some augmentation along these lines has involved the articulation of Hallidayan systemic functional descriptions of language, mainly focussed on verbal grammar, with the social semiotic descriptions of the meaning-making resources of images described in a grammar of visual design proposed by Kress and van Leeuwen. However, current research indicates that articulating discrete visual and verbal grammars is not sufficient to account for meanings made at the intersection of language and image. This paper adopts a systemic functional semiotic perspective in outlining a range of different types of such meanings in different kinds of texts, suggesting the significance of such meanings in comprehending and composing contemporary multimodal texts, and the importance of developing an appropriate metalanguage to enable explicit discussion of these meaning- making resources by teachers and students.

/pdf/towards-a-metalanguage-for-multiliteracies-education-1itlpgqzu9.pdf

Towards a Metalanguage for Multiliteracies Education: Describing the Meaning-Making Resources of Language-Image Interaction.

Foreword Note on graphic conventions 0. Introduction-Toward a Logic of Culture 0.1. Design for a semiotic theory 0.2. 'Semiotics': field or discipline? 0.3. Communication and/or signification 0.4. Political boundaries: the field 0.5. Natural boundaries: two definitions of semiotics 0.6. Natural boundaries: inference and signification 0.7. Natural boundaries the lower threshold 0.8. Natural boundaries: the upper threshold 0.9. Epistemological boundaries 1. Signification and Communication 1.1. An elementary communicational model 1.2. Systems and codes 1.3. The s-code as structure 1.4. Information, communication, signification 2. Theory of Codes 2.1. The sign-function 2.2. Expression and content 2.3. Denotation and connotation 2.4. Message and text 2.5 Content and referent 2.6. Meaning as cultural unit 2.7. The interpretant 2.8. The semantic system 2.9. The semantic markers and the sememe 2.10. The KF model 2.11. A revised semantic model 2.12. The model \"Q\" 2.13. The format of the semantic space 2.14. Overcoding and undercoding 2.15. The interplay of codes and the message as an open form 3. Theory of Sign Production 3.1. A general survey 3.2. Semiotic and factual statements 3.3. Mentioning 3.4 The prolem of a typology of signs 3.5. Critique of iconism 3.6. A typology of modes of production 3.7. The aesthetic text as invention 3.8. The rhetorical labor 3.9. Ideological code switching 4. The Subject of Semiotics References Index of authors Index of subjects

A Theory of Semiotics

This article presents a generalized system of image–text relations which applies to different genres of multimodal discourse in which images and texts co-occur. It combines two kinds of relations – the relative status of images and text, and how they relate to one another in terms of logico–semantics. Every instance of an image–text combination in the data sample is described by a selection of features from the system. The units of images and text between which the relations obtain are identified and the realizations of the logico–semantic and status relations are specified, both for the human analyst and a machine. Two application scenarios are discussed. The system should be useful for distinguishing between image–text relations for (genuinely) new and old media.

/pdf/a-system-for-image-text-relations-in-new-and-old-media-1b5w82z33v.pdf

A system for image-text relations in new (and old) media

This article first develops the system of image–text relations introduced in Martinec and Salway’s (2005) article – ‘A system for image–text relations in new (and old) media’) – mainly by focusing ...

Nascent and mature uses of a semiotic system: the case of image–text relations

The combination of different media types is a defining characteristic of multimedia yet much research has concentrated on understanding the semantics of media types individually Recently, systems have been developed to process correlated image and text data for tasks including multimedia retrieval, fusion, summarization, adaptation and generation We argue that the further development and the more general application of such systems require a better computational understanding of image-text combinations In particular we need to know more about the correspondence between the semantic content of images and the semantic content of texts when they are used together This paper outlines a new area of multimedia research focused on modeling image-text combinations Our aim is to develop a general theory to be applied in the development of multimedia systems that process correlated image and text data Here, we propose a theoretical framework to describe how visual and textual information combine in terms of semantic relations between images and texts Our classification of image-text relations is grounded in aesthetic and semiotic theory and has been developed with a view to automatic classification

Radan Martinec

Papers

A system for image-text relations in new (and old) media

Nascent and mature uses of a semiotic system: the case of image–text relations

Some Ideas for Modelling Image-Text Combinations