Multimodal sentences: gestural completion as a communicative resource
Abstract
In this paper we study multimodal utterances in which a gestural element completes a syntactic function within a verbally incomplete sentence. As it has happened with other phenomena typical of spoken language, its study has been disregarded because it has been considered a performance problem: gesturing would be simply a resource to make up for the speaker’s deficit in producing a verbal element. On the contrary, our analysis shows that gestural completions constitute a communicative resource with its own functions (intensification, attenuation and favoring discursive progression) that transcend the use of a possible verbal equivalent. By adopting the emic perspective of interactional linguistics, we argue that these utterances are considered as complete by the participants and, as such, constitute multimodal sentences. Thus, our paper proposes the inclusion of gesture within the study of grammar. The data stems from three audiovisual corpora that include samples of different varieties of Spanish.Downloads
References
Alcaraz Carrión, Daniel y Javier Valenzuela. 2022. “Time as space vs. time as quantity in Spanish: A co-speech gesture study.” Language and Cognition 14(1): 1-18.
Auer, Peter y Elisabeth Zima. 2021. “On word searches, gaze, and co-participation.” Gesprächsforschung - Online - Zeitschrift zur verbalen Interaktion 22: 390-425.
Briz Gómez, Antonio. 2011. El español coloquial en la conversación: Esbozo de pragmagramática. Barcelona: Ariel.
Calbris, Genevieve. 1990. The semiotics of French gestures. Bloomington: Indiana University Press.
Cantarutti, Marina Noelia. 2022. “Co-Animation in Troubles-Talk.” Research on Language and Social Interaction 55(1): 37-58.
Cestero Mancera, Ana María. 2017. “La comunicación no verbal.” En Manual del profesor de ELE, editado por Ana María Cestero Mancera y Inmaculada Calvo Penadés, 1051-1122. Alcalá: Universidad de Alcalá.
Couper-Kuhlen, Elizabeth. 1999. “Coherent Voicing: On Prosody in Conversational Reported Speech.” En Coherence in Spoken and Written Discourse (Pragmatics & Beyond New Series), vol. 63, editado por Wolfram Bublitz, Uta Lenk y Eija Ventola, 11-34. Ámsterdam: John Benjamins.
Couper-Kuhlen, Elisabeth y Margret Selting. 2018. Interactional linguistics: Study language in social interaction. Cambridge: Cambridge University Press.
Dankel, Philipp y Mario Soto Rodríguez. 2021. “Contacto y multimodalidad: Listas andinas y sus gestos, entre adaptaciones situacionales, tendencias regionales y pautas universales.” En Variedades del español en contacto con otras lenguas, editado por Élodie Blestel y Azucena Palacios, 157-180. Lausana: Peter Lang.
Deppermann, Arnulf y Alexandra Gubina. 2021. “When the Body Belies the Words: Embodied Agency With darf/kann ich? (‘May/Can I?’) in German.” Frontiers in Communication 6: 661800.
Dressel, Dennis. 2020. “Multimodal word searches in collaborative storytelling: On the local mobilization and negotiation of coparticipation.” Journal of Pragmatics 170: 37-54.
Ehmer, Oliver. 2011. Imagination und Animation: Die Herstellung mentaler Räume durch animierte Rede. Berlín: De Gruyter.
Ehmer, Oliver, Ignacio Satti, Angelita Martínez & Stefan Pfänder. 2019. „Un sistema para transcribir el habla en la interacción: GAT 2”. Gesprächsforschung - Online-Zeitschrift zur verbalen Interaktion (ISSN 1617-1837) 20: 64-114.
Ehmer, Oliver y Daniel Mandel. 2021. “Projecting action spaces. On the interactional relevance of cesural areas in co-enactments.” Open Linguistics 7(1): 638-665.
Enfield, Nick. 2009. The Anatomy of Meaning. Cambridge: Cambridge University Press.
Fornés Pallicer, María Antonia y Mercè Puig Rodriguez-Escalona. 2008. El porqué de nuestros gestos: La Roma de ayer en la gestualidad de hoy. Barcelona: Octaedro.
Garachana, Mar y María Sol Sansiñena. 2023. “¿Cómo explicar el cambio lingüístico marginal? Los modelos lingüísticos actuales y el estudio de los marginalia.” Boletín de Filología 58(1): 13-46.
González Temer, Verónica y Richard Ogden. 2021. “Non-convergent boundaries and action ascription in multimodal interaction.” Open Linguistics 7(1): 685-706.
Goodwin, Marjorie Harness y Charles Goodwin. 1986. “Gesture and coparticipation in the activity of searching for a word.” Semiotica 62(1-2): 51-76.
Holt, Elizabeth. 1996. “Reporting on Talk: The Use of Direct Reported Speech in Conversation.” Research on Language & Social Interaction 29(3): 219-245.
Imo, Wolfgang y Jens Philipp Lanwer. 2019. Interaktionale Linguistik: Eine Einführung. Heidelberg: J. B. Metzler Verlag.
Keevallik, Leelo. 2013. “The Interdependence of Bodily Demonstrations and Clausal Syntax.” Research on Language & Social Interaction 46(1): 1-21.
Keevallik, Leelo. 2018. “What Does Embodied Interaction Tell Us About Grammar?” Research on Language and Social Interaction 51(1): 1-21.
Kendon, Adam. 1988. “How gestures can become like words.” En Cross-cultural perspectives in nonverbal communication, editado por Fernando Poyatos, 131-141. Toronto: Hogrefe.
Kendon, Adam. 2004. Gesture: Visible action as utterance. Cambridge: Cambridge University Press.
Kita, Sotaro. 2000. “How representational gestures help speaking.” En Language and gesture, editado por David McNeill, 162-185. Cambridge: Cambridge University Press.
Ladewig, Silva H. 2014a. “Creating multimodal utterances: The linear integration of gestures into speech.” En Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/2, editado por Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill y Jana Bressem, 1662-1677. Berlín: De Gruyter.
Ladewig, Silva H. 2014b. “The cyclic gesture.” En Handbücher zur Sprach- und Kommunikationswissenschaft / Handbooks of Linguistics and Communication Science (HSK) 38/2, editado por Cornelia Müller, Alan Cienki, Ellen Fricke, Silva Ladewig, David McNeill y Jana Bressem, 1605-1618. Berlín: De Gruyter.
Le Guen, Olivier. 2018. “La importancia de la gestualidad en el estudio del maya yucateco actual.” Cuadernos de Lingüística de El Colegio de México 5(1): 427-469.
Li, Xiaoting. 2021. “Multimodal practices for negative assessments as delicate matters: Incomplete syntax, facial expressions, and head movements.” Open Linguistics 7(1): 549-568.
Linell, Per 2005. The written language bias in linguistics: Its nature, origins and transformations. Londres y Nueva York: Routledge.
Meo-Zilio, Giovanni y Silvia Mejía. 1980. Diccionario de gestos: España e Hispanoamérica. Bogotá: Instituto Caro y Cuervo.
McNeill, David. 1992. Hand and mind: What gestures reveal about thought. Chicago: The University of Chicago Press.
Mondada, Lorenza. 2001. Conventions for multimodal transcription. https://franzoesistik.philhist.unibas.ch/fileadmin/user_upload/franzoesistik/mondada_multimodal_conventions.pdf.
Mondada, Lorenza. 2014. “The local constitution of multimodal resources for social interaction.” Journal of Pragmatics 65: 137-156.
Müller, Cornelia. 1998. Redebegleitende Gesten: Kulturgeschichte, Theorie, Sprachvergleich. Berlín: Spitz.
Núñez, Rafael E. y Eve Sweetser. 2006. “With the Future Behind Them: Convergent Evidence from Aymara Language and Gesture in the Crosslinguistic Comparison of Spatial Construals of Time.” Cognitive Science 30(3): 401-450.
Oloff, Florence. 2014. “L’évaluation des complétions collaboratives: Analyse séquentielle et multimodale de tours de parole co-construits.” SHS Web of Conferences 8: 2125-2145.
Payrató, Lluís. 1989. “Assaig de dialectologia gestual. Aproximació pragmàtica al repertori bàsic d’emblemes del català de Barcelona”. Tesis doctoral, Universitat de Barcelona.
Pekarek Doehler, Simona, Leelo Keevallik y Xiaoting Li. 2022. “Editorial: The Grammar-Body Interface in Social Interaction.” Frontiers in Psychology 13: 875696.
Pfeiffer, Martin y Clarissa Weiss. 2022. “Reenactments during tellings: Using gaze for initiating reenactments, switching roles and representing events.” Journal of Pragmatics 189: 92-113.
Pomerantz, Anita. 1984. “Agreeing and disagreeing with assessments: Some features of preferred/dispreferred turn shapes.” En Structures of Social Action, editado por J. Maxwell Atkinson, 57-101. Cambridge: Cambridge University Press.
Posio, Pekka y Andrea Pešková. 2020. “Le dije yo, digo: Construccionalización de los introductores cuotativos con el verbo decir en español peninsular y argentino.” Spanish in Context 17(3): 391-414.
Poyatos, Fernando. 1994. La comunicación no verbal. Madrid: Istmo.
Raymond, Chase Wesley y Luís Manuel Olguín. 2022. Análisis de la conversación: Fundamentos, metodología y alcances. Londres y Nueva York: Routledge.
Sacks, Harvey, Emanuel A. Schegloff y Gail Jefferson. 1974. “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50(4): 696-735.
Sidnell, Jack. 2006. “Coordinating Gesture, Talk, and Gaze in Reenactments.” Research on Language & Social Interaction 39(4): 377-409.
Slama-Cazacu, Tatiana. 1976. “Nonverbal Components in Message Sequence: ‘Mixed Syntax’”. En Language and Man, editado por William C. McCormack y Stehpen. A. Wurm, 217-228. Berlín: De Gruyter Mouton.
Stivers, Tanya y Jack Sidnell. 2005. “Introduction: Multimodal interaction.” Semiotica 156: 1-20.
Streeck, Jürgen. 2009. Gesturecraft: The manu-facture of meaning. Vol. 2. Ámsterdam: John Benjamins.
- Authors keep the copyright and give the journal the right of the first publication, with the work registered with the Creative Commons Attribution-ShareAlike 4.0 International License, which allows third parties to use what is published whenever they mention the authorship of the work and the first publication in this magazine.
- Authors can make other independent and additional contractual agreements for the non-exclusive distribution of the article published in this journal (eg, include it in an institutional repository or publish it in a book) as long as they clearly indicate that the work It was published for the first time in this magazine.
- Authors are allowed and recommended to publish their work on the Internet (for example on institutional or personal pages).
