Using ELAN Video Coding Software to Visualize the Rhetorics of Translation

Contributor: Laura Gonzales
Affiliation: University of Texas at El Paso 
Email: gonzlaur at gmail.com
Released: 4 January 2017
Published: Spring 2017 (Issue 21.2) 

Introduction

Conversations about the fluidity of language and the importance of linguistic justice in writing classrooms have been ongoing and evolving in rhetoric and composition scholarship (Guerra, 1998, 2016; Smitherman & Villanueva, 2003; Young & Martinez, 2011). Thanks to this important work, rhetoric and composition has continued to push beyond the myth of linguistic homogeneity, resisting the flawed assumption that classrooms in the US are (or should be) guided by English-dominant ideologies (Matsuda, 2006). Furthermore, recent work in rhetoric and composition continues to move away from static conceptions of language, instead working toward an understanding that "communication always involves a negotiation of mobile codes,” including but not limited to alphabetic Standard Written English (SWE) (Canagarajah, 2013, p. 8).

Following these efforts to acknowledge and value linguistic diversity in writing research and pedagogies, my goal as a researcher (and as a Latina writing instructor) is to better understand the rhetorical strategies enacted by individuals who translate information across languages (e.g., Spanishes, Englishes) to accomplish work (Gonzales, 2015; Gonzales & Zantjer, 2016). While the communicative power of linguistically and culturally diverse students has been studied in rhetoric and composition for decades, my goal is to build on these conversations by visualizing what it is that individuals do as they transform information across languages for various audiences and communities. Given that so much translation work (like other communicative acts) now happens in collaboration with other people, tools, and technologies, I'm interested in understanding how individuals coordinate resources as they translate information. 

In order to study linguistic transformations enacted through translation processes, I use visual methods (e.g., video recording, capturing and creating images, using software to create workflow visualizations) to study how my participants use tools and technologies to translate information from Spanishes to Englishes (and vice versa). As Gail E. Hawisher, Cynthia L. Selfe, Patrick W. Berry, and Synne Skjulstad (2010) explained in their discussion of using video recordings to visualize literacy practices in Transnational Literate Lives in Digital Times, visual data like video recordings can "add additional semiotic information and more to alphabetic representations of research" (Conclusion). Using visual data and other visual methods and methodologies can allow researchers to account (and make space) for the diverse contexts and communicative practices of participants (McKee & DeVoss, 2007). In the case of my specific projects, I have found it useful to use video and visual data as a way to analyze and represent how my participants leverage their linguistic and cultural practices when creating and disseminating information for audiences that speak Englishes and Spanishes in different moments in time. Because languages and linguistic practices are constantly in flux, visual data can provide a useful heuristic for understanding the rhetorical work embedded in linguistic mobility. 

In this webtext, I describe how I used video data to illustrate how professional translators working in a nonprofit language services office use cultural and digital tools to translate information between Englishes and Spanishes for their communities. More specifically, I will explain how I used an open-source digital coding tool, ELAN, to help me visualize and code the translation practices of my participants. While collecting video footage is an important part of working with video data, I focus this entry on how ELAN allowed me to code visual interactions in my videos. In this way, I argue that by using video data in language-focused research projects, researchers can account for verbal, embodied, and material interactions among people and technologies (Bhatt & de Roock, 2014; Pigg, 2014). As I will demonstrate, in my work with professional translators, the added visual representations of data helped me understand how participants used a wide range of rhetorical strategies including gestures, storytelling, and drawing to translate information.  

Background: The Study

In a recent project, I partnered with the Hispanic Center of Western Michigan, a nonprofit organization aiming to serve the needs of the Latinx community in Michigan. As part of my work with this organization, I analyzed how participants in this office translated technical documents (e.g., birth certificates, medical records) for their community members. To do so, I video recorded all translation activities that took place in the Hispanic Center's Language Services office, recording my participants' computer screens as well as their verbal, embodied interactions during their processes of translation. Through this work, I hoped to answer the following questions:

1. What strategies do professional translators use during their translation processes?
2. What tools (digital, embodied) do professional translators use to translate information?

During my work with the Hispanic Center, I video recorded over 400 hours of footage illustrating the translation processes of 12 participants. By analyzing these recordings, I was able to see not only what participants were typing or clicking during their translation process, but also what they were saying, how they were moving, and how they were interacting (with both people and objects) as they translated information. While my video recording methods rendered a substantial amount of visual data, one of the challenges I faced was deciding on a method to analyze this footage. I turned to the digital coding software ELAN to help me analyze exactly what my participants were doing as they translated information. In the section that follows, I’ll briefly introduce ELAN—the name of which is not an abbreviation for anything—before moving on to provide specific examples of how I used this software to help me understand the translation strategies of professional translators.   

Background on ELAN Coding Software

ELAN is an open-source digital transcription and coding tool for multimedia developed by the Max Planck Institute for Psycholinguistics (MPI). Developed by linguists, ELAN is inherently designed to account for the fluidity and flexibility of language. Researchers can upload video data into ELAN, and can then slow down, speed up, and combine various sources of multimedia data throughout the coding process. The ELAN interface allows researchers to code data by time segments, annotating specific segments under various categories determined by the researcher. In this way, researchers can note the both the quantity (how many times something happens) with the duration (how long something lasts) of specific events in any given data set. 

Rather than constraining each instance or iteration to one specific code, ELAN allows researchers to identify one instance or activity as pertaining to multiple layers of codes, known as tiers. While coding categories are typically intended to be mutually exclusive and exhaustive (Blythe, 2007), layered coding provides a more contextualized way to analyze visual data (Gonzales, 2015). These tiers can be applied simultaneously to a specific section of video data, resulting in “complex referential structures” to code both “speech and gesture modalities” (Brugman & Russell, 2004, qtd. in Gonzales, 2015). During my work with translators at the Hispanic Center, I was able to use ELAN’s tiered coding approach to analyze translation in various simultaneous layers. In this way, by using ELAN, I was able to see (and make an argument for) the layered, iterative work that translators engage in as they adapt information. I didn’t have to rely just on what translators were saying or just on where translators were clicking during their translation process. Instead, using ELAN, I could account for these factors while also considering how translators were moving across their office space and how they were interacting with other humans, tools, and technologies. In the sections that follow, I’ll provide examples of how this tiered coding approach helped me understand and discuss the visual data collected for this project. 

Results: Affordances of Digital Coding

The video montage depicted below provides an introduction to the various layers of translation activity that take place at the Hispanic Center. As you watch this short video, I would like you to pay attention to what the participants are saying, and how they are moving––using “gesticulations on the fly, as they are speaking” to clarify and extend the meaning of their Spanish and English words (McNeill, 2002, p. 9). Captions are provided in both English and Spanish to further illustrate the linguistic and cultural adaptations taking place during these interactions. Included in this video are short clips displaying ELAN’s coding platform, where the video continues playing on the top left-hand corner of the screen while the coding categories and duration are displayed in the bottom half of the screen. 

As evidenced in the brief video above, much of the written translation work that takes place at the Hispanic Center happens on the computer. Participants in this organization also engage in verbal interpretation, translating information between Spanish-speaking community members and service providers (e.g., doctors, nurses, police officers). During both written and verbal translations, participants frequently use embodied strategies such as gesturing to clarify information, using their hands to signal meaning when words are not available or sufficient.

For example, in the image provided in Figure 1, Sara, a translator at the Hispanic Center, was translating a flyer for a health insurance organization. During her translation process, Sara paused to make a decision about how to translate the word “champion” into Spanish for a specific audience. During this pause in translation (what I call a translation moment), Sara used the digital translation tool Word Reference to look for a word in Spanish that would signal a “champion” in health insurance rather than a champion of a race or sports event. As she used Word Reference’s options to decide which word to use in her translation, Sara repeated each word provided by Word Reference aloud, using her indexed cultural knowledge and lived experiences to decide which word most accurately matches the rhetorical situations in which she had used this term before.

Sara repeated the words “campeón” and “triunfador” over and over again during her translation process, attempting to trigger her memories regarding previous contexts in which she has seen these words. As she moved back and forth between these two options, Sara began to move her fingers back and forth on the computer screen, touching each printed word and signaling a recursive back and forth movement as she made her final decision. 

As illustrated in Video 1 and Figure 1, ELAN’s coding tiers allowed me to code this specific instance in Sara’s translation process under several coding categories. For example, I coded this instance as “repeating,” when Sara repeated the words “campeón” and “triunfador” over and over again to decide which one was best to use. I also coded this instance as entailing “gesturing” strategies, as Sara moved her fingers back and forth on the screen. All of these movements, both digital and physical, helped Sara reach her ultimate decision in translation. For Sara, the digital translation tool was just a starting point in the translation process. As Sara explained during a follow-up interview, she often had to “feel the translation––it’s not enough to just read the word.” In this case, using ELAN’s tiered coding scheme, I was able to track how Sara was feeling the various translation options, moving her fingers back and forth and repeating specific terms to signal her recursive practice.

In my 400 hours of video footage, I ended up coding 2,871 pauses in my participants' translation processes (i.e., translation moments). During these pauses, translators used the strategies reflected in my coding scheme (e.g., repeating, gesturing, reading aloud, storytelling) 5,734 times. Using ELAN’s tiered coding system, I was able to note not only the large number of translation strategies used by my participants, but I could also account for the intricate combinations of strategies that took place in the office as translators adapted information across languages. For example, in some instances, my participants began their translation projects sitting in their office on their computer screen, referencing digital translation tools (e.g., Google Translate) when they had a question about how to translate a specific word or phrase. Sometimes, the digital translation tool triggered a specific memory for the translator, who could then make a more informed decision regarding the word or phrase in question. Other times, however, the digital translation tool was not sufficient, and the translator would choose to discuss the translation with another employee. During these moments, translators would tell stories, gesture, sketch, or use another combination of strategies to come up with what they perceived to be a successful translation for a specific audience and context. Using ELAN, in turn, helped me both see the individual strategies translators used in addition to providing a way for me to see how these strategies were combined in practice. At the end of my study, I compiled a list of rhetorical strategies frequently enacted by translators, as illustrated in Figure 2.

Conclusion and Implications

Using ELAN to code video data helped me see the connections between translators' written, verbal, and embodied practices, allowing me to develop a framework for understanding the rhetorical work embedded in translation. By being able to visualize the movement (both verbal and embodied) that took place during my participants' translation processes, and by being able to code these moments in relationship to each other through ELAN's tiers, I was able to develop a more comprehensive understanding of how translators combine rhetorical resources to transform information for their communities. I didn't have to rely on an analysis of translators' final products, but I could instead see in action how translations were accomplished through layered, iterative, and recursive processes. In addition, by being able to record, code, and then visually show my participants how I was coding their translation processes throughout my project, I developed a more rigorous coding scheme that accounted not only for how I was coding translation, but also for how my participants understood and presented their own practices. Using ELAN, I was able to show my participants how I was coding their work throughout our collaboration, in this way embedding opportunities for reflection and triangulation throughout my analysis.  

Since languages are fluid frameworks that are constantly being adapted and re-shaped through human interactions, translation entails much more than just identifying static representations and categorizations of words in any language system. Instead, translation is a rhetorical activity situated within human practices and interactions. Successful or accurate translations depend not on finding correct words, but on identifying the most appropriate utterances to meet the needs of specific audiences in specific moments in time. As I continue thinking about how to implement the findings of this project into my own writing pedagogies, I envision using video coding tools like ELAN to trace how students coordinate languages and technologies to communicate with audiences from different linguistic and cultural backgrounds. Using visual methods to visualize and acknowledge the rhetorical work of translation, I argue, can help rhetoric and composition researchers and teachers understand how multilingual students and professionals repurpose and adapt their ideas across and through languages, both in and beyond our classrooms. Although ELAN is certainly not the only video coding tool available for this work, visual and digital methods and methodologies can provide alternative avenues for multlingual communication projects. 

What is most exciting to me about tools like ELAN is that they allow us to continue developing frameworks for understanding a wide range of communicative practices beyond static, alphabetic discourses. For researchers who study multilingual communication, video coding can help us illustrate how participants are always combining, shifting, and shaping their communicative tools to make verbal and physical adaptations to their messages. Languages have always encompassed more than just words, including embodied lived experiences that mark our words and our bodies. Using digital tools and visual methods, I hope to continue highlighting the powerful rhetorical work of multilingual communication, giving credit to the people and the practices that make this work possible. 

References

Bhatt, Ibar, & de Roock, Roberto. (2014). Capturing the sociomateriality of digital literacy events. Research in Learning Technology, 21. Retrieved December 8, 2016, from http://dx.doi.org/10.3402/rlt.v21.21281

Blythe, Stuart. (2007). Coding digital texts and multimedia. In Heidi McKee & Danielle Nicole DeVoss (Eds.), Digital writing research: Technologies, methodologies, and ethical Issues (pp. 203-28). New York, NY: Hampton Press.

Brugman, Hennie, & Russell, Albert. (2004). Annotating multi-media/multi-modal resources with ELAN. In Maria Teresa Lino, Francisca Xavier, Fátima Ferreira, Rute Costa, & Raquel Silva (Eds.), Proceedings of the 4th International Conference on Language Resources and Language Evaluation (LREC 2004) (pp. 2065-2068). Paris, France: European Language Resources Association. 

Canagarajah, Suresh. (2013). Translingual practice: Global Englishes and cosmopolitan relations. New York, NY: Routledge. 

Gonzales, Laura. (2015). Multimodality, translingualism, and rhetorical genre studies. Composition Forum, 31. Retrieved December 8, 2016, from http://compositionforum.com/issue/31/multimodality.php

Gonzales, Laura, & Zantjer, Rebecca. (2015).Translation as a user-localization practice. Technical Communication, 62(4), 271-284.

Guerra, Juan. (1998). Close to home: Oral and literate practices in a transnational Mexicano community. New York, NY: Teachers College Press.

Guerra, Juan. (2016). Language, culture, identity and citizenship in college classrooms and communities. New York, NY: Routledge.

Hawisher, Gail, Selfe, Cynthia, Berry, Patrick, & Skjulstad, Synne. (2012). Conclusion: Closing thoughts on research methodology. In Patrick Berry, Gail E. Hawisher, & Cynthia L.Selfe (Eds.), Transnational literate lives in digital times. Logan, UT: Computers and Composition Digital Press/Utah State University Press. Retrieved December 10, 2016, from http://ccdigitalpress.org/transnational/conclusion5.html.

Matsuda, Paul Kei. (2006). The myth of linguistic homogeneity in US college composition. College English, 68(6), 637-651.

McNeill, David. (2002). Gesture and language dialectic. Acta Linguistica Hafniensia, 34(1), 7-37.

Pigg, Stacey. (2014). Emplacing mobile composing habits: A study of academic writing in networked social spaces. College Composition and Communication, 66(2), 250-275. 

Smitherman, Geneva, & Villanueva, Victor. (Eds.). (2003). Language diversity in the classroom: From intention to practice. Carbondale, IL: Southern Illinois University Press.

Young, Vershawn A., & Martinez, Aja. (2011). Code meshing as World English: Policy, pedagogy, performance. Urbana, IL: National Council of Teachers of English.