H.19 "Collecting, Analyzing, and Talking about Data"
Reviewed by Fredrik deBoer (firstname.lastname@example.org)
Chair: Jason Swarts, North Carolina State University, Raleigh
Karen Lunsford, University of California-Santa Barbara, “Building a Research Tradition”
Rebecca Rickly, Texas Tech University, Lubbock, “What To Do with What You’ve Got: Representing Research”
Jo Mackiewicz, Auburn University, AL, “Stats a Good Idea: A Brief Introduction to Useful Statistics for Mixed-Methods Research”
Jason Swarts, North Carolina State University, Raleigh, “The Order in Method: How the Messy World of Writers Becomes Tidy”
The use of methods and practices typical of the social sciences remains a controversial issue in rhetoric and composition. Articles such as Davida Charney’s (1996) “Empiricism is Not a Four-Letter Word,” Richard Haswell’s (2005) “NCTE/CCCC’s Recent War on Scholarship,” Susan Peck MacDonald’s (2007) “The Erasure of Language,” and several selections from the edited collection Writing Assessment in the 21st Century: Essays in Honor of Edward M. White (Elliot and Perelman, 2012) describe the ways in which composition studies became less hospitable to empirical work generally and quantitative work specifically. Speaking as a quantitative researcher in rhet/comp myself, I can say that while most people I interact with in the field are respectful and friendly, most are also somewhat skeptical of the use of statistics and techniques from social science in writing research. There are reasons to believe, however, that this is changing; see, for example, College Composition and Communication’s September 2012 special issue on research methodologies. On Friday afternoon at the 2014 Conference on College Composition and Communication, I attended a panel that spoke to the growing acceptance of data-driven and quantitative research in the field.
Panel H.19, “Collecting, Analyzing, and Talking about Data,” presented attendees with both advice on how to conduct data-driven research in composition and why such research is useful and appropriate in writing research. First, Jason Swarts of North Carolina State University spoke cogently about the Methods section in his presentation “The Order of Method: How the Messy World of Writers Becomes Tidy.” Swarts’s talk had several useful aspects: he defined the purpose and scope of the Methods section, presented some empirical research (and nifty data visualization) on Methods sections, and argued convincingly for why rhetoric and composition should more fully adopt the conventions of the Methods section that are typical of other fields. As someone who has long read and written Methods sections from a variety of fields and journals, Swarts’s presentation was something of a review for me, yet I found it valuable nonetheless. For one thing, having the reasoning and philosophy behind the Methods section articulated helped me to consider it with fresh eyes. When we become habituated to the conventions of certain types of academic writing, we can lose sight of their purpose. A fresh argument for their use, like Swarts’s, can aid those who would like to break from conventionality and formula in their production. And for those just embarking on this type of research, a cogent introduction can be invaluable.
In the second presentation, “Stats a Good Idea: A Brief Introduction to Useful Statistics for Mixed-Methods Research,” Jo Mackiewicz of Auburn University discussed some of the ways in which statistics can be incorporated into rhetoric and composition scholarship and provided a simple introduction (or refresher) on basic statistics like the mean, median, mode, standard deviation, and correlation. Given the format, Mackiewicz could not go into great depth about any of these, but her demonstration was a fairly effective primer for those coming new to these statistics. For a quantitative researcher such as myself, this was a review, but then this presentation wasn’t for someone like me. For its intended audience, someone without a background in numbers-based scholarship, this introduction was likely welcomed, particularly given how these techniques can appear obscure and intimidating.
In “What to Do with What You’ve Got: Representing Research,” Rebecca Rickly of Texas Tech University demonstrated both types of data visualization and common pitfalls that researchers often fall into when presenting their data. She displayed various kinds of visualizations like graphs, charts, heat maps, and infographics, explaining briefly what they conveyed and how. She also pointed out typical errors or failings in data visualization to avoid. Like Mackiewicz’s presentation, Rickly’s stuck to fairly elementary techniques, and anyone with even a basic grasp of visual rhetoric would likely find the presentation a review. But as with Mackiewicz, there was real value in letting people in on the ground floor in a friendly, charming, and unintimidating way.
Finally, Karen Lunsford presented “Building a Research Tradition.” This was, in my estimation, the most important paper of the panel. In her talk, she discussed the way that replication studies are an enculturated part of the European academic tradition, particularly in applied linguistics, and advocated for a similar commitment to replication in rhetoric and composition. She spoke specifically about the need for conferences and journals to demonstrate a commitment to accepting this type of research. As Lunsford hinted, methodological changes must ultimately be accompanied by changes in professionalization. Behaviors, intellectual or otherwise, are repeated when they are rewarded, and in the academy, the reward that matters most is publication. As Lunsford pointed out, the lack of replication studies in a field like rhetoric and composition becomes a self-sustaining phenomenon. Because so few are published, editors and reviewers lack an understanding of how to assess and edit them, making it less likely that they will be published. This in turn contributes to the perception that replication studies are not a part of rhetoric and composition scholarship, and the cycle reinforces itself. It will take the work of scholars like Lunsford to create a cultural and disciplinary expectation that replication is necessary and valued in our field.
Given that there were four presentations, there was limited time in the panel for questions, which disappointed the large crowd—one of the largest I’ve seen in my three trips to CCCC. However, one questioner did get at a prerequisite question for this type of panel: why should scholars in rhetoric and composition undertake numbers-based inquiry and social scientific scholarship at all? What’s the point? Swarts fielded the question, and provided a cogent answer. He justified his focus on the Methods section specifically and the broader theme of the panel generally as a kind of responsibility or accountability. As Swarts said, even those who are resistant to these methods in writing studies often use terms like “most” or “many.” To Swarts—and to me—using numbers and precisely defined methodology is a way to make these claims responsible and transparent. Statistics and quantitative methodologies are often represented as opaque or elitist by critics, but in fact, at these methods’ best, they are a way to open up research to others. By making both methods and findings available in specific terms, quantitative research can entail the kind of transparency and accountability that many researchers strive for.
This panel was a friendly, happy affair, with a number of attendees eager to ask questions. But there is little question that the larger methodological and epistemological issues contained within it are still areas of considerable controversy. There were many panels at the conference where this type of knowledge would have been unwelcome, and there are prominent figures within rhetoric and composition who are still deeply resistant to empiricism and numbers-based scholarship. I myself have often found CCCC to be a lonely place, owing to the sense in which quantitative and social science work is frequently seen as a political, theoretical, or disciplinary threat. But that impression is thawing, and thawing quickly, as our field responds to new institutional and economic pressures with pragmatism and adaptability. Ultimately, the key to widening our methodological horizons and incorporating more of this kind of scholarship will be to reinvest ourselves in the ecumenicism and diversity that have always been a part of rhetoric and composition. These methods and methodologies are not replacements, but rather additions, to the type of scholarship we now utilize. Viewed in that spirit, this panel and others like it are an invitation to a new kind of conversation.
Charney, Davida. (1996). Empiricism is not a four-letter word. College Composition and Communication, 47(4), 567-93.
Eliot, Norbert and Les Perelman. (2005). Writing Assessment in the 21st Century: Essays in Honor of Edward M. White. New York: Hampton Press.
Haswell, Richard. (2005). NCTE/CCCC’s recent war on scholarship. Written Communication, 22(2), 198-223.
MacDonald, Susan Peck. (2007). The erasure of language. College Composition and Communication, 58(4), 585-625.
Yancy, Kathleen Blake. (Ed.) (2012). Research methodologies [Special Issue]. College Composition and Communication, 64(1).