A.03: The Messy and Complex Middle Ground Between Method and Methodology: A Conversation on Research

Reviewed by Matthew Zajic, University of California, Davis (mczajic@ucdavis.edu)

Facilitators: Jennifer Clary-Lemon, University of Winnipeg
Derek Mueller, Eastern Michigan University
Kate Pantelides, Middle Tennessee State University
Thomas Passwater, Eastern Michigan University
Jana Rosinski, Syracuse University

Seeing a think tank session on research methods as one of the first sessions of CCCC was an exciting discovery. Attendees sat around numerous circular tables in a large ballroom while the facilitators began laying out what this session would entail. With presentations, small-group discussions, and a collaborative Google Document, we were in for a lively discussion about methods and methodologies.

Facilitator Discussion Points

Each facilitator began with a short (5-10 minute) presentation introducing themselves and their questions to the audience. Kate Pantelides began with her research on methodology sections of dissertations and posed questions targeting how, where, when, and with whom we learn our methods and methodologies. Jana Rosinski followed with a focus on the use of human bodies as material collected through self-quantification instruments, and she posed questions speaking to our own limitations with the research methods that we have come to privilege. Jennifer Clary-Lemon then segued into chorography (or the rhetoric of invention concerned with the history of place in relation to memory) and her work on interview-based oral history studies; her question focused on what exactly emergent methods and methodologies look like to us. Then Thomas Passwater introduced his work on new ways to visualize queer rhetorical work, emphasizing the phenomenological friction between disciplinary network nodes as spaces for queer disorientation, reorientation, and resistance; his questions focused on the relationship between troubling disciplinary assumptions and holding disciplinary footing. Finally, Derek Mueller emphasized his work on drawing from visual models and abstract representations as ways to think about how scholars can use methodologies to visualize disciplinary knowledge; with his work grounded in offering alternative methodologies, his questions focused on the adoption of new methodologies by asking us to think about the tradeoffs between continued development and elaboration with uptake and adoption.

Groupwork in the Google Document

These questions would echo with the audience as we turned to the shared Google Document to think a little bit about methods and methodologies in our own work. With a background in quantitative methodology (psychometrics and statistics), I focused on the need to talk about quantitative methods and methodologies, based on the fact that no facilitators mentioned any. The discussions up until this point had been heavily qualitatively situated, and I was curious to hear from others about their own perceptions and beliefs about the lack of quantitative methodologies in the composition field (and whether the reason for this was a perception towards quantitative methods, a pedagogical or training concern within our disciplinary silos, or something else entirely). Looking back at the Google Document, my questions came out as follows:

Can there be further room for discussion of quantitative analyses as a method for writing researchers/instructors, and where do quantitative frameworks fit into our discussion of research methods?
What can we learn from advanced quantitative methods that might contribute to our own voices as writing researchers, and how can our understandings of our projects help inform the ways we might use such methods?
What contributes to this pushback on quantitative methods as a perspective we draw from to enhance our own sense of methodologies?

What ensued at our table was a lively discussion about the role of quantitative methodologies and the roles of disciplinary experts within and across our fields. Among the six of us, we focused on the needed role of collaboration within composition studies, pointing out that the current structure of CCCC makes collaboration difficult (as everything appears to be single-author submissions); it may not be that composition experts must also be experts at methods foreign to their training, but they may instead need to be valued in their collaborations in order to make use of these methodologies. Training in methodologies also became a big issue for us, as the concern about what methodologies are offered, valued, or taught within our own programs can quickly distort methodological perceptions. We also discussed the messiness of the term methods versus methodologies, coming to the consensus on three categories—qualitative, quantitative, and mixed-methods—under which specific methodological practices fall (such as interviews or ethnographies for qualitative and multiple regressions and structural equation modeling for quantitative). We stressed the importance that the conception of the study must occur before the discussion of the methods, as the methodologies used within a study should be reflective of the goals of the study; when only qualitative methods are used, then scholars are designing detailed studies with poorly-fit methodologies that affect their findings and interpretations. Methodological training in both breadth and depth can be difficult to achieve in composition programs, a recognition that warrants further investigation into how further collaborations can be made to open further methods and methodologies into composition studies. Our takeaway emphasis echoed these major points: “The role of collaboration with disciplinary experts to consider the breadth of both qualitative and quantitative methods available to answer the questions that matter. Our engagement of methods is tied to our material conditions (which may be limited by our disciplinary discourse communities). Embrace the changes in method that may occur when all the data are collected.”

Takeaways from the Other Tables

While I did not get to hear the discussions at the other tables, I can offer their takeaway points from the Google Document. One table focused on the issues of messiness when it comes to both methods and methodologies. The second table focused on how the scale, system, and institutional constraints influence the process, time frame, and product around the methods and methodologies chosen by researchers. The third table focused on the secrecy of methods and methodologies, calling to attention the potential openness or closedness of one’s chosen methods or methodologies alongside their transparency of lack of transparency in their methods or methodologies. The fourth table focused on interests in new methods (grounded in critique of current methods), collective expertise to make sense of current methods, opportunities to learn new methodologies, and spaces for mentoring for both faculty and graduate students. The fifth table focused on the distinctions between different methods in composition or rhetoric as well as the potential for quantitative methods in both policy-focused and research-focused endeavors. Taken together, we covered a lot of ground across our six tables with some very common overarching themes revolving around the use and misuse of quantitative methods, the training needed to employ proper methods and methodologies, and the desire to expand beyond the current methods and methodologies in our repertoire.

Reflections and Comments: Resources for Learning about Quantitative Methods

I was amazed by such a lively discussion about quantitative methods at the first session of CCCC 2017. One of the highlights of composition and writing studies is the interdisciplinary nature of our work, though this interdisciplinarity mixed with departmental silos may be a negative factor when it comes to research methods and methodologies. When I think about composition and writing studies, I situate the field as one drawing from education, English, linguistics, and rhetoric programs (amongst others which I am sure I am forgetting). I come from a School of Education which offers a range of qualitative and quantitative methods training, and I have been fortunate enough to stumble into the areas of psychometrics, assessment, measurement, and statistics. However, the methodologies offered in these areas are not always offered in more traditional composition or writing studies programs, a fact that was echoed numerous times by both table and workshop members.

I view the need for methods collaboration as an important one to the composition field, and there already exist many great resources out there for junior and senior scholars to learn more. Talk with your local education or psychology programs, as they already might be offering workshops or classes on the methodologies that your students or your own work might benefit from. Find both online and offline resources for additional methods or methodological training; two great online resources are SAGE Research Methods (http://methods.sagepub.com/) and Laerd Statistics (https://statistics.laerd.com/). SAGE Research Methods offers a plethora of resources regarding research methods that allow you to dive deep into various methodologies. Laerd Statistics is geared towards helping you understand not only what statistical tests to conduct given the data you have, but also how to make sense of critical statistical assumptions and how to interpret your data. Also, great books are available for offline browsing, such as:

Vogt, W. Paul, Gardner, Dianne C., and Haeffele, Lynne M. (2012), When to use what research design: The authors go over in deep (but jargon-free) detail how to select different research designs for the type of study you are trying to conduct.
Vogt, W. Paul, Vogt, Elaine R., Gardner, Dianne C., and Haeffele, Lynne M. (2014), Selecting the right analyses for your data: Quantitative, qualitative, and mixed methods: Almost a companion to the above, the authors walk you through how to select the types of analyses you should use for the data you have or are going to collect. Written in an easily accessible format, this book helps identify the various methodologies you have at your disposal.
Vogt, W. Paul and Johnson, R. Burke (2015), The SAGE dictionary of statistics & methodology: A nontechnical guide for the social sciences: The authors have compiled a comprehensive, exhaustive, and non-technical introduction to any statistical term you could have ever wanted to know about. Want to know what a p value actually represents? How to determine if your data are nominal, ordinal, interval, or ratio? Want to determine the difference between an exploratory and a confirmatory factor analysis? This is a great resource to have to help everyone feel more confident with complicated statistical jargon.
Field, Andy (2013), Discovering statistics using IBM SPSS statistics (and sex and drugs and rock 'n' roll): Besides having the best title and hundreds of cat pictures, this comprehensive guide walks you through how to not only do various statistical tests in SPSS, but also how to check for statistical assumptions and make sense of the outputs. This is a great guide and reference to not only the why but the how behind numerous statistical tests.

With these resources, I hope you can make use of any methodology you have at your disposal for the types of questions you seek to answer. This think tank session at Cs opened a needed discussion about the role of methods and methodologies in composition and writing studies, and I hope this conversation continues for years to come regarding the role of methods and methodologies amongst writing scholars.

References

Field, Andy P. (2013). Discovering statistics using IBM SPSS statistics (and sex and drugs and rock 'n' roll) (4th ed.). Los Angeles: SAGE.

Vogt, W. Paul, Gardner, Dianne C., & Haeffele, Lynne M. (2012). When to use what research design. New York: Guilford Press.

Vogt, W. Paul, & Johnson, R. Burke (2015). The SAGE dictionary of statistics & methodology: A nontechnical guide for the social sciences (5th ed.). Los Angeles: SAGE.

Vogt, W. Paul, Vogt, Elaine R., Gardner, Dianne C., & Haeffele, Lynne M. (2014). Selecting the right analyses for your data: Quantitative, qualitative, and mixed methods. New York: The Guilford Press.