Transforming Writing Rubrics:

Assessment and Reflection in Process-Based Courses

Contributor: Elkie Burnside
Affiliation: Independent Scholar
Email: drelkieb at gmail.com
Published: Issue 29.1 (August 2024)
DOI: 10.7940/M329.1.PRAXISWIKI.BURNSIDE

Introduction

The use of assessment rubrics is a common topic for those training and specializing in the study of writing. As noted by Eric D. Turley and Chris W. Gallagher (2008), this approach to writing assessment is not the use of a "'merely' neutral tool" and should be employed in a way that is meaningful in the context of the course in which it is used (p. 87).  As a result, rhetoric and composition scholars have developed a broad foundation of theoretical and practical research while working to understand the use of rubrics for both classroom and programmatic writing assessment. As Chanon Adsanatham (2012) explained, when selecting a rubric process there are many models and approaches to draw on, ranging from but not limited to communal assessment, feminist approaches, and instructive evaluation. The approach explored in my project incorporates considerations from this range of scholarship on the use of rubrics, including the use of a writer's checklist, in a writing classroom.

The focus of this wiki entry is to share how I incorporated some of the rubric assessment strategies advocated for by a variety of scholars. For example, the scaffolded method of starting with a fully developed rubric for early projects and then asking students to provide more information as the class moves forward emulates some aspects of Asao B. Inoue's (2005) early work on community-based assessment and L. Dee Fink's (2003) iterative development process. Ultimately moving to the more fully, collaboratively developed rubric process explained in detail below demonstrates practices explored by Turley and Gallagher (2008). Specifically, I worked toward focusing on a rubric which is developed collaboratively with student input and is created after students have drafted a version of the project.

Another focus of my rubric development process works to incorporate the metacognitive reflective aspects advocated for by Kathleen Blake Yancey, Liane Robertson, and Kara Taczak (2014). These authors advocated for assessment practices which "build in metacognition, verbal and visual, balancing big picture and small practices" (p. 139). By combining the considerations of guided development and self-reflection, my goal was to develop an assessment method which could help students connect to building block skills provided through the course. Additionally, I also wanted to provide an opportunity for students to take these skills to other composing environments.

The process described below also works to incorporate clear evaluative criteria (Soles, 2001) and a clear structure for course activities (Fink, 2003). My process to collaboratively develop assessment rubrics with my students is an attempt to address Diane Penrod's (2005) call to understand that "writing standards adapt to shifts in both technology and culture. Standards are fluid and are formed by habituated practices that become internalized as ingrained representations" (p. 162). By considering how to incorporate student feedback on what they understand and how they value genre considerations, this approach may offer students a stronger opportunity to learn how to be comfortable with understanding fluid and (sometimes it feels like) ever-changing expectations for composing.

The activities examined in this project are part of a two-course general education writing requirement sequence at a mid-sized, private religious (Christian, Protestant) liberal arts college in northwestern Ohio (refer to further context in the following section). ENGL 282: Introduction to Writing for the Sciences is one of six courses offered as a second writing requirement option (taken upon successful completion of first-year writing). The class meets the common writing outcomes shared among these courses, while at the same time focusing on specific genres required in science writing (both for general public and specialist audiences).

Students in this course are typically at the freshman or sophomore level and are ideally in a field related to the sciences (including social sciences). This student population accounts for roughly 65% (based on 2019–2020 graduation rate, National Center for Education Statistics, 2020) of undergraduates at the institution and the course is the only course focused on composition that they will take during their undergraduate degree that relates to their field. The student cap for these courses was 18 per section, and I taught between two and three sections a semester (in addition to other courses).

Course and Participant Context

The undergraduate enrollment at the predominantly white institution (PWI) I was teaching at when developing this rubric approach averages a little over 3,500 students an academic year. These undergraduates significantly identify as female (67%, National Center for Education Statistics, 2020) and are drawn largely from northern, midwestern areas in Ohio and surrounding states. Students typically come from smaller, rural working-class communities and have a moderate-to-conservative religious upbringing.

As a cis-presenting female instructor who grew up in a rural, working-class area with a similar religious background myself, on the surface I had many factors in common with these students. During my time working at this institution, I came out (to both myself and those around me) as queer and actively participated in both university and area community LGBTQIA+ organizations. This act was part of my working to actively interrogate systems I had unconsciously upheld, both personally and professionally, and in several ways lead to the development of this project.

In ENGL 282, as a part of first week introductions, students articulated individual goals for the class. After one or two course introductions, I noticed a trend that many of these goals were focused on approaches that centered on learning how to please the professor or pass the task and get to real classes approaches. As the course went on, students informally shared that they viewed the class as a hurdle to get over rather than as a useful chance to engage in practical training for their profession. This trend became a main impetus behind using the approach described in detail in the next sections. Ultimately, I wanted to provide students with the opportunity to view the course as a way to build a rhetorical toolbox for their professional careers, not simply to pass and move on without taking any knowledge or practices forward with them.

Figure 1 displays a set of word clouds created from the initial goal setting I do with students during our introduction activities on day one of the course. I display these during the first week as they enter the physical classroom space to help them keep in mind that their goals are at the center of the class session. I also re-display these when we do our mid-semester survey to remind all of us of the progress they are making toward these goals. These word clouds are not meant to display research data; instead, they are a visual reminder that is easy to create and display in class.

 

Activity and Purpose

The course assessment process demonstrated here explores ways that student reflection and collaborative development of assessment rubrics—in the form of a writer’s checklist—can help increase their understanding that document requirements will change based on the rhetorical context and composing environments they will encounter as emerging science writers. In the context of this project, the writer's checklist is used in two ways. The first is as a developmental tool during the composing process, as one goal of this process was to frame the variety of composing tasks students would approach throughout the course as rhetorical tasks. This approach would thus begin by helping students to understand that every genre will have the following elements:

• consistent general structure (identified as format/layout)
• content specifics (identified as content/information)
• documentation requirements (identified as citation/references)
• overall quality elements (identified as document quality)

These are formal features that can be counted and quantified—the most commonly understood use of a checklist. The second way the checklists were used, once these were collaboratively developed and discussed as a part of class sessions, was to act as the final rubric for scoring a project. In the process explored in detail below, it should be noted that the process, planning, and rough draft elements of the assignment (including developing the checklists) were weighted at 70–80% of the final assignment score. The final draft submission of each project was the other 20–30%. Through this approach, I hoped to help students understand the process of considering and creating the checklist—identifying those formal features—was a part of the composing process: that even if the formal features were the same among writing situations, how these were completed could, and frequently would, vary. Using the terms checklists and rubrics in this interchangeable way helped students transfer previous experience with writing assessment rubrics into the less familiar context of the genres used in ENGL 282.

This set of activities went through three iterations before I arrived at the approach shown here. In the first iteration, I provided the complete project checklists to students, with no collaborative development nor reflection. I found through my informal end-of-the-semester evaluations and the reflective final exams that students did not "see the point" of the checklists. There was a disconnect between the task and assessment that providing the checklists did not overcome.

In the second iteration, I involved the students in the rubric development process, and we worked in partners and in small and large groups to create each element of the lists. However, students indicated that they would have rather spent the in-class time working on the project rather than the assessment. They also raised objections about doing the list but still getting a lower than desired grade. This feedback resulted in two changes to the process:

• introduction of the quality aspect of the checklist (as shown below)
• seeking a balance between student involvement in checklist creation and time expended for the perceived return

I wanted to maintain the student reflection and involvement in creating the checklist, but I also wanted students to have the in-class work time needed to complete the assignment successfully. The third iteration is the one described below. I used a scaffolding structure to create the writer's checklists and incorporate student involvement, and it seems to achieve the balance I sought. The scaffolding/reflection approach helps students explore how to rhetorically consider composing tasks and the types of multi-purpose composing tools they need, and it encourages them to use this reflective practice in other composing situations. After this change, student reflections (both formative and summative) focused more on the composing process, genre expectations, and rhetorical nature of creating documents for scientific purposes. I took this as a signal that the process was achieving the process goal and moved forward with what is described below.

At the time I was teaching at the institution, there was no formal process of requesting Institutional Review Board (IRB) approval for gathering feedback and student reflections that were part of normal classroom practice. Therefore, all student reflection, feedback, and examples were gathered at the time of teaching with an open-ended approval given by student participants to use contributions in presentations and publications. Student input into individual checklists was synthesized across the multiple sections being taught each semester, so individual indication of content contribution is not clearly delineated, with the exception that student examples are labeled as such when possible. 

Activity Approach and Structure

The concept of a writer's checklist as assessment rubric is commonly used in technical writing instruction, and I theorized that the crossover into the professional writing practices emphasized in ENGL 282 could be useful. Each checklist was developed using the same instructional sequence in each unit using the following steps:

1. Assign composing task
2. Use instructional, planning, and other drafting activities to teach about genre and create formative assessment opportunities to consider areas for further emphasis
3. Create draft of writer's checklist (individually or collaboratively)
4. Project rough draft due
5. Post final checklist and require reflective response
6. Final draft due

Scaffolding the Collaborative Development of Writer's Checklists

This section provides an overview of the scaffolding process for introducing the writer's checklist, using collaborative processes for creating checklists as the course progresses, and the reflective activities used to help students apply this practice in other writing environments. Each assignment and checklist strategy is summarized in a few bullet points, and then a brief narrative provides details about the strategy. The summary provides the assignment strategy and student engagement points for the process. While these examples are based on a semester-long science writing course, it is my hope that the process and prompts offered may help spark ideas for a variety of classes.

Communication between students, as randomly assigned partners and as intentionally created groups, and me as the instructor happened in a variety of ways. In the checklist development sections below, I provide examples of student replies (gathered with permission for use at the time of the course) and my written feedback. These examples were drawn primarily through the discussion boards in Canvas, our learning management system (LMS), and the Speed Grader feedback function. Some examples in the sections below show student-generated input by capturing the classroom whiteboard space after a large, full-class discussion session. In some of these instances, students worked in small groups first and then shared back their reflections and ideas a group at a time during the class session. There are also examples drawn from final exam reflection assignments, which were submissions provided through the LMS as well. This variety of communication approaches was also created intentionally as a scaffolding method, so that students wrote prompted reflections in early assignments and more open-ended and self-guided reflections in final projects.

Checklist 1: Full Checklist Provided

• Assign: In the first week of the class provide the full checklist for diagnostic assignment
• Student Engagement Points: Participation in class discussion about categories and elements; use large group discussion to explore main categories in checklist and introduce quality concept

The first assignment in the course sequence is an introductory diagnostic letter. Distributed during the first week of the course, the goal of this assignment is to introduce students to several core concepts used throughout the course and allow me to see where they are in their writing development. For this assignment, the writer's checklist rubric is provided in full with the assignment. In a large-group, in-class discussion, we look at the rubric and consider the four main categories. The main focus of the discussion is why the documentation section is not required for this first assignment but will be in other projects. Because students come to the course with an assumption that anything related to science means that there is a strict right and wrong approach, we also consider why quality is an element in the writing process during this discussion.

Because students often have this expectation that anything related to science is only quantitative, it means that they often set goals for the course that focus on grades. Students also tend to focus on completing the course only to get on to their "real field," and they additionally hold a variety of other perceptions that tend to work against their ability to view the class as one useful to their future career. It is during this discussion on quality that I introduce the idea of a more rhetorical writing process. We explore how their projects connect to all elements of the writing situation—including their own decisions as an emerging science writer—and not just a right or wrong approach.

The Introduction Letter Writer's Checklist (as a .docx file) is provided here.

Checklist 2: Cocreate Checklist—Partners to Individual Reflection

• Assign: Prior to project rough draft, students work with partner to identify potential elements for the checklist; individually respond to posted project checklist
• Student Engagement Points: Respond with partner to identify elements for project checklist and post to discussion board; individually respond to posted checklist and reflect on strategies needed to meet expectations

The next assignment in the course sequence is a review/critique essay of two to three peer-reviewed articles that follow the Introduction, Method, Results, and Discussion (IMRaD) structure on a topic of each student's choosing. The goal of this assignment is to introduce students to the IMRaD structure and work to practice using a summary/synthesis model of evidence in their writing, rather than the fully direct quotation model they typically rely on during early drafts. The assignment also allows students to more fully apply the writing process. For this assignment students first work with a partner in class to suggest three elements which may be expected in each of the four writer’s checklist categories.

In-Class Prompt

Sample Student Replies

This is the first time students are asked to generate checklist elements and typically replies are fairly basic and focused on strategies clearly developed from understanding a five-paragraph essay approach. During the large group share out, we look at how these expectations meet what we have been talking about during class activities. I also guide the discussion toward content and documentation practices more because that is what we are focusing on during in-class activities in the overall drafting process.

During the next class session, the complete writer’s checklist is posted for the project and students individually reply to a reflection prompt about what elements are on the checklist. By asking them to consider what is or is not on the checklist, students begin to consider how elements in the project may shift depending on their goals for the essay and their target audience. The question about what surprised the student or what they see as missing helps to show where more instruction on process specifics could be helpful.

Reflection Prompt

Sample Student and Instructor Replies

In their reflection posts, students tend to focus on elements that they have never seen listed before or things that might break a rule that they were taught in previous writing situations. In my replies, I encourage them to view the process as a way to expand on what they know and try out new approaches—a type of experimenting in their own writing process.

Having students work with partners to identify project checklist elements helps students who are still getting comfortable with the concept gain confidence that they understand what is important for the process. Additionally, the individual reflection helps me identify students who may benefit from individualized instruction on specific aspects of the essay. Particularly students often wonder why the five-paragraph essay expectation has been removed and they are now allowed more variation in paragraph length and other structures that they were previously taught had to adhere to a specific formula. You will notice that while content elements are listed in the checklist, placement and selection of the content is left up to the student. This approach helps them begin to understand that the goal of their writing task is just as important as getting the information right.

The Review/Critique Writer's Checklist (as a .docx file) is provided here.

Checklist 3: Co-Create Checklist - Project Group to Individual Reflection

• Assign: Prior to project rough draft, students work with group project to identify potential elements for the checklist; individually respond to posted project checklist
• Student Engagement Points: Work in pre-established group to identify elements for project checklists; individually respond to drafting strategies that may be needed to complete projects

The next assignments in the course sequence are related to a project in which students work in a small group of three or four to research a topic of the group's choice, propose a solution to the issue, and then conduct trial research (not requiring Institutional Review Board approval) to assess feasibility of the proposed idea. The goal of this series of projects is to allow students to begin to write in the IMRaD structure, gain more practice writing in a collaborative group, and begin to understand core concepts in the primary research process.

For both the research proposal and the feasibility report, students work in their groups to create a list of elements required for each project's writer's checklist. This approach is similar to the partner created lists from the review/critique essay, but because a small group is working on the list, there are more elements required. Typically, students are responding with longer lists closer to a full writer's checklist because they have started to think about this component as a regular part of their writing process (for this course at least).

In-Class Prompt

Proposal Sample Student Replies

For this project, student replies begin to show their understanding of the variety of elements needed for a longer project and the variety of ways to approach a proposal. By asking for more details on quality elements, the follow up large group discussion also allows for more in-depth discussion of the rhetorical nature of their decisions.

For example, in the introduction section, we explore that really the only element that is required to be in the order listed on the checklist is the solution summary (cost/time and benefit). All of the other elements must be present but can be organized in a way that the group believes is most effective for sharing the need for the solution they are working toward.

Feasibility Report Sample Student Replies

Student responses in this project continue to show how they are developing in their understanding of common genre elements and individual changes that can be made during the drafting process. The large group share-out after this prompt is often focused on the newer sections of this report. The interview and survey results they are working on obtaining.

For at least one group per semester, we discuss how to write up these sections if the information is missing. The ability to write about what information they could have gained but didn't allows these projects to move forward. It also opens the conversation that in scientific inquiry a null or no result can actually also be meaningful. This is another strategy that helps demonstrate the flexibility and rhetorical nature of science writing, by enabling a discussion of the wide variety of reports that may be required in their different fields.

Before the rough draft of each project is due, students individually respond to a reflection prompt that asks them to consider how addressing elements in the checklist will require them to shift drafting strategies or their planned process to be sure to include the information or element. The reflection prompts for the proposal and feasibility report are slightly different as the groups are in different drafting stages in the project overall.

Proposal Reflection Prompt

Proposal Sample Student Replies

Instructor Reply

For these reflections, I look at the trends of questions and present a frequently asked questions (FAQ) session at the beginning of a workshop day (ideally about 10 minutes maximum) and post a document in the digital classroom as a reference. I then refer to this document as I meet individually with the groups during workshop time as well. I do not reply individually like I did in the first checklist assignment, but I do want them to know I am reading their posts.

Feasibility Report Reflection Prompt

Feasibility Report Sample Student Replies

Instructor Reply

For these reflections I also do another FAQ session and document. At this point in the process, student replies also demonstrate which groups are functioning and which may need some intervention. I continue to address these questions during group conferences and may have individual follow up emails or class chats as needed as well.

By shifting the reflection from individual elements to drafting strategies, I am attempting to help students consider their role as author in a more active manner. Their reflection responses help to demonstrate that they are beginning to understand the more open nature of the project—for example, identifying that they do need to have a counter argument/naysayer section in their proposal introduction, but where that is placed and how they include that element will vary based on their project. This openness is also an element which arises in peer review workshops as well.

Students are fairly tentative at applying this approach in drafts, however. They tend to create a rough draft that follows the order we look at the required elements during class sessions. Then, as we work through peer review and conferencing sessions, we can talk more about how to move elements around and shift into deeper revision strategies. However, students do find the fact that they have the information created TO move around a positive in this process.

The Proposal Writer's Checklist and Feasibility Writer's Checklist (as .docx files) are provided here.

Checklist 4: Cocreate Checklist—Large Group

• Assign: Prior to project rough draft, work with class to identify MINIMUM requirements for poster sections; use posted checklist to assess project
• Student Engagement Points: In class discussion to create checklist

The final course project is to create a research poster using one of the Introduction, Method, Results, and Discussion (IMRaD) peer-reviewed articles students used in the review/critique essay from earlier in the semester. The goal of this assignment is to help students consider how to create a multimodal presentation of research for a conference presentation—a common requirement for the capstone courses in each of their majors and other conference opportunities.

For this project checklist students discuss the elements required for a research poster as a large group. Because this unit is often a shorter, end-of-the-semester assignment, we are able to cocreate the writer's checklist as we review the elements of the IMRaD structure and how that can be translated into the poster. The in-class discussions of what is in a poster and how to create a rough draft result in simultaneous visualizations of the writer’s checklist elements.

Figure 2 is an image of a classroom board taken after a group discussion about what elements are required for a research poster. Some elements are noted visually (poster layout, left), some are noted as options (use of square brackets), and other requirements are written as guidelines to consider using (word count, etc., right).

 

Figure 3 is an image of a classroom board taken after a group discussion about how font selection strategies may work for a research poster. Students discuss how accessibility, presentation method, and other aspects of poster design may influence their final decisions. Because students use a variety of programs to create these all-digital posters, this also allows the class to think about ways that circulation and composing tools influence final decisions.

At this point in the semester, students are more adept at considering all the sources of information at their disposal to create the writer’s checklist. The discussion focuses on the assignment, genre expectations of a research poster, understanding a primary and secondary audience, and research-based citation practices for both alphabetic text and visuals. By considering these elements individually in previous checklists, students are now ready to collaboratively create these lists as a part of their composing process.

The Summary Poster Writer's Checklist (as a .docx file) is provided here.

Checklist 5: No Checklist

• Assign: For final reflective exam, optional to create checklist for own use; remind that genre is similar to initial letter project
• Student Engagement Points: Provide final exam reflection letter; conference as requested

The final assignment in the course sequence is to create a letter reflecting on their growth through the course and what their experiences have taught them about writing and drafting. The letter requests that they consider how specific skills were developed and may help them in their future writing projects. No checklist is provided to help each emerging science writer decide what approach is the best to use for themselves. It has happened once or twice that students will set up individual conferences and show me their own created checklists; however, none of them have shared these with me in a way that I still have access to the document. In other instances, students will observe that the basics of the initial letter writer’s checklist could also be used as a foundation or guide for this letter.

Figure 4 is a set of word clouds created from a set of final reflections. I created these to compare how students may change their conceptions of themselves as writers over the term of the class. These word clouds are not meant to display data; they are just a visual reminder that is easy to create and display in class.

In these final exam reflections, most students consistently display a greater understanding of the composing process and articulate this growth in the ways they discuss themselves as writers and how they are learning about the process of writing projects. I use specific examples from these reflections in the following reflection section to consider how students conceptualize this growth in their own words.

Reflection in Practice

Ultimately, the scaffolding approach to cocreating each project writer's checklist does appear to meet the goals I originally set out to reach. It provides the students with a chance to view a general education course—often viewed as a hurdle to get over—as a meaningful learning experience for their science based professional practices. I also believe this strategy does provide students with the opportunity to view science (and scientific) writing as a rhetorical act, one in which they can have an active role as author.

However, as with any instructional endeavor, not everyone will be happy with the approach. For example, some students have requested that the full writer's checklist be provided before the rough draft is due. These excerpts highlight why students may want to see a more specific list as they are drafting; even as both note they still feel the course provided them an opportunity for success.

Student Final Reflection Excerpt 1

Student Final Reflection Excerpt 2

As part of the course’s assessment approach, I explain to students that points for the rough draft are earned on a completion basis—full credit if they attempt all of the required elements (accuracy is not required). Once I started using this overall assessment strategy, these requests for a checklist have stopped. I also am now at a place in my teaching where I would just provide a checklist to any student who asks for it in this way.

Compiling this project for sharing has allowed me to reflect on how much my own approaches to assessment in the classroom have changed. In particular, in the 4 years that I developed this writer's checklist process, I have learned so much more about different approaches to assessment (both on a project-by-project and course basis). Continually asking myself what elements are in my classroom because they need to be present or what exists out of habit or pre-existing structure has been a challenging approach to teaching. However, being able to see clear growth for both myself and my students urges me to continue asking this question.

In my reflections, I see that one element that helps make this rubric approach successful is making the collaborative creation of the writer's checklists meaningful to students. I learned in earlier iterations that simply asking the student to post responses and reflections was not enough. I had to find a way to bring their replies into class conversations and to demonstrate that their labor in these activities was actually used in the course.

Another element that helps make this approach successful is using the scaffolding approach to how much students need to generate for creating the checklists. This is achieved by varying from a fully provided model checklist, to partner and group created predictions of what the checklist will contain, to fully collaboratively creating a checklist for the final summary poster project.

Below are a few final student reflection excerpts that I believe demonstrate this success.

Student Final Reflection Excerpt 3

This student remained in touch after the course ended and came back for help in developing her (ultimately successful) veterinary school applications.
 

Student Final Reflection Excerpt 4

Student Final Reflection Excerpt 5

This student used the project proposal developed for the class to win an on campus entrepreneur contest and actually started the service dog training club on the campus. She and the group members came back to me after the class was over for advice on how to revise their work for contest submission. The campus club they created is still operating, eight years after it began and both the student and I had moved on from the institution.
 

There are many, many more examples like this where students leave the course being able to clearly articulate what they have learned and how they will bring the skills forward into their professional practice. The iterative, slowly built rubric creation process takes time for students to become accustomed to—but it is one that helps build core skills and see improvement for themselves—a goal I believe is worth pursuing.

References

Adsanatham, Chanon. (2012). Integrating assessment and instruction: Using student-generated grading criteria to evaluate multimodal digital projects. Computers and Composition, 29(2), 152–174. http://dx.doi.org/10.1016/j.compcom.2012.04.002

Fink, L. Dee. (2003). Creating significant learning experiences: An integrated approach to designing college courses. Jossey-Bass.

Inoue, Asao B. (2005). Community-based assessment pedagogy. Assessing Writing, 9(3), 208–238. https://doi.org/10.1016/j.asw.2004.12.001

National Center for Educational Statistics. (2020). University of Findlay. United States Department of Education. https://nces.ed.gov/collegenavigator/?id=202763

Penrod, Diane. (2005). Composition in convergence: The impact of new media on writing assessment. Routledge.

Soles, Derek. (2001). Grading as a teaching strategy. Teaching English in the Two-Year College, 29(1), 122–134.

Turley, Eric D., & Gallagher, Chris W. (2008). On the "uses" of rubrics: Reframing the great rubric debate. The English Journal, 97(4), 87–92.

Yancey, Kathleen Blake, Robertson, Liane, & Taczak, Kara. (2014). Writing across contexts: Transfer, composition, and sites of writing. Utah State University Press.