Loading...
 

Resources for Evaluating Digital Scholarship

Annotated Bibliography on Evaluating Digital Scholarship for Tenure & Promotion

Compiled by Cheryl E. Ball, Carrie A. Lamanna, Craig Saper, and Michael Day. If you have resources you would like to add to this list, please contact Douglas Eyman. See also the bibliography compiled in the 2012 issue of the Journal of Digital Humanities, which has a more DH-focused bent to it (while this list below has a more rhetorical bent to it): http://journalofdigitalhumanities.org/1-4/bibliography/(external link)

Digital Scholarship in Action

Scholarship that enacts the digital theories and principles it espouses.

Kuhn, Virginia, and Vitanza, Victor (Eds.). (2008). The components of scholarly multimedia. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 12(3):  http://kairos.technorhetoric.net/12.3/topoi/gallery/index.html(external link)

This collection of eight multimodal pieces is a curated version of a 2006 CCCC’s panel presentation. It aims to make audiences rethink what scholarship is and how we should engage with it: “Our goal remains not only to rethink what it means to make a presentation at our conferences but to challenge what it means to write in a digital age and complicate traditional academic practices (the conference paper, the journal article). As such, typical reading and listening protocols will not be of service as conferees negotiate these works; just as we engaged our bodies differently at the CCCC, moving among our audiencefrom elbows akimbo to touchingsour viewers-readers-listeners here online will be called upon to work in and with the texts to enhance the processes of making sense and meaning. To opening themselves to being touched by the works.”

Purdy, James P., & Walker, Joyce R. (2012). Scholarship on the move: A rhetorical analysis of scholarly activity in digital spaces. In Debra Journet, Cheryl E. Ball, & Ryan Trauman (Eds.), The new work of composing. Computers and Composition Digital Press/Utah State University Press. http://ccdigitalpress.org/nwc/chapters/purdy-walker/(external link)

This chapter rhetorically analyzes texts from different digital venues, including webtexts, blogs, Techrhet, and Twitter, to illustrate how moves that define traditional scholarship can also define digital scholarship, scholarly activity happens in new ways in these digital spaces, and digital spaces serve as a direct outlet to formal scholarly productions. They enact their theory that digital spaces allow for new forms of scholarly activity by employing some digital technologies, especially Prezi, in their chapter. (cross-listed in Evaluation Strategies)

Walker, Joyce. (2006). Hyper.activity. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 10(2): http://kairos.technorhetoric.net/10.2/binder2.html?coverweb/walker/index.html(external link)

Hyper.Activity is a web text that began as a conference presentation that began as a course paper that began as a simple idea: that composing a "new media" text is a completely different experience than writing a conventional paper. Both the content of this webtext and its visual design are an attempt to think about this issue and to consider the ways in which reading/writing processes for new media texts remain incompatible with the expectations of readers and composers of traditional scholarly work.

Taxonomies

Scholarship that discusses genres, categories and types of scholarship.

Ball, Cheryl E. (2004). Show, not tell: The value of new media scholarship. Computers and Composition 21: 403-425.

The article examines the changing nature of publications in relation to technology and tenure and presents a taxonomy of scholarly publications: online scholarship, scholarship about new media, and new media scholarship. It defines new media texts as ones that juxtapose semiotic modes in new and aesthetically pleasing ways and, in doing so, break away from print traditions so that written text is not the primary rhetorical means. By applying this definition to scholarly online publications, readers can be better prepared to recognize and interpret the meaning-making potential of aesthetic modes used in new media scholarly texts.

Barber, John F. (1997). The seven ages of computer connectivity. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder.html?features/barber/ages_intro.html(external link)

Brent, Doug. (1997). Rhetorics of the web: Implications for teachers of literacy. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1).: http://kairos.technorhetoric.net/2.1/binder.html?features/brent/wayin.html(external link)

Krause, Steven D. (2007). Where do I list this on my CV? Considering the value of self-published websites - version 2.0. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 12(1): http://kairos.technorhetoric.net/12.1/binder.html?topoi/krause/index.html(external link)

This is an updated version of Krause’s 2002 article in CCC Online. He examines several examples of self-published websites in addition to the four discussed in the original article, and he adds a discussion of blogs. Krause describes his central argument as follows: “I believe self-published Web sites have the potential to rise to the level of Scholarship (with a capital S) in the abstract sense of advancing knowledge in a dynamic, evolving, and exciting way, and I believe these sites should be rewarded by institutions as scholarship (with a small s) in the more pragmatic and tangible sense of how they count on a CV. The question is how, since, by definition, these new forms fall outside the traditional mechanisms of production, dissemination, and evaluation. Given the high value that most institutions put on scholarship that appears in refereed journals or in books produced by well-respected presses, how are innovative, intellectually valuable, well-researched, self-published Web sites to be counted in the processes of promotion, merit, tenure, review, and recognition?”

Logue, Alexandra W. (2009, February 2). The scholarship of administration. Inside Higher Ed. Retrieved from http://www.insidehighered.com/views/2009/02/02/logue(external link)

Louge argues for what she calls “the scholarship of administration, using a scholarly approach in performing higher education administration.” He provides several hypothetical and real examples of such an administrative approach and its value, but does not address how this new scholarship of administration should be judged by evaluating bodies such as tenure and promotion committees.

Nahrwold, Cindy. (1997). 'Just' professing: A call for the valuation of electronic scholarship. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/nahrwold/kairos.html(external link)

Nahrwold uses the prototype as a metaphor for the open-ended, collaborative, and boundary-crossing nature of digital scholarship. She argues that such scholarship “can perform, through their cyclical dialogue, the theories of ‘writing as process’ and ‘social construction of knowledge’ that inform current pedagogical practices,” and in turn argues such experimental work is essential to the knowledge construction. (cross-listed in Special Issues)

Saper, Craig. (2008). Toward a visceral scholarship online: Folkvine.org and hypermedia ethnography.” Journal of E-Media Studies 1(1): 1-26.

Warner, Allison. (2007). Constructing a tool for assessing scholarly webtexts. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 12(1): http://kairos.technorhetoric.net/12.1/binder.html?topoi/warner/index.html(external link)

This webtext presents a tool for assessing the scholarly value of online journal publications. It is part of a larger study that uses Kairos webtexts to investigate the scholarly nature of online texts. The goal of this larger study is to deliver a rubric as an instrument to facilitate the acceptance of online texts within English Studies as evidence of scholarship for professional advancement. The assessment tool presented in this webtext is comprised of questions that help to reveal commonalities and deviations in the function and value of traditional (print) scholarly conventions toward defining an emerging genre of online scholarship. (cross-listed in Evaluation Strategies)

Evaluation

Strategies Scholarship about how to evaluate digital scholarship.

Anderson, Steve & McPherson, Tara. (2011). Engaging digital scholarship: Thoughts on evaluating multimedia scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.136(external link)

This article offers a critical survey of contemporary institutional struggles to come to terms with multimedia scholarship. Building on their work as coeditors of the online journal Vectors, the authors put forward requirements that are essential to the future of emerging scholarship: respect for experimentation and emerging genres, appreciation for transdisciplinary and collaborative work, the updating of models of citation and peer review, rewards for openness and contribution to a public commons, and valuing the development of tools and infrastructure. (cross-listed in Special Issues)

Braun, Catherine. (2013). Cultivating ecologies for digital media work. Carbondale, IL: Southern Illinois University Press.

Braun’s in-depth study documents English professors and the challenges they face in both career and classroom as they attempt to gain appropriate value for digital teaching and creation within their field, departments, and institutions. Braun proposes that to move English studies into the future, three main questions must be addressed. First, what counts as a text? How should we approach the reading of texts? Finally, how should we approach the production of texts? In addition to reconsidering the nature of texts in English studies, she calls for crucial changes in higher-education institutional procedures themselves, including new methods of evaluating digital scholarship on an even playing field with other forms of work during the processes for promotion and tenure.

Braun, Catherine C., & Gilbert, Kenneth L. (2008). This is scholarship. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 12(3): http://kairos.technorhetoric.net/12.3/topoi/braun_gilbert/index.html(external link)

“This Is Scholarship” takes the 2006 findings of the Modern Language Association’s task force on tenure and promotion in the U.S. as its starting point. While the findings argue a redefinition of scholarship is necessary, the task force leaves the specifics of re-defining scholarship and creating new evaluation procedures open to interpretation in local contexts. However, the document does provide examples of online publications that might represent an expanded conception of scholarship and help guide departments in rethinking their evaluation procedures and criteria. This webtext is a movie that attempts to fill this gap The movie provides examples and argues existing online scholarship is multiple and varied enough already that a more flexible definition of scholarship has emerged, even if it is not yet valued by many tenure-granting units. The movie then explains how these examples can guide tenure and promotion committees in creating new evaluation procedures and offers suggestions for how to build new values into tenure and promotion policies.

Cross, Janet, & Fuglevik, Kristian. (1997). Jesters get serious. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/candf/cvr.html(external link)

This webtext studies academic MOOs to develop a new model for tenure and promotion. Cross and Fuglevik advocate for a model the better represents the interconnectedness of research, service, and teaching, arguing that the traditional model attempts to artificially compartmentalize and quantify the three. (cross-listed in Special Issues)

Fitzpatrick, Kathleen. (2011). Peer review, judgment, and reading. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.196(external link)

This article argues that evaluating new forms of scholarship for tenure and promotion requires taking those forms, and the methods of peer review they bring with them, on their own terms. Even more, it requires exercising the critical judgment on which our profession relies instead of outsourcing that judgment to others. Such evaluation requires reading both the work and the available evidence of the ways that scholars have responded to that work. (cross-listed in Peer Review and Special Issues)

Nowviskie, Bethany. (2011). Where credit is due: Preconditions for the evaluation of collaborative digital scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.169(external link)

According to Nowviskie, when assessing digital humanities scholarship for purposes of tenure and promotion, committees must focus as much on process as on product, because digital work is situated in especially complex and collaborative networks of production and reception. These changes in evaluation require a rethinking of notions of solitary authorship, development of new standards for attribution, and a revising of institutional policies that govern intellectual property. The article argues that fair and full acknowledgment of the work of others (including non–faculty members and alternative academic contributors) will lead to a system in which new work in the humanities is better fostered, designed, distributed, and preserved. (cross-listed in Collaboration and Special Issues)

Purdy, James P., & Walker, Joyce R. (2012). Scholarship on the move: A rhetorical analysis of scholarly activity in digital spaces. In Debra Journet, Cheryl E. Ball, & Ryan Trauman (Eds.), The new work of composing. Computers and Composition Digital Press/Utah State University Press. http://ccdigitalpress.org/nwc/chapters/purdy-walker/(external link)

This chapter rhetorically analyzes texts from different digital venues, including webtexts, blogs, Techrhet, and Twitter, to illustrate how moves that define traditional scholarship can also define digital scholarship, scholarly activity happens in new ways in these digital spaces, and digital spaces serve as a direct outlet to formal scholarly productions. They enact their theory that digital spaces allow for new forms of scholarly activity by employing some digital technologies, especially Prezi, in their chapter. (cross-listed in Digital Scholarship in Action)

Purdy, James P., & Walker, Joyce R. (2010). Valuing digital scholarship: Exploring the changing realities of intellectual work. Profession, 177-195. Retrieved from http://susandelagrange.com/cccc/PurdyWalker.pdf(external link)

Purdy and Walker argue that attempts to argue for the value of digital scholarship by individuals and organizations such as the MLA and CCCC have unintentionally reaffirmed the dominance of print culture, by “focusing primarily on establishing digital work as equivalent to print publications to make it count instead of considering how digital scholarship might transform knowledge-making practices” (178). They go on to call for a new approach to evaluating both print and digital scholarship that considers them “in relation to larger, more systemic issues regarding the nature and value of various kinds of scholarly work: design and delivery, recentness and relevance, and authorship and accessibility” (179).

Rockwell, Geoffrey. (2011). On the Evaluation of Digital Media as Scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.152(external link)

Rockwell argues we need to develop a culture of conversation around the evaluation of digital academic work because where there is a conversation around evaluation in a department, both hires and evaluators are more likely to come to consensus as to what is appropriate digital research and how it should be documented. The article surveys common types of digital scholarly work, discusses what evaluators should ask, discusses how digital researchers can document their scholarship, and then discusses the types of conversations hires and evaluators (like chairs) should have and when they should have them. (cross-listed in Special Issues)

Schatzki, Theodore R. (2009). Digital research and tenure & promotion in colleges of arts and sciences: A thought piece. In Timothy W. Luke and Jeremy Hunsinger (Eds.), Putting Knowledge to Work and Letting Information Play: The Center for Digital Discourse and Culture. Blacksburg, VA: Center for Digital Discourse and Culture. 271-289. http://www.cddc.vt.edu/10th-book/(external link)

Walker, Janice R. (1997). Fanning the flames: Tenure and promotion and other role-playing games. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/walker/intro.html(external link)

Warner, Allison. (2007). Constructing a tool for assessing scholarly webtexts. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 12(1): http://kairos.technorhetoric.net/12.1/binder.html?topoi/warner/index.html(external link)

This webtext presents a tool for assessing the scholarly value of online journal publications. It is part of a larger study that uses Kairos webtexts to investigate the scholarly nature of online texts. The goal of this larger study is to deliver a rubric as an instrument to facilitate the acceptance of online texts within English Studies as evidence of scholarship for professional advancement. The assessment tool presented in this webtext is comprised of questions that help to reveal commonalities and deviations in the function and value of traditional (print) scholarly conventions toward defining an emerging genre of online scholarship. (cross-listed in Taxonomies)

Digital Tenure Examples

Links to digital portfolios, arguments for digital works, narratives of digital T&P practices, etc.

Amy, Lori, & Crow, Angela. (2000). Shaping the imaginary domain: strategies for tenure and promotion at one institution, Computers and Composition 17(1): 57-68. http://dx.doi.org/10.1016/S8755-4615(99)00030-4(external link)

This article examines one department in the midst of establishing an independent writing major and program. The local situation is analyzed in the context of professional concerns regarding tenure and of disciplinary concerns regarding the recognition of computers and composition labor. The possibility to shape, to some degree, the guidelines for tenure and promotion, enables the authors to explore potential strategies for recognizing alternative forms of labor within and against tenure traditions. (cross-listed in Special Issues)

7Cs Subcommittee. (2001). Tenure and Promotion Cases for Composition Faculty Who Work with Technology. Retrieved January 28, 2015, from  http://www.ncte.org/cccc/committees/7cs/tenurepromotioncases(external link)

This archived site outlines five fictional tenure and promotion cases of composition faculty who work with computer technology—addressing their contributions in the area of teaching, scholarship, and service. This site is sponsored by the CCCC Committee on Computers in Composition (the 7 Cs) and compiled by the following sub-committee members in 2001: Cynthia Selfe, Linda Hanson, Gail Hawisher, Victor Villaneuva Jr., & Kathleen Yancey. Although the site and examples are a bit old now (in 2015), they represent one of the first sustained efforts to catalog attitudes and strategies for evaluating digital tenure cases.

Ball, Cheryl E. (2009). Digital Tenure Portfolio submission letter. http://ceball.com/research/tenure-letter/(external link)

This letter (about 4 pages single-spaced) comes from Cheryl Ball’s all-digital tenure portfolio in 2009. In the letter, which served as her T&P application cover letter to the provost of her institution at the time (Illinois State University) narrates why a digital portfolio was necessary to present her heavily multimediated work, what the contexts of that work is within her field, and why the digital portfolio (and its embedded digital scholarship) should be read in their original media. Ball presented the first completely digital portfolio (via a Wordpress site) for tenure at a university. The links are primarily dead, but the content is still useful for scholars looking to provide disciplinary and methodological justification for their digital work.

Ball, Cheryl E. (2009). On a digital tenure portfolio Video. Retrieved January 28, 2015, from  https://www.youtube.com/watch?v=oJJER7diM6c(external link)

This video was composed (in draft form) to show Ball’s dean why her digital scholarship needed to be presented in a digital portfolio (not a print-based one), based on her disciplinary research questions and academic identity as a new media scholar. It accompanies the letter above, and was shown to Ball’s dean over a year before she went up for tenure, to gain permission to use an all-digital portfolio. (His response was: “That’s all nice, but all we really care about is whether your work is peer reviewed.”)

Katz, Seth R. (1997). One department's guidelines for evaluating computer-related work. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/katz/art2.html(external link)

In this webtext Katz recounts his experience as part of a committee to revise his university’s tenure and promotion guidelines to include work with computers. He provides artifacts from the process including the original and revised T&P documents. Of most interest today is that the department’s dilemma in 1997 is still an issue in many English departments today: “We have recognized the reality of our situation: we are caught between tradition and transition, attempting to evaluate a technology and practice with which we have inadequate experience, and which keeps evolving as we watch.” (cross-listed in Special Issues)

Ellison, Julie, & Eatman, Timothy K. (2008). Scholarship in Public: Knowledge Creation and Tenure Policy in the Engaged University. Syracuse, NY: Imagining America. Retrieved from http://artsofcitizenship.umich.edu/documents/TTI_FINAL.pdf(external link)

The Imagining America Tenure Team Initiative (TTI) was formed at the 2005 Imagining America conference, and this report is the result of the surveys conducted by the TTI over a two-year period. The Team’s focus is on revising tenure requirements to support public scholarship in the arts and humanities, which includes expanding definitions of scholarship to include digital work. The report calls for  “a continuum scholarship” that includes “a continuum of scholarly and creative artifacts” that count toward tenure, and it outlines twelve recommendations for making such changes on individual campuses.

Lee, Valerie, & Selfe, Cynthia L. (2008). Our capacious caper: Exposing print-culture bias in departmental tenure documents. ADE Bulletin, 145: 51-58. Retrieved from http://susandelagrange.com/cccc/Lee&Selfe.pdf(external link)

Lee and Selfe narrate two sides of the attempt to revise The Ohio State University English Department’s tenure and promotion guidelines to include language addressing digital scholarship. Lee relates the perspective of the literature faculty from her position as department chair. Selfe presents the view of the department’s eight digital media studies scholars who penned a manifesto addressing the shortcomings of the Executive Committee’s proposed changes to the Department of English Appointments, Promotion, and Tenure (APT) Document. The article provides insight into the (sometimes subconscious) beliefs and values English Studies scholars attach to print-based scholarship.

Davidson, Cathy. (2012, Sept. 17). How Can A Digital Humanist Get Tenure? Hastac blog. Retrieved January 4, 2017, from https://www.hastac.org/blogs/cathy-davidson/2012/09/17/how-can-digital-humanist-get-tenure(external link)

Hoelscher, Karen & Dagnon, Paula. (2012, June 22). Digital Makeover in Your Future? Retrieved January 4, 2017, from https://www.insidehighered.com/advice/2012/06/22/electronic-tenure-portfolio-right-you-essay(external link)
 
Sample, Mark. (2012). Tenure as a Risk-Taking Venture. Journal of Digital Humanities, 1(4). Retrieved January 4, 2017, from http://journalofdigitalhumanities.org/1-4/tenure-as-a-risk-taking-venture-by-mark-sample/(external link)

Organization Reports, Position Statements, and Recommendations

Documents from professional organizations.

AHA Tenure, Promotion, and the Publicly Engaged Academic Historian (2010) https://www.historians.org/jobs-and-professional-development/statements-and-standards-of-the-profession/tenure-promotion-and-the-publicly-engaged-academic-historian(external link)

This statement is the product of the Working Group on Evaluating Public History Scholarship (WGEPHS) convened by the American Historical Association, Organization of American Historians, and National Council on Public History and provides guidelines for history departments and university administrators to use when evaluating the work of scholars hired to be public historians. While it does not specifically address digital scholarship, it acknowledges that the work of public historians takes forms that go beyond the traditional, single-authored monograph, and that this work is peer is often published in non-academic venues and reviewed by historian employed outside universities, by museums or state agencies, for example. In short, it asks tenure and promotion guidelines to take a rhetorical approach and consider audience, purpose, and context when evaluating scholarship.

CCCC Position Statement on Scholarship in Composition: Guidelines for Faculty, Deans, and Department Chairs (1987) http://www.ncte.org/cccc/resources/positions/scholarshipincomp(external link)

This, the earliest of the CCCC’s statements on tenure and promotion, discusses the collaborative and interdisciplinary nature of writing studies scholarship and directly addresses the non-traditional forms such scholarship takes, citing “textbooks, computer software and programs, and curricular development” as legitimate forms of publication.

CCCC Position Statement on Promotion and Tenure Guidelines for Work with Technology (1998; revised 2015) http://www.ncte.org/cccc/resources/positions/promotionandtenure(external link)

The newly updated statement emphasizes that the work of establishing clear tenure and promotion guidelines for digital scholarship begin during the hiring process where both the job candidate and the hiring committee have responsibilities for establishing expectations. These mutual responsibilities continue after the candidate is hired. Evaluation committees must acknowledge that digital technologies have fundamentally changed teaching, service, and scholarship and must educate themselves accordingly. Digital scholars must make connections with faculty across campus who do digital work and can thus be allies, and they must help educate colleagues in their home departments on the value of their digital work.

CELJ Best Practices for Online Journal Editors (2008) http://www.celj.org/files/downloads/CELJEjournalEditorsGuidelines.pdf(external link)

The CELJ guidelines are important to those producing and evaluating digital scholarship, because it outlines best practices for peer reviewing, indexing, and archiving publications that exist only in digital format.

MLA Guidelines for Evaluating Work in Digital Humanities and Digital Media (2012) https://www.mla.org/guidelines_evaluation_digital(external link)

The MLA Guidelines are similar to those published by the CCCC in that they describe responsibilities for both digital scholars seeking tenure and promotion and for those charged with evaluating their work. The MLA advocates for clear, written institutional guidelines and that those guidelines be reflected in a digital scholar’s job description. Departments should educate their internal reviewers and engage qualified external reviewers when necessary. Digital scholars must negotiate for access to tools and other necessary working conditions and explain their work for evaluators.

Report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion (2006) http://www.mla.org/pdf/taskforcereport0608.pdf(external link)

This report analyzes the results of a 2005 survey of 1,339 departments in 734 U.S. institutions. The overarching finding was that the majority of institutions still privilege the single-authored monograph for tenure despite the changing face of scholarship that makes the monograph less relevant and more difficult to publish because of the shrinking number of academic presses. The task force recommends institutions increase the importance of other forms of scholarship in tenure and promotion evaluation, including digital scholarship, and outlines the changes necessary to insure fair evaluation of such work.

WPA Evaluating the Intellectual Work of Writing Administration (1998) http://wpacouncil.org/positions/intellectualwork.html (external link)

Special Issues/Research on Digital Scholarly Evaluation

The articles in the four special issues listed here are cross-referenced in the categories above.

Journal of Digital Humanities--Closing the Evaluation Gap, 2012

Living in a Digital World: Rethinking Peer Review, Collaboration, and Open Access
Sheila Cavanagh

Evaluating Collaborative Digital Scholarship (or, Where Credit is Due)
Bethany Nowviskie

A Call to Redefine Historical Scholarship in the Digital Turn
Alex Galarza, Jason Heppler and Douglas Seefeldt

How to Evaluate Digital Scholarship
Todd Presner

Short Guide To Evaluation Of Digital Work
Geoffrey Rockwell

Promotion and Tenure for Digital Scholarship
Laura Mandell

Evaluating Scholarly Digital Outputs: The Six Layers Approach
James Smithies

Evaluating Multimodal Work, Revisited
Shannon Christine Mattern

Evaluating Digital Humanities Work: Guidelines for Librarians
Zach Coble

Let the Grant Do the Talking
Sheila Brennan

Tenure as a Risk-Taking Venture
Mark Sample

Explaining Digital Humanities in Promotion Documents
Katherine D. Harris

Evaluating Digital Scholarship: Experiences in New Programmes at an Irish University
Michael Cosgrave, Anna Dowling, Lynn Harding, Róisín O’Brien and Olivia Rohan

Profession—Evaluating Digital Scholarship, 2011.

Anderson, Steve &  McPherson, Tara (2011). Engaging digital scholarship: Thoughts on evaluating multimedia scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.136(external link)

This article offers a critical survey of contemporary institutional struggles to come to terms with multimedia scholarship. Building on their work as coeditors of the online journal Vectors, the authors put forward requirements that are essential to the future of emerging scholarship: respect for experimentation and emerging genres, appreciation for transdisciplinary and collaborative work, the updating of models of citation and peer review, rewards for openness and contribution to a public commons, and valuing the development of tools and infrastructure. (cross-listed in Evaluation Strategies)

Rockwell, Geoffrey (2011). On the Evaluation of Digital Media as Scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.152(external link)

Rockwell argues we need to develop a culture of conversation around the evaluation of digital academic work because where there is a conversation around evaluation in a department, both hires and evaluators are more likely to come to consensus as to what is appropriate digital research and how it should be documented. The article surveys common types of digital scholarly work, discusses what evaluators should ask, discusses how digital researchers can document their scholarship, and then discusses the types of conversations hires and evaluators (like chairs) should have and when they should have them. (cross-listed in Evaluation Strategies)

Nowviskie, Bethany (2011). Where credit is due: Preconditions for the evaluation of collaborative digital scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.169(external link)

According to Nowviskie, when assessing digital humanities scholarship for purposes of tenure and promotion, committees must focus as much on process as on product, because digital work is situated in especially complex and collaborative networks of production and reception. These changes in evaluation require a rethinking of notions of solitary authorship, development of new standards for attribution, and a revising of institutional policies that govern intellectual property. The article argues that fair and full acknowledgment of the work of others (including non–faculty members and alternative academic contributors) will lead to a system in which new work in the humanities is better fostered, designed, distributed, and preserved. (cross-listed in Evaluation Strategies)

McGann, Jerome (2011). On creating a usable future. Profession.  http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.182(external link)

McGann asks the question, how should humanities scholars, and especially their educational and research institutions, deal with the digital transformation of their libraries and publishing venues? The article uses The National Digital Public Library initiative, launched in 2010, as a decisive event that allows reflection on the early history of digital technology in the humanities, and argues that the most pressing need is for the profession at large to become an informed and active player in the transformation of postsecondary education and scholarship. (cross-listed in Preservation)

Fitzpatrick, Kathleen (2011). Peer review, judgment, and reading. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.196(external link)

This article argues that evaluating new forms of scholarship for tenure and promotion requires taking those forms, and the methods of peer review they bring with them, on their own terms. Even more, it requires exercising the critical judgment on which our profession relies instead of outsourcing that judgment to others. Such evaluation requires reading both the work and the available evidence of the ways that scholars have responded to that work. (cross-listed in Peer Review)

Computers and Composition—Tenure, 2000.

Maid, Barry. (2000). Yes, a technorhetorician can get tenure, Computers and Composition 17(1), 9-18. http://dx.doi.org/10.1016/S8755-4615(99)00026-2(external link)

Maid claims “technorhetoricians” vulnerabilities at tenure time are similar to those who take on discipline-related administrative positions (e.g., Director of First-Year Writing, Writing Center Director, Writing Across the Curriculum Director). Drawing on his eleven years of administrative experience and work mentoring junior colleagues, he examines the problem of gaining tenure in English departments when one is not a literary specialist. Then, using Ernest Boyer’s (1990) new definitions of scholarship as a springboard, he suggests several possible approaches to establishing a successful tenure case. (cross-listed in Strategies for Success)

Rickly, Rebecca. (2000). The tenure of the oppressed: ambivalent reflections from a critical optimist, Computers and Composition 17(1), 19-30. http://dx.doi.org/10.1016/S8755-4615(99)00027-4(external link)

This article presents an individual reflection on the optimism and ambivalence associated with self-assessment and tenure and promotion assessment, particularly in work with technology. Rickly claims that junior faculty may be setting themselves up for failure by identifying more with the values of the computers and writing community than those of their home institutions and argues that unless the work we do impacts pedagogy, service, or publication digital scholars should not consider seeking value for it within the institution. (cross-listed in Strategies for Success)

Day, Michael. (2000). Teachers at the crossroads: evaluating teaching in electronic environments, Computers and Composition 17(1), 31-40. http://dx.doi.org/10.1016/S8755-4615(99)00028-6(external link)

Raises questions about the evaluation process for composition faculty who use computer and Internet technologies in the classroom and for distance learning. Discusses the "panoptic" effect made possible by the accessibility of class Web pages to administrators. Concludes with a set of practical recommendations for faculty and their departments on evaluating those who work with computer and Internet technologies. (cross-listed in Digital Pedagogies and Online Teaching)

Gruber, Sibylle. (2000). Technology and tenure: creating oppositional discourse in an offline and online world, Computers and Composition 17(1), 41-55. http://dx.doi.org/10.1016/S8755-4615(99)00029-8(external link)

This article points out the shortcomings of the positions assigned to technorhetoricians in traditional institutions, and argues that they are a diverse and multilayered group whose members occupy many positions, emphasizing the shifting and sometimes contradictory nature of marginality in cyberspace. Gruber concludes by pointing out possible effects of our diverse positionalities on our retention, tenure, and promotion efforts. (cross-listed in Strategies for Success)

Amy, Lori, & Crow, Angela. (2000). Shaping the imaginary domain: strategies for tenure and promotion at one institution, Computers and Composition 17(1), 57-68. http://dx.doi.org/10.1016/S8755-4615(99)00030-4(external link)

This article examines one department in the midst of establishing an independent writing major and program. The local situation is analyzed in the context of professional concerns regarding tenure and of disciplinary concerns regarding the recognition of computers and composition labor. The possibility to shape, to some degree, the guidelines for tenure and promotion, enables the authors to explore potential strategies for recognizing alternative forms of labor within and against tenure traditions. (cross-listed in Digital Tenure Examples)

Dorwick, Keith, Grigar, Dene, Barber, John F., Fanderclai, Tari, Howell, Karen, & Jorn, Linda. (2000). Looking elsewhere: career options other than the tenure-track teaching position for M.A.s and Ph.D.s in English, Computers and Composition 17(1), 69-95. http://dx.doi.org/10.1016/S8755-4615(99)00031-6(external link)

This article is an edited transcript of a LinguaMOO conversation between five scholars (Four computers and writing specialists and one librarian) who work in nonteaching positions. They discussed their careers and afterwards annotated and responded to the transcript. (cross-listed in Strategies for Success)

Kairos: A Journal of Rhetoric, Technology, and Pedagogy—Tenure and Technology: New Values, New Guidelines, 1997.

Cross, Janet, & Fuglevik, Kristian. (1997). Jesters get serious. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/candf/cvr.html(external link)

This webtext studies academic MOOs to develop a new model for tenure and promotion. Cross and Fuglevik advocate for a model the better represents the interconnectedness of research, service, and teaching, arguing that the traditional model attempts to artificially compartmentalize and quantify the three. (cross-listed in Evaluation Strategies)

Katz, Seth R. (1997). One department's guidelines for evaluating computer-related work. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/katz/art2.html(external link)

In this webtext Katz recounts his experience as part of a committee to revise his university’s tenure and promotion guidelines to include work with computers. He provides artifacts from the process including the original and revised T&P documents. Of most interest today is that the department’s dilemma in 1997 is still an issue in many English departments today: “We have recognized the reality of our situation: we are caught between tradition and transition, attempting to evaluate a technology and practice with which we have inadequate experience, and which keeps evolving as we watch.” (cross-listed in Digital Tenure Examples)

Nahrwold, Cindy. (1997). 'Just' professing: A call for the valuation of electronic scholarship. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/nahrwold/kairos.html(external link)

Nahrwold uses the prototype as a metaphor for the open-ended, collaborative, and boundary-crossing nature of digital scholarship. She argues that such scholarship “can perform, through their cyclical dialogue, the theories of ‘writing as process’ and ‘social construction of knowledge’ that inform current pedagogical practices,” and in turn argues such experimental work is essential to the knowledge construction. (cross-listed in Taxonomies)

Walker, Janice R. (1997). Fanning the flames: Tenure and promotion and other role-playing games. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder2.html?coverweb/walker/intro.html(external link) (cross-listed in Evaluation Strategies)

Browning, Tonya. (1997). Embedded visuals: Student design in web spaces. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder.html?features/browning/index.html(external link) (cross-listed in Digital Pedagogies and Online Teaching)

Brent, Doug. (1997). Rhetorics of the web: Implications for teachers of literacy. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder.html?features/brent/wayin.html(external link) (cross-listed in Taxonomies)

Barber, John F. (1997). The seven ages of computer connectivity. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder.html?features/barber/ages_intro.html(external link) (cross-listed in Taxonomies)

Anderson, Daniel, & Chevalier, Joi Lynne. (1997). Collaborative spaces and education. Kairos: A Journal of Rhetoric, Technology, and Pedagogy, 2(1). Retrieved September 16, 2015, from http://kairos.technorhetoric.net/2.1/binder.html?features/andchev/index.html(external link)

 

For more literary-critical perspectives, the Journal of DH has a whole volume on evaluating digital scholarship: http://journalofdigitalhumanities.org/1-4/(external link)

 

Related: Digital Pedagogies and Online Teaching

Scholarship on evaluating digital pedagogies and online teaching

Anderson, Daniel, & Chevalier, Joi Lynne. (1997). Collaborative spaces and education. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): http://kairos.technorhetoric.net/2.1/binder.html?features/andchev/index.html(external link)

Browning, Tonya. (1997). Embedded visuals: Student design in web spaces. Kairos: A Journal of Rhetoric, Technology, and Pedagogy 2(1): from http://kairos.technorhetoric.net/2.1/binder.html?features/browning/index.html(external link)

Day, Michael. (2000). Teachers at the crossroads: Evaluating teaching in electronic environments. Computers and Composition, 17(1), 31-40.

Raises questions about the evaluation process for composition faculty who use computer and Internet technologies in the classroom and for distance learning. Discusses the "panoptic" effect made possible by the accessibility of class Web pages to administrators. Concludes with a set of practical recommendations for faculty and their departments on evaluating those who work with computer and Internet technologies. (cross-listed in Special Issues) Strategies for Success Scholarship that provides strategies for how digital scholars and their allies can build successful careers and evaluation environments

Day, Michael, Delagrange, Susan H., Palmquist, Mike, Pemberton,  Michael A. & Walker, Janice R. What We Really Value: Redefining Scholarly Engagement in Tenure and Promotion Protocols

This article argues that tenure and promotion decisions should reflect the fundamental ways in which the academy and our positions within it have changed. Calling attention to the role senior scholars can play, the article considers the challenges offered by activity in four areas: digital and new-media scholarship, editorial and curatorial work, administration and leadership, and mentoring.

Dorwick, Keith, Grigar, Dene, Barber, John F., Fanderclai, Tari, Howell, Karen, & Jorn, Linda. (2000). Looking elsewhere: career options other than the tenure-track teaching position for M.A.s and Ph.D.s in English, Computers and Composition 17(1), 69-95. http://dx.doi.org/10.1016/S8755-4615(99)00031-6(external link)

This article is an edited transcript of a LinguaMOO conversation between five scholars (Four computers and writing specialists and one librarian) who work in nonteaching positions. They discussed their careers and afterwards annotated and responded to the transcript. (cross-listed in Special Issues)

Gruber, Sibylle. (2000). Technology and tenure: creating oppositional discourse in an offline and online world, Computers and Composition 17(1): 41-55. http://dx.doi.org/10.1016/S8755-4615(99)00029-8(external link)

This article points out the shortcomings of the positions assigned to technorhetoricians in traditional institutions, and argues that they are a diverse and multilayered group whose members occupy many positions, emphasizing the shifting and sometimes contradictory nature of marginality in cyberspace. Gruber concludes by pointing out possible effects of our diverse positionalities on our retention, tenure, and promotion efforts. (cross-listed in Special Issues)

Maid, Barry. (2000). Yes, a technorhetorician can get tenure, Computers and Composition 17(1): 9-18. http://dx.doi.org/10.1016/S8755-4615(99)00026-2(external link)

Maid claims that “technorhetoricians” vulnerabilities at tenure time are similar to those who take on discipline-related administrative positions (e.g., Director of First-Year Writing, Writing Center Director, Writing Across the Curriculum Director). Drawing on his eleven years of administrative experience and work mentoring junior colleagues, he examines the problem of gaining tenure in English departments when one is not a literary specialist. Then, using Ernest Boyer’s (1990) new definitions of scholarship as a springboard, he suggests several possible approaches to establishing a successful tenure case. (cross-listed in Special Issues)

Rickly, Rebecca. (2000). The tenure of the oppressed: ambivalent reflections from a critical optimist, Computers and Composition 17(1): 19-30. http://dx.doi.org/10.1016/S8755-4615(99)00027-4(external link)

This article presents an individual reflection on the optimism and ambivalence associated with self-assessment and tenure and promotion assessment, particularly in work with technology. Rickly claims that junior faculty may be setting themselves up for failure by identifying more with the values of the computers and writing community than those of their home institutions and argues that unless the work we do impacts pedagogy, service, or publication digital scholars should not consider seeking value for it within the institution. (cross-listed in Special Issues)

 

Collaboration

Scholarship about the value of collaboration in scholarly, research-based, or academic writing and how it may be assessed.

Nowviskie, Bethany. (2011). Where credit is due: Preconditions for the evaluation of collaborative digital scholarship. Profession. http://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.169(external link)

According to Nowviskie, when assessing digital humanities scholarship for purposes of tenure and promotion, committees must focus as much on process as on product, because digital work is situated in especially complex and collaborative networks of production and reception. These changes in evaluation require a rethinking of notions of solitary authorship, development of new standards for attribution, and a revising of institutional policies that govern intellectual property. The article argues that fair and full acknowledgment of the work of others (including non–faculty members and alternative academic contributors) will lead to a system in which new work in the humanities is better fostered, designed, distributed, and preserved. (cross-listed in Evaluation Strategies and Special Issues)

Preservation

Scholarship about the importance of digital preservation and access to scholarly work.

McGann, Jerome. (2011). On creating a usable future. Professionhttp://www.mlajournals.org/doi/pdf/10.1632/prof.2011.2011.1.182(external link)

McGann asks the question, how should humanities scholars, and especially their educational and research institutions, deal with the digital transformation of their libraries and publishing venues? The article uses The National Digital Public Library initiative, launched in 2010, as a decisive event that allows reflection on the early history of digital technology in the humanities, and argues that the most pressing need is for the profession at large to become an informed and active player in the transformation of postsecondary education and scholarship. (cross-listed in Special Issues)


Created by admin. Last Modification: Wednesday 04 of January, 2017 22:16:32 UTC by cheryl.