Rethinking Grading: An In-Progress Experiment

16Feb16

Today I started my spring course, Television and American Culture, a class I have offered around 15 times. It’s the course that inspired my textbook (of the same name), and my co-edited book How to Watch Television also was structured to fit with the course’s design. In short, it’s the course that I’ve dedicated the most work to honing, and I feel that overall it works quite well… except for one facet: grading.

I hate grading. I hate how grades function in higher education for students, for faculty, for parents, and for institutions. I hate how grades often work as an obstruction for learning, rather than a motivation, reward, or neutral assessment. I firmly believe that, at least here at Middlebury, figuring out a way to rethink the culture of grades would be the most effective and impactful reform we could make. Such reforms are challenging and slow-moving at an institutional level, but I was moved to jump into the deep-end to rethink how grading works in this course. And thus I’m running an experiment this semester by completely changing the course’s grading system.

The approach I am taking is called Specifications Grading, which emerged from a fairly well-established alternative approach to grading typically called Contract Grading. I first learned about contract grading a few years ago through Cathy Davidson blogging about her use of the system at Duke. The idea bubbled around in my head for years, but I decided to give it a whirl after reading this piece by Linda Nilson on specifications grading, which is based on her book. The difference between specifications & contract grading is a bit fuzzy, and ultimately not as important as their similarities, which are tied to three key principles:

• All individual assignments are graded on a Pass / Fail or Satisfactory / Unsatisfactory basis. The bar for Satisfactory is set higher than what we typically think of as “passing work” (more like a typical B than a C), with a satisfactory assignment being one that meets its clearly-articulated specifications and learning goals. This means that an assignment that meets some but not all of the goals & specs is Unsatisfactory, a much more rigorous bar than how most faculty (especially in the humanities) grade papers. This also means that you need not spend time quibbling between giving a paper a B+ vs. A–; it either meets the expectations, or it doesn’t. Instead, I can spend my assessment time providing qualitative feedback, which is more rewarding for everyone. Plus the system has options for revision so that a student receiving an Unsatisfactory can choose to improve their work and hopefully satisfactorily accomplish the assignment goals.
• Assignments are designed to demonstrate that students have achieved the course’s specific learning goals. This seems obvious, but I was surprised by how weakly the old assignments for my course were connected to stated learning goals. Under this approach, you should be able to clearly highlight how each assignment serves the stated goals. Making those connections explicit greatly improved the conceptual basis for the assignments I give, and I hope will make assessing whether they accomplish those goals easier.
• Final grades are determined by students’ accomplishments in a hierarchy of “assignment bundles.” If we set the passing bar for the course at a C, then we designate which quantity and depth of assignments are necessary to accomplish the course’s base learning goals. Additional assignments are added to that base to reflect more sophisticated and deeper learning, creating bundles for B and A levels. This system gives student full control over which of these bundles they will strive to accomplish, based on their own learning priorities and self-aware judgment over time-management and intellectual goals.

This last system of bundles is kind-of a “hack” to the system: because most of us teach in institutions that require us to enter a single letter grade into a transcript at the end of the semester, we need to be able to produce such a metric. However, the specifications approach eliminates the stresses of grading each assignment by designing a course which allows students to choose their own learning paths transparently, as linked to grades at the end of the process. Hopefully, at the end of the semester, I can know that a student who received a B demonstrated that they learned four of the course’s explicit learning goal, while a student who received an A learned all five. (See below for the specific language laying out this system for students.)

In designing my syllabus, I embraced a tiered set of learning goals, based on various schema of levels of learning and cognition. The base level focuses on learning and comprehending the information covered in the course, and being able to express this knowledge effectively: this is what any student who passes the course should accomplish, and the C bundle assesses this knowledge. The next tier involves applying that knowledge to analyzing new examples and scenarios, with assignments in the B bundle requiring such analytical application. The highest tier invites students to generate their own arguments and synthesize both information and analytic approaches across realms of knowledge, captured in the additional requirements of the A bundle. Thus a student receiving a high grade is not an indication of doing the same assignments particularly well, but demonstrating more challenging modes of engagement and analysis. This seems like a more accurate demarcation of learning.

So today in class, I rolled it out for students, walking through the policy statements reproduced below.* It took some time for them to grapple with the new system, but I think they got it, and I sensed that they mostly thought it was a cool idea. One said, “it’s kind of like a board game” – which I affirmed, but emphasized that “winning” means understanding the system enough to actively engage in the material to achieve the level of learning you aim to accomplish, not gaming the system. We will see how it unfolds, and I will try to update the blog on the experiment in progress. I’d love to hear what readers think of such a system, whether you’ve tried anything similar, and any advice for what might emerge as the semester progresses.

• A few have asked the size of the course: around 30 students, mostly sophomores & juniors. About 1/3 declared majors, with another 1/3 who might become majors.

How This Course Works:

This course uses an unconventional approach to assessing student learning called specifications grading. (If you’re curious about this approach, see this overview.) Instead of grading each assignment on a measure of “quality,” everything will be assessed as Satisfactory / Unsatisfactory based on whether a student demonstrates the stated learning goals. Assignments will be “bundled” into three tiers that reflect a hierarchy of learning goals for the course. Final grades will be assigned based on which bundles of assignments a student satisfactorily completes—these final grades are not the goal or outcome of the course, but are designed to indicate which learning goals students demonstrate that they accomplished.

Note that this is a new system for Professor Mittell (and, presumably, Middlebury College!)—he agrees to take however much time is needed to ensure students understand expectations and practices, and are poised to succeed to their desired level within the system. His goal is to help each student achieve Satisfactory levels of learning on all components of the course that they undertake, and to be transparent about expectations for learning throughout the semester.

Learning Goals:

All students who pass the course (with a minimum grade of C) will have demonstrated the ability to:

• Describe how American television works as a commercial industry, functions as an aesthetic and communication medium, and both shapes and is shaped by American culture and society
• Apply specific vocabulary and concepts to explain television’s industrial, formal, cultural, and technological facets
• Communicate their ideas with fluency and clarity

Students who achieve a higher level of mastery (with a minimum grade of B) will have also demonstrated the ability to:

• Analyze television’s industrial, formal, cultural, and technological facets with original insights and connections between different examples and contexts

Students who achieve the highest level of mastery (with a grade of A) will have also demonstrated the ability to:

• Create, substantiate, and communicate an original analytic argument that synthesizes multiple facets of television

Grades:

All assignments in the course will be assessed as Satisfactory / Unsatisfactory, with the specifications required for Satisfactory articulated on each assignment. In general, Satisfactory should not be viewed as “minimally competent” (as is typical for a C grade at Middlebury), but rather as a mark of having achieved the assignment’s learning goals and specifications (probably more like a B grade in an average Middlebury course). Either an assignment meets the goals, or it does not—there is no gradation of assessment.

The only letter grade that will be given in the course will be your final grade, and it will reflect the “bundles” of assignments and requirements you have satisfactorily accomplished in the class. That final letter grade is not an assessment of your intelligence, your abilities, or your value as a person—in fact, Professor Mittell never will grade “you” directly, and grading is never a reflection of who you are as a person. Rather, the grade reflects what you demonstrated that you learned in the course: no more, no less.

Built into this system is a good deal of choice as to how much you wish to learn and how hard you want to work to demonstrate and apply that learning. You might choose that passing the course with a C is sufficient for your goals—it is perfectly appropriate and worthy of respect for you to make that choice, especially if it allows you to proactively allocate your time to other endeavors at Middlebury or beyond. If you strive to get an A in the course and maximize your learning, you should know that you are taking on that work and challenge yourself, and should make sure you are in a personal and academic situation to achieve that level of engagement.

C Bundle – Students who complete the following will pass the course with a grade of C:

• Actively attend all course meetings, with up to five absences, per the attendance policy below
• Complete at least 7 weekly screening responses to a Satisfactory level
• Complete all 6 questions (at either Basic or Advanced levels) on the three take-home essay exams to a Satisfactory leve

B Bundle – Students who complete the following will pass the course with a grade of B:

• Actively attend all course meetings, with up to three absences, per the attendance policy below
• Complete at least 9 weekly screening responses to a Satisfactory level
• Complete all 6 questions on the three take-home essay exams to a Satisfactory level, including at least 3 Advanced level questions

A Bundle – Students who complete the following will pass the course with a grade of A:

• Actively attend all course meetings, with up to two absences, per the attendance policy below
• Complete at least 10 weekly screening responses to a Satisfactory level
• Complete all 6 questions on the three take-home essay exams to a Satisfactory level, including at least 5 Advanced level questions
• Complete the original argumentative essay assignment to a Satisfactory level

Modified grades of + and – will be used when a student’s Satisfactory activities fall between the bundles. For instance, a student who met the requirements for the B Bundle, as well as completing 5 Advanced questions on the exams would receive a B+, while a student who fell just short of the B Bundle requirements would likely receive a B– final grade. Grades of D will only be given in rare cases where a student meets most of the C Bundle requirements but falls short in one area—typically, a student who does not meet the requirements of the C Bundle will fail the course.

Tokens & Flexibility:

Since every element of the course is assessed on an all-or-nothing basis, it might be stressful to strive for Satisfactory given that the stakes for not meeting that threshold may be significant. To ease stress, to allow for flexibility—and most of all, to maximize opportunities for learning—every student starts the course with 3 virtual tokens that can be “exchanged” for some leniency or opportunities for revision. Using a token will allow a student to do one of the following:

• Eliminate an absence from their attendance record
• Count an Unsatisfactory or not completed screening response as Satisfactory
• Revise and resubmit one of the questions on the second or third exam (note that the first exam has a built-in revision option)
• Submit an exam or essay assignment up to 48 hours late

Professor Mittell will track a student’s tokens throughout the semester. Exchanging them for absences or missed screening responses will happen at the end of the term. There will be an opportunity for students to earn an additional token later in the semester by attending and responding to public lectures relevant to the course.

Assignments:

Weekly Screening Responses:
Each week, students will watch a selection of television material on Wednesday evening. By Thursday morning, students are expected to post a response to Moodle for that week’s screening. The goal of response is to connect at least one program screened to concepts from the course readings. To earn a Satisfactory, each post must do the following:

• Be submitted on Moodle by Thursday at 9am
• Consist of at least 300 words
• Demonstrate that you have watched and read the material
• Be accurate about the course material
• Draw explicit connections between ideas raised in the readings to material from the screening
• Convey something that personally struck you as interesting, compelling, engaging, or otherwise moved you to write about this aspect of the course materials
• Write in a clear manner – your style can be less formal than typical academic prose, but it should be serious and engaged with ideas
• Contain no more than 3 errors to Standard Written English

It is recommended that you compose your responses in a word processor and paste the text into Moodle, as web browsers can crash as you are writing. Screening responses cannot be revised or submitted late.

Take-Home Essay Exams:

There will be three take-home essay examinations, all of which are required to be Satisfactorily completed to pass the course. Each exam will be distributed on a Thursday, and due the following Tuesday. Students may consult notes, readings, screenings, and other online material, but cannot consult with other people (besides Professor Mittell) during the writing process. Per Middlebury’s Honor Code, all work submitted is presumed to be your own.

Each exam will have two general prompts, with each prompt offering two options: a Basic and an Advanced essay. The Basic essay will require students to demonstrate that they understand and can communicate course material in their own words. The Advanced essay will require students to apply course material to an original analysis, requiring higher-level engagement with the material. Students are free to choose which option they choose for each prompt. For an example of this difference, a hypothetical Basic prompt might ask, “How does the television industry use the Nielsen ratings to sell audiences to advertisers?”; the Advanced version might ask, “How effective are the Nielsen ratings measuring viewer behaviors, and how might this system better serve the interests of television viewers?”

In order to pass the course, students must earn a Satisfactory on every exam prompt at least at the Basic level; completing more Advanced prompts will allow students to earn a higher final grade, per the description above, by demonstrating more sophisticated levels of learning. Students may opt to revise and resubmit any answer that receives an Unsatisfactory within one week of receiving their essay back. The first exam automatically allows for revisions without requiring tokens, while the other exams require students to use a token to revise an answer. If a student receives an Unsatisfactory on an Advanced question, they can opt to revise the answer to address the Basic prompt. More detailed specifications will be included with the exam prompts.

Final Argumentative Essay:

The final paper in the class will be an analytic essay about a television program of your choosing, due on December 20. The basic idea is to write an essay that could function as an additional chapter of How to Watch Television. Because the essay is due during the exam period, revisions will not be allowed after the due date; however, students electing to complete this assignment are encouraged to meet with Professor Mittell and submit drafts for feedback in advance. A proposal for the essay will be due April 7. More detailed specifications will be included with the assignment.

Active Attendance:

You are expected to attend all class meetings on time, having done the readings, thought about the material, and prepared to engage in discussion and in-class activities. This is not a lecture course, so active participation and engagement is required. Attendance will be taken regularly—students who are present in class but who are unprepared or do not participate adequately will be marked as absent. Being late two times counts as an absence. Students who miss a class should find out what they missed from their classmates and learn the necessary material.

Filed under: Academia, Teaching, Television   |  11 Comments
Tags: specifications grading