Specification ungrading

This is a third post about my adventures in alternative grading (here are the links to the first, and the second posts in the series).

Briefly: after reflecting on my “Mastery grading” saga last semester, I decided to once again change everything in my "Intro to Neurobiology” class. This time around, I’ll try a revolutionary yet controversial “Ungrading” approach, toned down using a hybrid “Specification” scheme. I am pretty excited for this experiment, as I feel that it may be uniquely suited for teaching intro classes in particular. If this sounds interesting, read along!

First of all, why changing the grading system again? See, I kind of enjoyed my experience with mastery grading last semester, but it made me realize several things. One: it was really hard on my time, and I don’t think it is sustainable in the long-term, just because weekly verbal exams with several students, even if very short, are a huge time commitment. Second: as we spent all office hours talking about science, we had no time left to talk about more human, and thus potentially more important things, such as life, college, careers, art, courses, learning strategies etc. The existential component was gone, and it is a shame, especially for an introductory course, that is mostly taken by first-years. 

But the most important concern, is that, based on student evaluations, as well as some anecdotal evidence, students didn’t enjoy the class as much, compared to its previous renditions. In fact, it is the first time in history that my course appreciation rating tanked slightly below 4.0 on a 5-point scale (p=0.03, compared to my typical performance of about 4.5). This numerical analysis is, of course, all sorts of problematic (I hope to write a separate, referenced post on this topic), but it is still curious. Students learned a ton more than in any of my previous classes; and not by a bit, but something like 3-4 times more (see my previous analyses). Moreover, at least nominally, in an abstract, cerebral way, they knew that they learned a ton, as I kept reporting back to them in class. They also unambiguously reported lower levels of anxiety. And still, they liked the class significantly less (both in the statistical, and in the IRL sense of this word). Weird!

There are several ways to look at it, philosophically. One can of course say that it is more important to be a good professor than students’ favorite professor [1]. One could also argue that I should have done a better job pitching the class to the students. Or that we can just ignore these ratings, as first-years, taking their first course in the discipline, don’t have a good reference point for judging anything. Or we can go full groachy and complain that students’ goals are no longer about education, as they don’t want to walk uphill both ways. But regardless of whether anything of it is true, the fact is that I worked more, students learned better, and yet they were about 10% less happy (4.5 to 4.0), on average, than in all previous years. And I don’t like it. Because arguably, the main goal of an intro course is not to prep students for the second-year sequence, but to give them a taste of, and an appreciation for science. Here, the emotional goal, the enjoyment of the process, may be more important than objective learning. And if this is true, then having lower student appreciation is actually worrisome. Not to mention that that the second most important goal for a typical intro course is, probably, the metacognitive training: learning how to learn. Which, again, suffered with “mastery training”, because of its emphasis on skills. And so, even though it was fun to try, I think that “mastery grading” may just be poorly suited for intro courses in biology; at least of a type we teach here at Bard. 200-level biology, or some electives - maybe. 100-level math: maybe, for a different reasons. But not the bio intros.

What could be an alternative? Do I have to return to a vastly inferior point-based grading, just because it feels more familiar to students, and so less intrusive? Maybe not! Introducing: yet another revolutionary grading technique, called “The Ungrading”! While lots is written about it [2, 3], here are the key points:

• Lots of feedback, but no letter grades or even points
• 2-3 written self-reflections (self-evaluations), in which students are encouraged to, essentially, grade themselves, and give themselves advice on how to do better [4]
• Followed by a one-on-one discussion in office hours
• The final grade in the course, for each student, is assigned by this student themself. If the grade does not feel right to the instructor, they negotiate. In extreme cases, instructor reserves the right to override the grade, but in practice it happens very rarely, and even then, usually upwards, rather than downwards.

What are the benefits of this approach? The biggest two are: (1) a replacement of extrinsic motivation (grades) with intrinsic motivation (learning), which is known to be better [5], and (2) explicit introduction of metacognitive exercises and discussions. Which was precisely what mastery grading lacked, as it shifted the focus on prepping a bit too much for my liking. And again, prepping is OK and fun in some classes, if students are into it, and know what they are doing. But in an exploratory intro class, a metacognition-oriented design may be much more impactful [6].

And yet, in its pure form, “Ungrading” feels very dangerous. It is known to be paradoxically anxiety-triggering: it is so unlike anything students have experienced before that they freak out, and cannot believe that it is not a trap [7]. And even at the existential level, pure self-evaluation can be incredibly stressful (think of the impostor syndrome!). This downside could potentially be corrected with thoughtful one-on-one discussions on inclusion, and personalized advice on learning strategies, but I think that for most faculty, me included, it would be a tough ask. It would come dangerously close to life coaching, or maybe even therapy, which we are never good at.

Moreover, pure “Ungrading” feels particularly dangerous for teaching a highly heterogenious group of students: if a third of your class suspects that they may not belong, while another third are competitive “gunners” that try to out-achieve anything that moves, it is easy to see how a system of complete self-grading can be perceived negatively by everyone, and become really detrimental for everyone’s learning. And then there are the mundane issues of class attendance and the use of mind-altering substances immediately before class. In essence, pure “Ungrading” feels like a case of a very “Not-inclusive pedagogy”, in the sense that it does not offer students enough structure and motivational support to help them succeed with their studies [8].

Is there a way to fix this system, and negate the negative sides of it, while keeping the good? I think the answer is “yes”: if we combine Ungrading with a simple version of Specification grading, to make sure that the course has some strong, fine-grained structure, but still retains the reflective aspect of “Ungrading”. The structure will help to reduce anxiety, both for those students who don’t yet know how to work and how to feel about their own success in class, and for those that are concerned with permissive grading systems being somehow unjust. Remember that prof who gave a “B” to everyone, but required extra work for an “A”, which really was the most run-down version of specification grading? [10] We could use a similar approach! The coolest thing about specifications is that they give you a powerful tool for combining very different types of assignments, by separating them into different strata (aka “Bundles”). With specifications, we can have the best of all worlds: a structured “minimum required performance”, a free-form reflective “Ungrading”, and a “challenging bonus assignment” on top, for those who want to walk an extra mile. We can reap all the benefits of not grading, without paying the price of it!

But enough rambling; at this point I’ll just post the draft of my new syllabus below, which I hope is rather self-explanatory. What do you think? Would it work? Any suggestions, while I can still make changes to it?

Intro to Neurobiology, Draft Syllabus

(Section on grading)

This course is graded using a “Specification Grading” system, which is known to work better than the standard “Point-based” system, both in terms of final results, and student experience. The gist here is that each letter grade comes with a certain list of criteria, and it is up to you to pick the level you want to achieve.

To get a C: 

• Participate in 80% of classes
• Participate in 80% of labs
• Submit 60% of short written assignments (weekly reading reflections, labs, short exercises etc.) They don’t have to be perfect, but they need to be reasonable. If any given submission is problematic, I’ll let you know within a week, so that you could improve next time.

To get a B:

• Participate in 90% of classes
• Participate in 90% of labs
• Submit 90% of short written assignments
• Submit 2 reflective letters (each about 3 pages long; see the descriptions below). One before the mid-semester break; the other one before the completion week. The letters should be meaningful; if they are not, I’ll let you know, and give you a chance to resubmit.
• Come to office hours at least once in the first half, and once in the second half of the semester.

To get an A:

• On top of everything specified in “B”, write a good final essay about a neuroscience paper; then meet with me during the completion week, to discuss both the paper and the essay. See the instructions below.

Reflective letters

To get a “B” or higher, you need to write two letters, reflecting on this class, and your work in it.

For the first reflective letter (to be written before the mid-semester break), please answer these questions. Each answer should be about half a page (200-300 words).

1. Please describe your personal goals for this course.  What do you want to take from it? What do you want too learn and understand? Why? 
2. Describe the coolest, most interesting thing you learned in this course so far. Try to write as if you were describing it to your friend who is not taking this course. Please tell me why you, personally, find it interesting.
3. Describe one topic you studied in this course that you still do not completely understand, but would like to understand. Try to explain how you know that you don’t understand it fully yet. This is a tricky question, as it is hard to notice that you don’t understand something, and it is even harder to write about it. Note that if what separates you from knowing this topic is a few minutes of googling, it clearly does not count as an answer. Try to think deeply about everything we studied in this course, and whether you really got it. It is an important skill: to know what you know, what you don’t, why you want to know it, and how to get there.
4. Describe how you study for this course (half a page). How much do you study each week? How do you use this study time? What sources do you use? Have you tried to study with someone? Have you looked for help? Note that I will not judge your studying schedule, even if you confess that you don’t study at all. I just want to know what you do, and I want you to think about it.
5. Tell me something about this class, your work in it, and your experience of it. What seems to work well? What does not? Why? How to make it work better? If there are any particular things you would like me to address, please let me know. I’d like to know what you think, and how you feel.
6. If you had to give yourself a mid-term letter grade for the first half of this course, what grade would it be? Why not a higher, or a lower one? Note that this self-grading won’t affect your actual grade at all, so let’s just honestly think, and later talk about it.

Second paper (to be written in the second half of the semester, before the completion week). Each response should take about half a page (200-300 words).

1. Describe the most important, most useful thing, either conceptually or practically, that you learned in this course. Not necessarily the most interesting one, but the one that is consequential, for you, personally. What makes it important?
2. Reflect on your work in the second half of this course, compared to the first half of it. Is there anything that you do differently now, compared to 2-3 months earlier? Have you tried to deliberately change something about your studies? Did it work? Have you stopped doing something because it did not feel productive? 
3. If you had to give yourself a letter grade for this course, what grade would it be? Why not a higher grade? Why not a lower one? Again, this self-grading won’t affect your actual grade, but I’d like us to talk about it.
4. What are you plans for next semester, and next year? What do you want to do? What do you want to learn? Are they the same plans that you had as you arrived at Bard, or did anything change? If yes, why? Did the courses you took this semester affect your plans for the future, in any way?

Final essay, and a discussion of it

This assignment is only required if you want to go an extra mile (above a letter grad of “B”). If you put good work in it, you can get an “A”; if you don’t put enough work, you can get an “A minus”, or a “B plus”, depending on how far you get.

First, you need to find a neuroscience paper that you’d like to write about, from a journal named eLife ( https://elifesciences.org/ ). It is an open-access journal, so all papers are freely available online. It may make sense to find a paper that has something to do with something we learned in class, but if you want to go rogue, it is also possible (just make sure it’s neuroscience). Also, it has to be a primary paper: an experiment, a model, or a meta-analysis, but not a literature review, or an opinion piece. 

Send me an e-mail with a link to the paper, to claim it. If I think that the paper is a poor choice, I’ll let you know (it’s unlikely, but just in case).

The earliest you can claim the paper is a week before the mid-semester break. The latest you can claim it, is immediately before the completion week. The more time you have, the better you can prepare, and the more feedback I can give you, when you ask for it.

Read the paper of your choice. Read some of the papers that it references, to learn the background (especially those referenced in the “introduction” section). Make sure you understand what question this paper asks, and why it is important. What are the consequences of knowing it? What are the hypotheses the authors had? You need to understand the methods. You need to know the figures, and understand what they are trying to say. You also need to read the exchanges that the authors and the reviewers had before the paper got accepted (eLife is one of the few journals that publish these exchanges openly). You need to get a good idea of what happened there, at the review stage; what the concerns were, and how the authors responded to these concerns.

Then write your essay. It should clearly address the following points:

1. The rationale for the paper (what makes its question important), and the background needed to understand the study. Explain what we need to know before we even start reading the paper of your choice.
2. The narrow question (or questions) the paper posed, and associated hypotheses.
3. The methods used. You need to have a good grasp of the general idea of each method, and some meaningful details about them. Outline weak and strong points of these methods, compared to others methods authors could have conceivably used within the same time and financial budgets.
4. The overall structure of the paper, including the key message of each panel in each figure. How do these messages contribute to the answer the paper eventually provides? How do different figures interact with each other?
5. The answer eventually provided by the paper. Make sure to refer back to the narrow question the authors posed, and the hypotheses they had. What does this answer mean, in a broader scale of things?
6. What are the limitations of this study? What follow-up studies can be inspired by this study? Some of these are probably outlined in the Discussion, some may be identified from the peer review materials, some you can discover yourself. Make sure to go beyond the obvious; if a limitation is true for every paper ever written, it is probably not the most useful thing to discuss.

You need to submit the final version of the essay by the first day of completion week, and we’ll schedule a meeting to talk about it. In the meeting, we will discuss the paper you wrote about, using your essay to answer the six questions outlined above. 

There will also be a deadline to submit a draft version of your essay. If you submit something by this deadline, I will read your draft, and offer you feedback. We can also meet during drop-in hours, to talk about your project.

The measure of success here is the depth of your research towards understanding this one single paper. Some topics, methods, and analyses are inherently more complicated, while some are simpler, so we are not going after some predefined level that you need to reach. Rather, I want to see your independent work, and your ability to use what we learned in class to learn more science on your own. If the paper is “easy”, I will expect a deeper understanding of it, and I may ask more follow-up questions. If the paper is hard or long, a deep understanding of only one part of it, perhaps as little as one figure, may suffice. What matters is the amount of thought you put in your project. You definitely don’t need to become perfect (it is impossible to know everything!) but you need to kinda become a specialist in this one particular paper, compared to a person reading it for the first time.