When I first started using Perusall, “annotation content” was the only grading option available, which made it fairly easy to establish grading criteria. As Perusall has grown and adapted to respond to user needs, so too have the grading options. Users can now choose between “Annotation Content Only” and “Holistic” grading schemas. While the “Content Only” option focuses specifically on the quality of student comments as determined by Perusall’s algorithm, the “Holistic” grading option allows you to reward students for engaging in effective reading behaviors. Under “Holistic” grading, you’ll find options to give students points based on

• the number of times they open an assignment;
• how much of the document they read;
• how much time they spend actively engaging with content (reading, viewing, commenting);
• how many responses their posts garner from other students;
• how many upvotes their comments receive from peers.

One of the questions I most frequently get from faculty is, “Which grading schema should I use and why?” The answer here really depends on your preferences and pedagogical needs. To help you make an informed decision, this two-post series offers a deep dive into the why, what, and how of both grading options. In this post, I’ll focus on “Content Only” grading and provide an overview of my “quick and easy” tips for setup. If you’re more interested in “Holistic” grading, stay tuned for my next post!

Why use “Content Only” grading?

This option is good for instructors new to Perusall who do not have the bandwidth to determine their “Holistic” grading preferences just yet. While there are many pedagogical benefits to the holistic settings (which I’ll discuss in a future post), first-time users sometimes find these settings a little intimidating to process, though the recent updates to the “Scoring” page layout significantly lower the barrier to entry. If you’re learning Perusall on a crunch, “Content Only” grading can reduce the time you spend on setup. Veteran users may choose the “Content Only” option if their sole purpose is to get students to critically engage with reading content. These are the most common reasons I hear from faculty. You may have other reasons why this option best suits your needs.

What is “Content Only” grading?

This setting is exactly what it sounds like: the content of student annotations is the criteria used to establish their grades. Students need to complete the minimum number of annotations before the deadline to receive full credit. You also have the option to add a penalty for clustering comments rather than evenly distributing them, but student grades will otherwise be determined by what the algorithm deems the quality of their annotations, low, medium, or high. Each annotation receives an individual score based on quality. If a student writes more than the minimum annotations required for full credit, the algorithm averages the highest scoring annotations to arrive at the grade. Perusall’s “How It Works” guide provides more details about the nuts and bolts of how it works.

How do I set up “Content Only” grading?

To reach this scoring option,

1. Select “Settings” in the left menu pane.
2. Select “Scoring” in the “Settings” ribbon menu at the top of the page.
3. Select the “Annotation content only”.

When you select this option, the “annotation content score target” should be set to 100%, as in the screenshot below.

Once you expand the “Content Only” accordion menu, you will have a few more settings to configure. I offer some suggestions for quick set up below, but obviously, you can configure these settings to best meet your needs.

Annotations to Grade

The first decision you will need to make is how many annotations Perusall should score. If students complete more than the required number of annotations, (set to 5 in the screenshot below), the algorithm will average their top five annotations to arrive at the final grade. Students who do not complete the required number of annotations will be docked according to the number of missing comments.

To determine the number of annotations you should require for full credit, consider the average length of your readings. Let’s say you average twenty pages of reading per assignment. How frequently do you want students to leave comments? I typically recommend somewhere around half of the average number of pages for the required comments, in this case ten annotations, but anywhere between seven and fifteen would be appropriate. The idea is to encourage students to comment regularly throughout the reading.

Pro tip: On occasion, I’ve found that I assigned a reading significantly shorter than average, maybe only two or three pages. The brevity of the reading makes it nearly impossible for students to reach a seven to ten comment requirement. In such cases, you will want to override the course pre-set grading options when you create the assignment to lower the number of annotations required for full credit to something more reasonable for the assignment length. I learned this the hard way. To override your scoring presets:

1. Add a new assignment.
2. In the new assignment window, select “Scoring” from the top ribbon menu.
3. Select the “Specify custom scoring settings for this assignment” bubble.
4. Set the “Annotation content score target” to 100% to maintain “Content Only” grading.

As you scroll down, you will see the “Annotations to Grade” option, which will be set to your course standard. Here you can update this number to something more reasonable for the length of the assignment.

Relative Point Value

The “Relative Point Value” option allows you to customize points the algorithm will assign to comments that do not meet, meet, or exceed quality expectations. By default, the system assigns each comment a 2, 1, or 0 based on the quality of the content.

I typically recommend that faculty change the “relative point values” to 3, 2, 1 and reserve 0 for students who simply do not do the assignment, but this change only matters if you check the “Quality scores are absolute box.”

Additional Options

Depending on your pedagogical purposes for using Perusall, you may want to consider whether a “post-deadline reply window” or “late annotation period” is right for you. For the sake of simplicity, you can say no to both of these options. However, if your goal is to foster ongoing dialogue outside the classroom, you may find that allowing students to receive credit for responses posted after the assignment deadline allows you to reward students for continuing the conversation. Giving credit communicates your value for that kind of activity. If you enable this option, students will only receive credit for responses to other students’ annotations. The “late annotation period” gives students some flexibility and grace by allowing them to receive partial credit for new annotations posted after the deadline.

The last option available for “content only” grading is the distribution penalty. This setting determines what percentage of a student’s grade will be docked if their annotations are clustered too closely together. In the early development stages of Perusall, users found that students would read and heavily annotate the first few pages of a text. Once they met the required number of annotations, they would stop reading. With that in mind, you will likely want to establish some level of penalty for this behavior as it’s usually indicative that students are not engaging with the reading as desired. I go with something between 25-50%, but there’s no right answer.

Advanced

Once you’ve chosen your “content only” settings, pop over to the “advanced” tab where you will find a few more grading-related options. To calculate the top of my “assignment score range,” I multiply the number of annotations for full credit by the maximum score on an assignment (5 annotations x 3 max score on each = 15 points each assignment).

You don’t have to do it this way, but it’s the easiest option. I set the bottom of my score range to 0 and check the box that says “assign a score if a student completes no work.” In my grading schema where students receive a 1 for a low quality comment, only students who do not submit any work receive a 0 on the assignment. I do this because I want to encourage students to engage in Perusall, so in that regard, even an emoji response is worth something.

The “threshold grading” option is particularly useful for instructors who want to use Perusall as a completion grade. Think of threshold grading as pass/fail. Once students reach the threshold, they receive credit for completing the assignment. For example, the maximum point value for my assignments as calculated above is 15; I set the threshold to 10.

Students will automatically receive credit for completing the assignment once they reach 10 points. Anything lower than 10 points will receive a 0. Threshold grading may be particularly appealing to anyone concerned about how well an algorithm can grade student responses as this setting does away with the grade nuance. You can focus on giving students a completion grade instead of assessing the quality of their content through the grade.

These are my recommendations for quick and easy auto scoring set up. As you get more comfortable with Perusall, you may want to explore the holistic grade settings to encourage your students to engage in certain types of beneficial behaviors. In a future post, I’ll explore the holistic settings along with potential benefits to students.