Apply the results of a Buddycheck evaluation

How to use peer scores to adjust marks for individual students in Canvas.

The guidance below explains how to use the results of Buddycheck evaluations to amend marks for individual students directly in Canvas Gradebook using Grade Sync functionality. Here you will also find information about the adjustment factor and how it is calculated. Guidance is also provided on the option to download the results into Excel.

See our ’Conclude a Buddycheck evaluation’ page for guidance on reviewing the results and, if necessary, removing students from an evaluation, prior to this step.

Update grades in Canvas

This short video tutorial provides an overview of the main steps detailed below.

• Buddycheck - Grading: How to include Buddycheck results in your students' grades from your learning environment.

In Canvas 

Prior to this step you should have:

1. set up a group assignment in Canvas
2. set the assignment up so that you are marking the students as a group.

Once your assignment deadline has passed:

1. mark the students as a group
2. change the setting on the assignment to ‘Assign grades to each student individually’.

In Buddycheck

1. Go to the Buddycheck evaluation you created and the students have completed.
2. Remove students who are no longer taking the assessment.
3. Click the More button and select Publish grades. This will not publish the grades to students, the adjustments still need to be selected and the grade needs to be posted in the Gradebook. You can only ‘publish grades’ after ‘locking’ and or ‘releasing’ the evaluation results.
4. Select the assignment you wish to amend the mark for by clicking to highlight the row and then scrolling down and clicking on continue. If the assignment is set up as a group submission in Canvas then before using Buddycheck to adjust the marks the assignment must be changed to allow individual marks to be entered. Unless this is changed Buddycheck would not allow this assignment to be selected and will display as ‘No’ under ‘Changing Possible’.
5. Review the adjustment factor and grade changes and amend impacts (if need be). Buddycheck will at this point present you with a summary of how grades for the assessment you selected will be updated. You can change how the adjustment factor will impact grades here  Ensure that any adjustment factor calculation is rounded, to do this ensure the New grade options is set to 0, otherwise grades cannot be brought from Canvas to SussexDirect.

   

   1. Choose to include/not include self-scores in adjustment factor calculations. Note: It is recommended to turn student self-scores off. They will still receive feedback comparing their self-scores to their peers’.
   2. Decide what weight the adjustment factor should have on final scores (a different weight can be applied for a positive/negative adjustment factor).
   3. Apply caps to individual marks. For example where there are high marks it is possible to select ‘grade cannot exceed max points’ to prevent a score higher than is available for the assignment being generated. 
      Make any changes as necessary and review how the marks are affected - this will be detailed on screen. Changes will not impact student grades until you proceed to the next step.
6. Scroll down and click ‘Accept and Continue’ once satisfied with the final marks. You are then given the option to either update existing grades or create new grade.
• Update existing grades: This option will update the existing Gradebook column and the marks that have been entered into it.
1. To update the existing grade ensure the correct assignment is highlighted and then scroll down and select ‘update existing grades’. Note that there is an option to take a back-up of the original mark (recommended).
• Create new grade: This option will create a new additional gradebook column and keep the original group mark in the original gradebook column alongside it.
1. Note: this will also create a new assignment that will automatically publish in Canvas (this is required to create a gradebook column). Setting the until date in the assignment settings to a date in the past will hide this new ‘assignment’ from students.
2. Enter a name for the column (e.g. Buddycheck adjusted final mark for xxx).
3. Click ‘create new grade’ at the bottom of the screen. Buddycheck publishes the grades to the Gradebook, but does not post them to students. The grades need to be separately posted in the gradebook.
7. Select Download backup which will keep a record of your previous marks and help you if you want to restore the grades (if concerned you might also download a copy of your Canvas module gradebook).
8. In the Canvas grades area, note that the students will receive a comment telling the student that their grade was adjusted and by how much.
9. Consider changing any grades and feedback relating to the change in grades made by Buddycheck.

• The Buddycheck adjustment factor

  About the adjustment factor

  The adjustment factor is a value calculated by averaging the students’ scores across all question categories and dividing this by the average of the overall rating each team member received. It can be used as a percentage if multiplied by a hundred. 

  The adjustment factors is meant to be used to evaluate how well students can work in teams.

  This factor can be used to adjust the grade of a student in the Canvas grades area. The About the Buddycheck Adjustment Factor document includes more information on how the adjustment factor is calculated.

  How is Adjustment Factor is calculated

  An Adjustment Factor value is a calculated number that attempts to show the contribution of a given student relative to the other members of their team. Specifically, this Adjustment Factor is the average rating of the student divided by the overall average rating for all members of the team.  

  This video is from a previous version of Buddycheck but describes in detail how the adjustment factor is calculated.

  With or without self?

  With Self means that the student's self-ratings factored into the adjustment factor. In some cases, exceptional conditions such as 'Cliques' may have an impact on the validity of the Adjustment Factor values, and in these cases, the Adjustment Factor columns will be highlighted.  

  Original or capped?

  Tutors have the ability to choose between the original or capped adjustment factor. By default, the capped adjustment factor value is capped at a maximum value of 1.05. Also, values above 0.95 are rounded up to 1.00 (their findings are that values in this range are just 'noise'). You can get an explanation of the capped adjustment factor by hovering over the switch on top of the results table. 

  What if a student hasn’t completed an evaluation? 

  If a student does not complete the evaluation, there is less peer evaluation data in the group of this student. This means that the results will be less reliable especially if it is a small group or if there are more students in the same group that have not completed the evaluation. That being said, an average and an adjustment factor of a student is simply calculated using completed evaluation data from the peers in the group. So, you can just use all results even if 100% of students have not completed the evaluation.

  Students that have not submitted do get an average and an adjustment factor because peers probably did complete an evaluation with this person included. 

  Using weighting to limit the impacts of adjustment factors

  Weightings can be used to limit the impact of the peer score derived adjustment factors on individual marks, relative to the mark awarded by the tutor for an assessment. The About the Buddycheck Adjustment Factor document includes more information on how the weighting and adjustment factors.

Downloading results

Results can be exported into Excel if you wish. You can then use the adjustment factor (student average/group average) to calculate the final mark for each student. The ‘original’ adjustment factor is given alongside the capped adjustment factor (cap set when setting up evaluation).

Click on the title on the Buddycheck peer evaluation screen.

Select ‘More’ and then ‘Download results’.

In the Export data screen you can choose:

• whether to turn self-scores on or off (we recommend the latter)
• whether to download as CSV or Excel.

See more from Buddycheck