AutoGrader Setup


The AutoGrader is currently a private beta feature. Contact us if you would like to request access.

The AutoGrader uses an independent service to automatically grade Jupyter Notebook files. Then the grade is posted to the course's Grades section where users can view assignment grades and the submitted files for further grading and/or commenting.

The general workflow to prepare a Jupyter Notebook for auto-grading consists of the following steps:

  1. Create an assignment within the FormGrader tab
  2. Create Jupyter Notebooks with solution and answer cell meta-data
    3. Release Jupyter Notebook as an assignment
  3. Create an assignment within the Canvas LMS's course

AutoGrader's Roots

The AutoGrader from IllumiDesk is heavily influenced by the awesome nbgrader project. We recommend looking through their docs to gain a better understanding of how to set up assignment solutions and tests.

Define AutoGrader Solutions and Tests

The first step to enable the AutoGrader cells is to define an AutoGrader solution and assign the Autograded Answer value to the cell. After you have defined a solution, define the tests that validate the output. Remember to add the Autograded tests cell metadata value for the tests:

AutoGrader solution and test cells

AutoGrader solution and test cells

Notice how the total for all tests appear in the Total Points section of the Notebook's toolbar. Take note of this number once you have completed setting up your Notebook as this value should be considered when defining the total grade for the assignment.

Define AutoGrader Assignment In Canvas

There aren't any additional steps when adding an Assignment with AutoGrading enabled in the Canvas course. It's just important to note the following:

  • In many cases it is useful to combine auto graded and manually graded cells in an assignment. For these use cases, define the cell meta data accordingly.
  • The grader can override the grade calculated and posted by the AutoGrader service. For example, if the solution from a student deserves extra points, then the grader is free to update the calculated grade within the assignment's grade section.
  • AutoGraded tests need to be setup like unit tests. Therefore use manually graded tests for output types that cannot be validated with unit tests, such as visualizations.

AutoGrader Setup

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.