The AutoGrader is currently a private beta feature. Contact us if you would like to request access.
The AutoGrader uses an independent service to automatically grade Jupyter Notebook files. Then the grade is posted to the course's
Grades section where users can view assignment grades and the submitted files for further grading and/or commenting.
The general workflow to prepare a Jupyter Notebook for auto-grading consists of the following steps:
The AutoGrader from IllumiDesk is heavily influenced by the awesome nbgrader project. We recommend looking through their docs to gain a better understanding of how to set up assignment solutions and tests.
The first step to enable the AutoGrader cells is to define an AutoGrader solution and assign the
Autograded Answer value to the cell. After you have defined a solution, define the tests that validate the output. Remember to add the
Autograded tests cell metadata value for the tests:
AutoGrader solution and test cells
Notice how the total for all tests appear in the
Total Points section of the Notebook's toolbar. Take note of this number once you have completed setting up your Notebook as this value should be considered when defining the total grade for the assignment.
There aren't any additional steps when adding an
Assignment with AutoGrading enabled in the Canvas course. It's just important to note the following:
- In many cases it is useful to combine auto graded and manually graded cells in an assignment. For these use cases, define the cell meta data accordingly.
- The grader can override the grade calculated and posted by the AutoGrader service. For example, if the solution from a student deserves extra points, then the grader is free to update the calculated grade within the assignment's grade section.
- AutoGraded tests need to be setup like unit tests. Therefore use manually graded tests for output types that cannot be validated with unit tests, such as visualizations.