Autograding Math Exercises using Octave and GitHub Classroom

Checking math exercises can be easily done in a automated way, by using GitHub Classroom together with GNU Octave. We have recently discussed how to autograde C++ programming exercises, and now we extend this knowledge for math exercises. This tutorial is intended for college teachers/professors, and if you want to learn by practice, just follow this invitation link: https://classroom.github.com/a/M8rsjRoB

As usual, you need to log-in into GitHub Classroom (https://classroom.github.com) and create a new Organization that will hold the repositories (or “answers”) of all students. Personally, I recommend creating a new organization for every different course, in every year/semester (and name it like MyCourseName-Semester-Year). This way, it is simpler to manage dozens of students in a course, and computational limits/quota from GitHub are also limited to that organization (if too much is used, it won’t interfere with other courses).

After creating an Organization, you need to create a repository with the instructions for the exercise (in a README file), and some file for students to fill (in my case, it’s named student.m). We recommend that this file has the basic function prototypes for each exercise, so that students will have to complete the spaces with the correct answer.

As an example, suppose we have two exercises (from Calculus): (1) compute the derivative of function h(x)=x²+x; (2) compute the integral of function g(x) =x. We could create the following student.m file in GNU Octave (note that we need specific packages for symbolic computation):

pkg load symbolic;
function sol = exercise1()
   syms x;                       # declare symbolic variable x
   dh = 2*x+1;                   # solution is: h'(x) = 2x+1
   sol = function_handle (dh);   # converts symbolic to function
endfunction

If you have GNU Octave installed locally (not necessary), you can invoke exercise1:

octave --eval "source('student.m'); exercise1()"

Output should be:

Symbolic pkg v2.8.0: Python communication link active, SymPy v1.5.1.
ans =
@(x) 2 * x + 1

Now we need to ensure that student program will fail if output function is not the expected one… for that, we can use method assert. We transform output of exercise1 into char/string version of the symbolic function, and check against string “2*x+1”, which is the correct answer:

octave --eval "source('student.m'); assert(char(sym(exercise1())), '2*x + 1')"

This way, the student code will only work if it gives precisely the expected function (including blank spaces). Note that this exposes the solution to the students (if they take the time to inspect GitHub Autograding files), so another possibility is not to check the precise function, but a few expected values over it. For example, instead of explicitly requiring some h’(x)=2x+1,we test against any function h(x) that returns 21, given x=10, and that returns 101, given x=50:

octave --eval "source('student.m'); assert(exercise1()(10), 21); assert(exercise1()(50), 101)"

This way, student cannot know the correct symbolic answer, even by inspecting the hidden GitHub Autograding files. This also prevents symbolic issues with representations including blank spaces and equivalent answers, such as h’(x)=1+2x, that wouldn’t be recognized as a correct answer. Feel free to include more assertions and precisely describe the expected answer to each exercise.

What is the solution for exercise 2? Please accept this invitation and try to fix the student.m file generated for you. Remember that a “red x” indicates that GitHub Autograding has failed to execute student file, and a “green mark” indicates that all tests successfully passed.

Back to the instructors side… I’ll show how to create an assignment. Enter the classroom, hit “create assignment” and give it a name (in this example, it’s called exercises-math-octave). Then, select the repository that you have created for the README and the student.m file (in this example, it’s called basic-math-octave). To add tests, click on Add Tests (select type Run Command).

Autograding of type Run Command

The first test is named ex1, and has the following setup command (installs gnu octave):

sudo apt update && sudo apt install octave octave-symbolic

For the Run Command, we use the command described before:

octave --eval "source('student.m'); assert(exercise1()(10), 21); assert(exercise1()(50), 101)"

Let’s give 3 minutes maximum to this exercise, and a grade of 50 points:

Creating test case for exercise 1

Now repeat the process and create the test case for exercise 2.

Creating test case for exercise 2

After adding tests, just click “create assignment”

Now, you may copy the invitation link and send to the students.

Invitation link for the activity

As a student, this is how the activity appears after accepting the invitation link:

Accepting invitation link

Wait a few seconds to generate the student activity and refresh the page

Activity is successfully generated for student

The yellow dot indicates that autograding process is being executed

The red cross indicates that some activities/test cases have failed

Clicking the red cross indicates that we already have 50/100 points (exercise1 is correct, but exercise2 is not)

Student may click on student.m file and hit the pencil to edit it online (no need to install GNU Octave on local computer)

When correct answer is given on student.m, a green mark appears

That’s it. I hope you have learned how to create math exercises using GNU Octave and GitHub Classroom. Feel free to leave questions and to practice on the given public activity. See you soon on a next post!

Igor Machado Coelho is Computer Science professor at Universidade Federal Fluminense since 2020.