Yesterday’s post was about how my teaching has changed since the pandemic hit. One big thing that I’ve left out of it is my approach to midterm and final exams.
The suggested approach to invigilating (proctoring) online final exams from Central Connecticut State University (CCSU) was to use a Blackboard plugin called Proctorio. I tested it out with a few willing students, but decided that it was way too intrusive.
Because I’m in the good position of having relatively few classes and each class is generally capped at 24 students, I decided to go with online oral exams.
There are several pieces to this that need to be unpacked, so I thought I’d write out what I used to do for finals and midterms first. And then look at what’s changed.
Finals in the before times
My final exams before COVID-19 were pretty standard fare:
- Ninety minute unseen exam,
- Written answers,
- One sheet of Letter paper of notes,
- Calculators or matlabTM (on one of the lab computers),
- No text book,
- No internet, and
- No communication with others.
Let’s examine all those items one-by-one.
The ninety minute limit forced me to write papers of a given length. I generally aimed to have half the class be able to finish them in an hour.
All students had to take the exam at the same time, so I needed a room with enough space to “proctorially” distance students. The whole session took about two hours.
The trouble then, is that I needed to collect the exam papers and grade them.
I do give the students a sample exam, but the exam they get on the day is completely unseen by them. The topics and general difficulty are roughly the same as the sample, but everything about the exam is new to them otherwise.
The students write their answers either on the exam sheets themselves or on booklets provided if the exam is more text-based than numerical.
One sheet of Letter paper of notes
From a pedagogical viewpoint, I think that letting the students make their own course summaries is a good way to prepare for exams. So I let them bring in something of their notes.
Calculators or matlabTM (on one of the lab computers)
My more electronics-focused courses require some numerical work, so the students need to use calculators. Some of the work requires solving matrix equations with complex-valued entries, and some student calculators aren’t up to it. For that reason, I also allow (encourage?) the use of matlabTM.
No text book, No internet, No communication with others
These are all fairly standard restrictions.
How long does that take?
Unless you’ve graded an exam before, you may not realize something: really excellent and really poor papers take almost zero time to grade. It’s the other eighty percent that take time: you have to find arithmetic errors and check that their eventual answer is correct; you have to read through stream of consciousness to see if they do actually say something relevant.
Let’s look at a single class of 24 students. My time taken up is:
- 120 minutes for the exam
- 24 x 10 = 240 minutes to grade all exams (assuming 10 minutes average time)
- 30 minutes to enter grades into the Learning Management System (LMS), which is usually Blackboard.
This gives a total of about 390 minutes of my time or six and a half hours.
Sometimes, it takes me longer. This is because I can’t just sit down and grade 24 papers in one sitting. I know my sister laughs at me for this, but I just can’t. Alex’s class sizes are significantly bigger than mine, so she has to deal with an order of magnitude more student exams. But, that I need to take breaks means that getting back into the grading groove takes extra time.
Finals in the after times
My final exams after COVID-19 are as follows:
- Fifteen minute oral seen exam,
- Immediate feedback to the students on the answers to the oral component,
- Written answers to some questions, uploaded electronically into the LMS,
- Access to whatever they want during the oral exam, provided they answer questions in a timely manner
- For the written component, calculators or matlabTM (on one of the lab computers), and
- Exams are recorded.
The nice thing about an oral exam is that I get a one-on-one interaction with the student. This lets me attempt to calm them if they are nervous, and gives me a much clearer picture of who they are as a student for my course.
The sessions are generally fifteen minutes, though just as for written exams, this can be less for better (and worse) students.
One thing that I am still working out is how to assess students. One thing that I’ve noticed with students (going all the way back to my first teaching experience in the 90s at ANU) is that even good students can be put off their A game if they are surprised or discomfited by something on the exam.
So, I release my oral final exams approximately two weeks before the exam.
One question type that I’ve come to favor is to create a long list of topics, say 20, in a particular area covered by the course. And then ask the student to explain eight of them: four of which they can choose, and four of which I will choose.
This adds some uncertainty to the exam, but doesn’t seem to scare as many students as a completely unseen exam.
But the students will all get 100%
Look, let’s face it, I’m a student. I’ve pretty much always been a student. If a professor gave me the exam paper two weeks before sitting the exam, I’d nail it.
Some students do that.
But many students don’t. Many students still struggle with organizing their approach to subject matter. Some students can’t easily deal with oral communication.
The only students who seem to benefit from this approach are those students who would be getting an A anyway. And the benefit, for them, is that they can take less time to get there.
One thing that bugs me about written exams, particularly written final exams, is that there is no real way to correct the student.
With the oral exam format, I can give qualitative feedback on all the student’s answers immediately. The hope being that they’re more receptive to the feedback in the moment.
Written answers to some questions, uploaded electronically into the LMS
In the before times, scanning and uploading the students’ answers would have taken too much time. But, with the COVID-19 situation and the advent of easy-to-use scanning apps like Adobe Scan, I can get the students to upload electronic versions themselves.
For most electronic formats, this means I can just view the document in the LMS.
Access to whatever they want during the oral exam
The exam rubric includes a part for timely answers. If the student has to flick through notes or do a Google search to answer… then they are penalized. However, most professionals in employment use Google or other information websites on a daily basis. So I see little reason for completely disallowing access.
I’m even tempted to teach a lesson about Google-Fu : the fine art of constructing a Google query.
Calculators or matlabTM
The post-oral component of my electronics courses tend to require use of these. Thankfully, CCSU makes matlabTM available via its Citrix virtual servers, so students can access them remotely.
Exams are recorded
Sometimes I get questions from students about their performance on exams. In the before times, the only record I had was their written submissions. Because I record all exams now, I can point to very specific things that the students said or did to give feedback to these queries.
How long does that take?
Unsurprisingly, these exams take longer.
The calculation is:
- 24 students x 15 minutes per student = 360 minutes for the oral component
- 24 students x 5 minutes per student = 120 minutes for grading the written component
This gives a total of 480 minutes, about an hour and a half longer than the usual approach.
However, the benefits I get are worth it:
- I get to see each student individually, and they get to see me,
- I can use the feedback component to help teach, and
- I get electronic copies of everything.
Another tool: Bookings or Calendly
Another tool that helped me do this is the Microsoft Bookings tool which integrates with my Outlook calendar. It lets me set up timeframes to allow students to book their own exams, and for me to block out parts of my day when I don’t get exams.
For those without the right version of Microsoft tools, Calendly provides similar functionality.
I’m going to keep doing it
I’ve decided that, even when I go back to in-person teaching completely, I’m going to keep doing midterm and final exams this way. I think the pedagogical and logistical benefits far outweigh the costs.
The only slight caveat is that this approach doesn’t scale. If I had to deal with 240 students per class instead of 24, the numbers may not add up.