Scantron Drama
For our very large courses, we use multiple choice examinations. I should say I would prefer to have short essay style examinations, however, the idea of marking almost 600 papers would be a big reason why this is not the evaluation method we use and why I am not going to pretend that I think it would be a good idea. Although scantrons are not without their work, it is still better than marking written work (hence my mid day post….the stack of written marking is piling up and I’m choosing to ignore those at the moment).
However, students seem to think that marking should be completed in a day or less with scantrons. They couldn’t be more wrong. Certainly, I did not come in over the weekend and submit my scantrons from Friday nights exam. But after I dropped these off (after counting all booklets, nominal rolls and scantrons to ensure the numbers matched), it was a matter of hours before the data popped up on my computer on Monday afternoon. Since then, it’s a painstaking task of evaluating the exam. So student, if you want to know what happens after you write a multiple choice exam, here it is:
1. I record the difficulty rating and point biserial value of each question (this tells us how hard each question rated and if the question was “good” based on the students who answered correctly and also did well on the exam), a lovely software program that doesn’t just report the grade of each scantron.
2. Highlight any distracting answers that students picked instead of the correct answer to see if there was an issue with wording, or if it was just a good distractor.
3. Email all professors in the course who taught and asked them to individually evaluate each question they wrote based on the statistics to identify if there are any corrections that need to be made.
4. Review the front of each exam booklet for students notes to us about the exam. Which questions were they confused about, was there issues with interpretation/wording or any answers in which they thought there was multiple correct answers.
5. Email that information to each professor who teaches to see if the comments are valid.
6. Collect all responses from the teaching faculty and then determine what corrections if any need to be made (always in favour of the student in the upward direction).
7. Go through the scantron files to see if there are any errors (students didn’t bubble hard enough on the scantron, or missed answering a question)
8. Find their original scantron to see if the answer didn’t show up on the electronic file were due to not bubbling in correctly, or because they didn’t fill it in.
9. If the latter, go back to the booklets (which have yet to be alphabetized), find that students booklet and see if they did indeed circle an answer for that question and just forgot to transfer. If yes, give the student the benefit of the doubt and fill it in the electronic file.
10. Manually fill in an electronic scantron for all students who decided to answer the original scantron in ink, 2 students filled their entire scantron in ink….. facepalm.
11. Ensure that each student number was correctly filled in and that their mark actually transfers to the learning management system. This midterm, 7 students didn’t fill in their student number correctly….
12. Ensure that students who obtained an extremely low mark, didn’t accidentally bubble in the wrong exam key code. Go back to the exam booklets and see if the student wrote version 1 or indeed wrote version 2. If they filled in the wrong version on their scantron, remark their scantron with the correct exam key. 2 students did that this midterm and achieved radically higher marks when marked with the correct answer key.
13. Use the cheating analysis software to identify any possibility of cheating. If there is, go back to the nominal call sheets and identify if any proctors made notes about these students and if these students were sitting close to each other. Speak with the Undergraduate chair regarding any situations of cheating.
14. Alphabetize the exam booklets for pick up. 600 exam booklets = hours of alphabetizing
15. Double check that the correct grades were posted…I never fully trust the transfer of grades from our scantron program to our LMS.
16. Check that students without grades did email me that they were unable to write both the exam and make-up. For students I haven’t heard from, check the registrar to see that they are still enrolled or have dropped the course. Email the students who didn’t write the midterm and haven’t dropped the course to see why they didn’t write the exam or the makeup or why they didn’t email.
17. Post grades.
18. Wait for the emails….
Follow