A number of the sessions at Educause 2015 shared the important, but often unexpected, pedagogical benefits of online testing and assessment. These sessions dealt how the challenges associated with online testing, such as cheating, and how to address and eliminate them. Whether the testing group is small or large (where large can mean 1,200 students at a time) and whether using school-owned devices or BYOD, there are now ways to insure fair and efficient assessments that provide more insight into student understanding as well as feedback to the teachers, than ever before possible. The sessions also emphasized the vital role of network analytics in online testing.
Rob Peregoodoff, director of learning services at the University of British Columbia’s Sauder School of Business, described how they have eliminated cheating without resorting to the prison model of locking down students during testing. All students do not have the same versions of the test, and they are told this. Invigilators constantly keep their eyes scanning the student screens and it’s fairly easy to tell if any screen changes when it shouldn’t. Any invigilator who strays from the task at hand is summarily dismissed, so they get the point. All keystrokes are recorded, not so much to identify cheating as to confirm it and remove all misunderstanding. With these precautions, it is possible for students to use their own devices and still be confident that the test results are fair.
Designing Tests For Online Usage
To get the most benefit from online assessments, the tests themselves can be redesigned to take advantage of electronic media. For example, answer forms can be presented as spreadsheets with highlighted cells for answers (see figure below). Results of the individual questions can be analyzed for level of difficulty and item discrimination index, which indicates if a given question was a true measure of knowledge or simply confused the students. Even more importantly, the tests can identify where the students took the most time and where they may need additional review.
The idea is to move the instructors up Blooms hierarchy. Let the testing software keep track of the students’ knowledge and comprehension; free the instructor for the higher level tasks of application, analysis, synthesis, and evaluation.
Online Testing Disasters and How To Prevent Them
Peregoodoff related a story that demonstrates how important it is to have access to network analytics during online testing. Several weeks prior to Educause 2015, he had volunteered to oversee an online midterm exam to 940 students spread across 21 rooms, including the exam theater on campus, on a Friday evening. As the exam was getting underway, hands began raising and students said they could not access the exam site. The campus phone line to the IT help office was also down. When Peregoodoff did get through to the IT folks by mobile phone, he found they were unaware of the situation. When IT called back later to say they had found there was indeed a problem, Peregoodoff not so patiently explained that he had 940 students waiting to take their midterms, and told them to call back as soon as they knew when it would be solved.
Without network analytics the IT department was not only initially unaware of the problem, but even when they were alerted, they were unable to pinpoint the location of the outage. It turned out the problem was with a core router to BCNET. The campus LAN worked fine and the students used it for karaoke as they killed time awaiting the doomed exam. Had they been equipped with Purview network analytics, they could have constantly monitored the entire network from each student’s device through the school’s internal infrastructure, all the way to the remote testing servers.
The Future of Online Testing
Here’s a glimpse of the future. Finals time arrives; a truck delivers 1,000 tablets with the exam preloaded. Students begin their exams by recording a photo of themselves. They open their exams which provide carefully-controlled access to the different areas of the answer sheets. Their screens include collaboration spaces; that’s another important aspect of education. If someone doesn’t understand the wording of a question, they can use the collaboration areas of the screen to ask. Extra credit is awarded to those who help out their fellow students. A variety of tools are available to the students including keyboards, styluses, and custom applications. While the test is underway, the instructors monitor real time network analytics to see how the students are doing and if any special intervention is required. That last part is available today.