Let's get straight to the point. Scio is taking online college exams in Exceptional position. As one of the few in Europe testing:
Companies usually only deal with one part. Since we have both pieces in our hands, we have 100% control over the rules and we decide what role we play with artificial intelligence. And the role assigned to her is to highlight potentially troublesome moments during the exam, but never to decide whether or not such a moment was actually a violation of the rules. Artificial intelligence escapes the context of human action. It identifies a suspicious situation, but is not always able to determine exactly why the situation occurred and cannot make decisions in marginal situations. For this reason, the control is overseen by experienced human evaluators, who have the final say in the final decision on the exam.
Candidate before starting the test, legitimizes with the help of a webcam -- shows ID and then himself. The photo of the pass is checked by the assessor as in the case of the full-time exam. Thanks to this, we know that the right person is taking the exam. The candidate then uses a webcam to film the surroundings of his workplace, i.e. the table and the room in which he takes the exam. “This shows that he does not have any unauthorised devices ready anywhere and there is no friend nearby to advise him.” explains Jáchym Vintr, head of development of the ScioLink test application.
Before the test begins, ScioLink provides desktop sharing, monitor events, and external display connections, which verifies what is happening on the monitor or if an external help screen is connected next to it. And then time runs out, and the applicant at ScioLink answers the questions. “Both the webcam and microphone are running throughout the testing. At the end of the test, ScioLink stores both the answers and all video and audio recordings. And the tracking ends there.” describes Vintr.
Assists. It helps. Practically, this means that AI marks suspicious moments from the course of the exam. For example, if a candidate moves away from a job during a test, pulls out a cell phone on which he is looking for help, or violates the rules in some other way, the AI will mark that passage of the record with a “flag”.
“We have deliberately “trained” artificial intelligence to be more sensitive. It marks the slightest hint that something suspicious has happened. Perhaps even self-apology, when the candidate says to himself what counts,” gives the example of Vintr. We have a sensitivity set this way because after artificial intelligence everything is controlled by man.
Let's say that even with the initial preparation, the exam lasts an hour and 500 applicants take part in it. This creates 500 hours of recording from webcam and microphone, as well as 500 hours of recording of events on the monitors of individual applicants. But, for example, the annual reality of our National Benchmarking Exams represents an even greater volume of data, and it is virtually impossible for the control to be based solely on people going through individual records from start to finish.
Artificial intelligence will mark suspicious places and only then Human evaluators are in control.. “We have a team of more than 100 people, among which the individual records are divided. They follow them mainly in accelerated mode so that they can concentrate fully on the marked suspicious locations. We train the evaluators, give them feedback, and the algorithm divides the records for review according to the key, which ensures randomness and prevents the same evaluator from reviewing the records of the same applicant.” points out Vintr.
Unlike most foreign solutions, we do not allow AI at all for the initial identification part of the exam. “Applicants often have an ID with a photo of their fifteen year old. Now they're in their 20s, and AI all too often sees the problem where it's not.” reveals Vintr. The validity of the document, the matching of the photo with the applicant, as well as the recording of the surroundings of the workplace is always checked by a person in ScioLink.
Human evaluation takes place at several levels. If the evaluator finds that the AI has used the flag incorrectly, they will remove it with one click. If the AI is right and determines that there really has been a violation of the rules, the record goes to the commissioner, who either confirms the assessor's verdict or corrects it (there is even a third instance for online National Comparative Examinations, the so-called Appeal Commissioner).
After checking all records, we send the results to the faculty, which sets the line — accepted/not accepted. And for applicants in whom we have uncovered wrongdoing against the rules, note describing where they erred. The final verdict, however, is again up to faculty representatives.
Over the past three years, via ScioLink, the Over 60,000 online college exams. “We made it possible for many students to take the exam from home at an important moment in their lives. They avoided complicated travel and unnecessary stress. ScioLink has also made things easier for a lot of faculties, so I am convinced that online proctoring is the right direction,” summarizes Vintr and adds: “Transparent rules will only help.”
We welcome and closely monitor the current debate on the regulation of artificial intelligence in education. We agree with the opinions that:
However, we do not agree that the use of AI in education is universally perceived as harmful. If artificial intelligence does not make decisions on its own, but serves only as an assistant, it brings a lot of positives to education for students and universities.