An open proctoring system for remote support of online exams

    Last summer, ITMO University passed a distance examination for graduate students, and in January of this year, students of the National Open Education Platform who wish to receive a certified certificate were given the opportunity to pass the final certification at the chosen course under the supervision of a proctor. All this was organized on the basis of an open remote surveillance system specially designed for these purposes. In this article I will try to review the system, as well as talk about some features of its development.

    Proctor distance test

    The remote monitoring system (hereinafter referred to as the DLS) is intended to accompany the process of geographically remote passage of exams, confirm the identity of the subject and confirm the results of his certification. Of course, this system is not unique, there are foreign services that provide remote assistance services, such as ProctorU , Mettl Online Assessment , ProctorExam or Pearson VUE , but there is a domestic Examus service and the ProctorEdu system. However, the SDN being developed has the main difference - this is not a service, but open source software that anyone can use for their own purposes, whether it is organizing their own proctoring center, conducting certified certifications in their e-learning system, or something else . Of course, for the full functioning of the system you need people (proctors), workstations with cameras and audio headsets, as well as server equipment and system administrators to deploy and maintain the system. But when the necessary infrastructure is already there, the question of software remains. And it is precisely this question that the given system answers.

    Work with SDN is divided into two main parts: the subject’s interface and the proctor’s interface. First, the test subject downloads and runs the application, logs in and sees the list of exams available for him to pass in proctoring mode (the list of exams is created in advance, or is automatically loaded from the LMS via the API). Further, he can plan the time for passing these exams; for each planned exam, the subject is assigned the currently available proctor. Available time is formed on the basis of the schedule, which are made by the proctors, they have an appropriate interface for this. Next, the student at the time indicated by him through the application connects to the proctor and the exam begins. Proctor accompanies the test subject throughout the passage of the exam, his responsibilities include conducting an identification procedure and tracking violations of exam rules. After completing the proctoring session, the student sees the proctor’s opinion on the exam and his comment.

    Subject interface prior to exam
    Subject interface prior to exam

    Proctor also works with SDN through the app. The main interface of the proctor is the exam schedule table, where he sees all the exams for which he is assigned, and their status, which changes in real time. When a student connects to the exam, the proctor sees this, connects to it, and the exam begins. Proctor sees the test subject through a webcam, and also sees his desktop. Before the exam begins, the proctor must conduct an identification procedure in which the student shows an identity document, and the proctor checks the specified data in the subject’s profile with a photograph of the document and the subject’s face on the webcam. Upon successful identification, the data and photo of the document are saved in the exam report, and the examinee is given access to the exam in the LMS. After that, the subject enters the LMS and proceeds with the assignments. Throughout the exam, the proctor can take notes and save screenshots in the exam report, which may be useful in resolving disputed issues in the future. At the end of the exam, the proctor signs the protocol and makes a conclusion, which is displayed to the student and transmitted to the LMS by API. In case of violation of the rules for passing the exam, the proctor may interrupt the exam indicating the reason for the early completion of the exam. A proctor can simultaneously monitor several subjects, the number of simultaneous sessions is limited by the proctor's capabilities, his computer's resources, and network bandwidth. which may be useful in resolving disputes in the future. At the end of the exam, the proctor signs the protocol and makes a conclusion, which is displayed to the student and transmitted to the LMS by API. In case of violation of the rules for passing the exam, the proctor may interrupt the exam indicating the reason for the early completion of the exam. A proctor can simultaneously monitor several subjects, the number of simultaneous sessions is limited by the proctor's capabilities, his computer's resources, and network bandwidth. which may be useful in resolving disputes in the future. At the end of the exam, the proctor signs the protocol and makes a conclusion, which is displayed to the student and transmitted to the LMS by API. In case of violation of the rules for passing the exam, the proctor may interrupt the exam indicating the reason for the early completion of the exam. A proctor can simultaneously monitor several subjects, the number of simultaneous sessions is limited by the proctor's capabilities, his computer's resources, and network bandwidth.

    Proctor Interface Exam Table
    Proctor Interface Exam Table

    Proctor Interface, Identity Procedure
    Proctor Interface, Identity Procedure

    The general architecture of the SDN is shown in the figure below. All technologies used are open and distributed under free licenses.

    SDN architecture

    The system is completely written in JavaScript, the cloud IDE Cloud9 was used as the development environment . The main parts of the system are a server on node.js and express, a MongoDB database with storage of small files in GridFS, a Kurento media server, a media archive storage (video recording of proctoring sessions) with WebDAV access, and a node-webkit desktop application (as an alternative the application can use the special Chrome extension to work from a web browser).

    WebRTC protocol and Kurento media server were chosen as the basis for video transmissionto organize and manage media streams between clients, as well as record these streams. At the moment, Kurento meets all its needs both in terms of functionality and performance, however, other similar solutions worth mentioning are Licode and Jitsi Videobridge .

    Authorization in the SDN is based on the PassportJS module; currently login with a username and password or through OAuth is supported. If necessary, you can add an authorization strategy from those already supported in PassportJS or write your own, as was done for authorization through the ITMO University single sign-on system (a special passport-ifmosso strategy has been written for this ).

    The client is implemented as a single-page web application that can run in a web browser, however, the screen capture function and screenshots are only available after installing the special Chrome extension or through the special node-webkit application , which is available for Windows, Linux and OS X platforms. the application is especially useful if you don’t want to get attached to the web browser and its versions, where, in addition, third-party extensions can be installed that can affect the operation of the application in the browser, which is undesirable.

    DLS does not contain the functions of a Learning Management System (LMS), but can be connected to it through the API. To interact with the National Open Education Platform(NPOO) a special API has been implemented that can be extended to interact with other LMS. The figure below shows a diagram of the interaction of system participants using this API.

    Diagram of the interaction of participants of the SDN
    Within the framework of one session, several video streams are transmitted: the subject’s camera, the proctor’s camera and the subject’s desktop. The parameters of each stream are set in the application settings, where you can set the video and audio source, maximum resolution and frame rate. To check the quality of communication and pictures, there is an appropriate interface, which allows you to know in advance about possible technical problems.

    All video and audio streams of the proctoring session are recorded. Video recordings of sessions are stored on a separate storage with access via the WebDAV protocol, which allows you to logically separate the storage from the system itself. Thanks to the WebDAV protocol, for storing videos, you can, for example, use Yandex.Disk . Later, these records and exam records can be viewed through the DLS interface. To transcode the videos and upload them to the server, a special module is written, the description of which can be found in a separate article .

    The application uses the exact server time, which is independent of the computer clock; for convenience, the time is adjusted taking into account the client’s time zone.

    To access the VDS database, a special JSON REST API with flexible configuration of access rights and data output parameters is implemented. For example, you can configure token access to a trusted server that will aggregate and process statistics. This API fully supports CRUD (create, read, update and delete) for working with MongoDB collections based on the Mongoose models created in the SDN, and the original Mongo query syntax is also supported.

    Minimum Technical Requirements

    ParameterMinimum requirements
    Client part
    operating systemWindows XP +; OS X 10.7+; Linux
    CPUIntel i3 1.2 GHz or equivalent
    Network connection speed1 Mbps
    Free disk space100 MB
    Free RAM1 GB
    Webcam resolution640x480
    Webcam frame rate15 frames / s
    Monitor screen resolution1280x720
    Server side
    operating systemUbuntu 14.04 (64 bits)
    CPUAMD Six-Core Opteron 2427 2.2 GHz or equivalent
    CPU average load5% / session
    RAM2 GB + 100 MB / session
    Network connection1.5 Mbps / session
    Burning to disc150 KB / s / session
    Disk space10 GB + 500 MB / hour / session
    Video archiving100 MB / hour / session (5 frames / s with sound)

    At the moment, about 120 proctoring sessions have been conducted through the system, but this year it is planned to hold several thousand sessions. The system can be deployed independently, or use the ready-made Vagrant box. Instructions and source codes are posted on GitHub .

    Media Mentions


    Also popular now: