From data subjects to data suspects: challenging e-proctoring systems as a university practice

1

Lately, teaching has had to adapt to fundamental and urgent shifts. After more than two years into a global pandemic and several COVID-19 variants, life is progressively returning to normal. The majority of governments have lifted several, if not all, restrictions. This trend is valid for academic life as well. With Higher Education Institutions (HEIs) shut down for a large part of 2020 and 2021 and engaged with dual delivery and gradual return to in-person teaching in 2022, the academic year 2022/2023 is seeing a general resumption of on-campus activities. However, it is unlikely that things will return to exactly as they were before the COVID-19 pandemic. Firstly, as public health experts warn, COVID-19 is still “a global health threat”. [1] Hence, we might need to remain flexible and be prepared to face future emergencies. [2] Secondly, education has seen a paradigmatic digital shift over the past three years. Several investments have been made, new service providers have entered the market and offered additional services, staff have been trained on these services, further teaching methodologies have been developed, and several have proven pedagogically helpful or simply more efficient in addressing some issues (e.g., the lack of teaching spaces). Hence, some tools introduced during the pandemic are likely to remain.

2

This might be the case with e-proctoring systems. These are technologies designed to monitor student behaviour during online exams. Their function is to replicate in-person invigilation and guarantee the integrity of exams. [3] However, the extensive intrusion into the private sphere of students and numerous publicised cases of discriminatory outcomes [4] have raised several questions about these tools.

3

Used for many years now in some parts of the world, online proctoring software entered European HEIs during the first COVID-19-related lockdowns. Faced with urgent rules, HEIs were forced to reflect on how to guarantee the integrity of online exams, opting in some cases for e-proctoring solutions. This brought the conversation on the credibility, necessity, and reliability of e-proctored assessment methods to the forefront of academic discourse.

4

The growing use of e-proctoring tools in European HEIs during the pandemic is confirmed in an explorative study the authors conducted between April and July of 2021. [5] The research targeted 38 HEIs in the United Kingdom, Italy, and the Netherlands, collecting 194 responses to a 32-question survey. It resulted that 13.10% of educators have been using e-proctoring systems during the pandemic, and 8.70% were offered the possibility to opt-out and choose a non-e-proctored alternative. [6] These results cannot be generalised, but they signal the emergence of e-proctoring usage among traditionally non-distant education providers. At the same time, they show that e-proctoring was not the only means to guarantee the integrity and validity of exams, as a large part of the respondents organised online exams without remote invigilation.

5

By now, most universities are back to on-site exams. However, the possibility of yet another upsurge of the coronavirus has led a few HEIs to ensure that formal rules for reinstalling online proctoring processes are in place for when the circumstances might make it necessary. These rules, incorporated, for example, as Examination Board rules and responsibilities, describe the framework for organising distance (written) exams with online fraud prevention measures. While almost lifted everywhere, pandemic-related restrictions have left long-standing traces in the functioning of HEIs and the performance of educational activities. This legacy makes the critical evaluation of e-proctoring systems a necessary exercise for the determination of academic education imaginaries in a hybrid future.

6

The paper aims to map the legal landscape of e-proctoring in the EU. To this end, this contribution provides a brief overview of the technical aspects of e-proctoring systems (Section 2) and existing literature (Section 3) to map the state of the art of the debate. Section 4 critically discusses the case law consolidated over the past two years around e-proctoring systems, identifying points of convergence and divergence between the decisions. Section 5 reflects on the role of collective actions to enforce data protection rights and on the limited role it played in the e-proctoring controversies. After the submission of this contribution for review, a preliminary decision in the field of anti-discrimination law was issued for the first time in the Netherlands. Section 6 includes this relevant update and focuses on the legal means beyond data protection law for countering the discriminatory effects caused by the adoption of some e-proctoring tools. Section 7 sums up the results of the research.

7

E-proctoring systems include a set of methods, software, and devices to monitor students at distance during an online test or exam. Online proctoring systems were already developed and used before the pandemic. [7] This was not only the case for massive open online courses (“MOOCs”) and online HEIs but, in some circumstances, also traditionally non-distant learning institutions were relying on them (e.g., to organise computer-based tests at universities for a large cohort of examinees or to introduce more flexibility for some categories of students, such as athletes). [8] However, during the pandemic, their use has become much more widespread as, in some cases, it was considered the only available solution to perform exams and ensure their integrity.

8

Nowadays, various third-party commercial options are specifically designed to manage online exams and remote student invigilation. In principle, such tools enable HEIs and staff members to verify the student's identity at the beginning of the exam, monitor their activity, set up technical restrictions on their computer (e.g., block browsing during the exam or disable copy-paste shortcuts), remotely control and manage the exam and generate a report out of the monitoring activity. [9]

9

With reference to the invigilation modalities, Hussein et al classify e-proctoring tools into three main categories: live proctoring, recorded proctoring, and automated proctoring. [10]

10

The first solution, live proctoring, essentially replicates the physical surveillance but via webcams and microphones. Here, a physical proctor remotely verifies the student’s identity at the beginning of the exam and monitors their video and audio during the whole duration of the session. In some cases, the invigilator can require a video scan of the workspace to verify that the student does not have any forbidden material at hand.

11

The second category, recorded monitoring, involves capturing and storing students' video, audio, computer desktop, and activity log for a subsequent human check.

12

Finally, automated proctoring relies on artificial intelligence systems to verify, for example, students’ identities via a biometric recognition system and/or to automatically detect suspicious behaviours. In this latter case, the algorithm processes students’ data (e.g., eye or facial movements, voice, keystroke loggings) and environmental data (such as background noise and the presence of other people in the room) to spot signs of cheating. [11] In case of anomalies, the system flags the issue for human review—usually the professor or the trained proctor—or can automatically terminate the assessment.

13

Many e-proctoring services usually offer a combination of the features mentioned above. As this brief overview suggests, there are various levels of intrusiveness in the students' personal sphere depending on the proctoring modalities or the adopted settings. In any case, they all process personal data (relating to the examinees, the examiners, and potentially third parties entering the room), thus triggering the application of data protection law. In the next Section, we will outline the risks and legal issues raised by e-proctoring as emerging from the literature.

14

The increased use of e-proctoring systems has raised several concerns, including from a legal perspective. In the literature, many authors have stressed the potential clash between the use of such tools and fundamental rights and freedoms, particularly regarding the right to privacy, data protection, and non-discrimination.

15

For instance, it has been emphasised that the use of e-proctoring tools is likely to create or foster inequalities, e.g., for disabled people (who can be penalised by the anti-fraud system because they need to use screen readers or dictation software), [12] people with caring responsibilities (whose exam can be disrupted if the person they care for requires their immediate attention), or low income students who might not be able to afford suitable technical equipment, a reliable internet connection, or a room of their own. [13]

16

Moreover, the risk for ethnic minority groups is particularly high when using facial recognition technologies. Several studies have shown that such software is often trained on biased datasets and are systematically better at recognising white people, and particularly white men. [14] Hence, negative consequences for certain groups may occur due to the error rates of such tools or their deployment in a particular context.

17

Concerning privacy, the tracking of students’ activity increases the risks of surveillance. [15] In this respect, scholars have warned against the chilling effect that pervasive monitoring can have on “students’ intellectual freedom” [16] and their educational privacy. [17]

18

Scholars have also expressed serious concerns about e-proctoring from a data protection perspective. The automated decision-making process performed by such software can impact examinees in a significant way (i.e., the suspected behaviour can be reported, or the exam can be automatically terminated), and it remains unclear to what extent the ex-post human review is an appropriate guarantee in practice. [18]

19

Moreover, unlike its analogic counterpart, e-proctoring technologies inevitably generate new data and favour their collection and storage. The retention of such amounts of data increases, as a consequence, the risks of the re-purposing and sharing of data without the data subject’s awareness. [19] Such risks might range from situations where the HEI has an obligation to disclose such information to the commercial uses performed by the e-proctoring tools or to data breaches. [20]

20

Data security is, indeed, another critical point highlighted in the literature. Security concerns are even more worrisome considering the intimate nature of the data processed via e-proctoring (i.e., exam results, but potentially also biometric data). [21]

21

Furthermore, the potential threat to fundamental rights caused by e-proctoring is directly recognised in the AI Act proposal, [22] where AI systems intended to assess students or determine their access to educational programs are classified as high-risk (hence, subject to stringent rules for their authorisation). [23]

22

Lastly, the choices that dictate the use of e-proctoring systems contribute to shaping our modern educational infrastructures, with a potential effect on education itself. This means that the existing risks for privacy, data protection as well as the discriminatory effects of these systems all become particularly salient beyond the individual, on a broader societal level.

23

Some of the legal issues just outlined in this paragraph have been challenged before European courts and supervisory authorities over the past few years, mainly from a data protection perspective. Very recently, the discriminatory effect posed by these systems has been challenged in the Netherlands.

24

In the following Sections, the decisions concerning e-proctoring will be critically analysed to understand the state of play of this evaluation of practice within the EU legal framework. Section 4 will focus on data protection: the paper will assess the main arguments used in the decision to see to what extent the GDPR can protect against the risks raised by monitored online exams. Section 6 will discuss the anti-discrimination case and the possible remedies available under the equality framework.

25

The data protection implications of e-proctoring tools have been assessed in a few European legal systems so far and with different outcomes. We counted one pre-pandemic decision [24] and seven decisions from 2020 onwards. [25] In terms of data protection authorities (“DPA”) guidance, the French Commission Nationale Informatique & Libertés (“CNIL”) has issued a note with recommendations on surveillance and online exams in 2020. [26]

26

The e-proctoring systems examined in the court decisions varied, ranging from recorded to automated proctoring. [27] Such systems were partly customisable, and the HEIs adopted different features and retention policies. None of these cases reported the use of a facial recognition system to authenticate the examinees, nor the adoption of a fully automated decision-making process (i.e., there is no automated termination of the exam, but the recorded video/audio and the score for the deviant behaviour are reviewed ex post and the final decision is made by the examiner or the examination board). [28]

27

The table below includes the list of decisions summarising their main outcome.

Table 1. List of the e-proctoring cases with data protection claims and summary of the main outcomes of the decisions

28

In terms of general outcomes, the DPA decisions (with the only exception of Denmark 2) found at least one, but usually numerous, GDPR violations in the application of e-proctoring systems, notably with regard to transparency rules (Denmark 1, Iceland, Italy) or the safeguards on extra-EU transfer (Italy, Portugal). On the contrary, Dutch courts reached an opposing conclusion.

29

Hence, in order to have a clearer understanding of the state of the art of the case law concerning e-proctoring tools and data protection, it becomes relevant to comprehend the different points raised in the decisions, the arguments used, and the friction within the legal framework.

30

In the following subsections, we will focus on the four main points that emerge from the cross-analysis of the decisions on e-proctoring, notably: 1) the actors involved in processing for e-proctoring purposes and the allocation of responsibility between them; 2) the lawfulness of the processing; 3) the respect of the right to information towards the data subject when deploying the e-proctoring system; 4) the challenges of cross-border data transfer.

77

GDPR enforcement processes—either through the exercise of data rights before the data controller, or through recourse to a DPA or courts—are key in understanding how e-proctoring systems can be inspected, challenged, and lawfully restrained. Whether at a university or an e-proctoring service provider level, personal data processing left unchecked risks, principally, harming students’ fundamental rights. It is thus important to inspect the degree to which students and other (collective) entities are empowered to challenge e-proctoring systems bringing claims in front of the relevant authorities to contest the exclusionary and intrusive effects of online invigilation.

78

As our above analysis shows, e-proctoring systems can and have been challenged in both national courts and data protection authorities with relative success. It is interesting to note that on many occasions, universities were ordered to stop using specific e-proctoring software due to the GDPR violations observed by the DPAs. To this day, national courts have not delivered similar decisions.

79

Student complaints vis-a-vis the national DPAs is what instigated the decisions against the use of e-proctoring systems in Italy, Portugal, and Iceland. As we briefly presented above and as summarised in the Annex, students were able to raise arguments ranging from GDPR violations (unlawful consent to the processing of personal data, etc.) to violations of fundamental rights such as privacy and data protection. [112] These cases stayed well within the realm of individual direct action that aims to counter harms experienced by students in the deployment of e-proctoring systems.

80

Personal data protection normative frameworks tend to centre around the individual. This perspective is an important dimension of the way in which data protection law ensures (levels of) control over personal data. Yet, there are different ways in which data protection law—and the GDPR in particular—enables collective empowerment beyond the individual. While understudied in scholarship and underused by policymakers, judges, and authorities, it is vital to explore GDPR collective action as a tool to challenge e-proctoring systems, especially as it has become widely accepted that data-driven technologies often provoke harm beyond the individual level.

81

The GDPR creates the procedural framework within which individuals can claim redress of individual harms incurred to each data subject respectively, but through acting collectively. [113] The recent Ola/Uber cases [114] show how these types of processes can empower groups of individuals when they exercise their rights in a coordinated manner. The Oracle/Salesforce case [115] is another stellar example of this understanding of collective action because its reasoning goes well beyond the limited number of individual claimants. This procedural framework requires a representative organisation or simply the coordination of numerous individuals, who bring one single procedure forward. Collective action can also refer to a single action on behalf of a group of individual data subjects operating to obtain a collective gain.

82

In the case of e-proctoring, we believe that GDPR collective action can be viewed as one available tool to tackle and counter harms suffered by specific groups of students or even by the student body as a whole. However, their (limited) exercise has not been particularly successful.

83

Collective student action has been a key instrument for ensuring the broader impact of the desired outcome. In Germany, for example, a complaint contesting the use of e-proctoring software was filed jointly by a university student and a digital rights non-profit organisation (the Gesellschaft für Freiheitsrechte (“GFF”)). [116] The complaint regarded the storage and processing of video and screen-recorded data by the e-proctoring software that the university chose for conducting student exams during the lockdown. The requested injunction failed to produce the desired outcome of restricting the storing of exam video recordings, as the motion was denied by the Court on the basis of procedural elements, preventing the examination of substantive aspects. [117] In particular, the Court stated that it could not address the lawfulness of the processing in the emergency proceeding. An overview of the injunction reveals that the urgent procedure which was chosen due to the student circumstances was not the appropriate juridical forum, especially when the objective was to produce an impactful decision that would contribute to counter surveillance and e-proctoring system normalisation in HEIs. This case is a representative example of litigation efforts to ensure a strategic outcome against the use of e-proctoring software.

84

When thinking about the effects upon individuals of powerful systems mediated through data-driven technologies, strength in numbers is critical. For this reason, GFF is decidedly attempting to create the necessary conditions for introducing collective action representing multiple students. Namely, the NGO has launched a public call looking for affected students. According to the call, GFF “want(s) to win fundamental decisions against excessive surveillance through online proctoring - and the best way to do that is with several cases that illustrate the problem”. [118]

85

Similarly, in Netherlands 1 and 2, rather than individually instigated student complaints, it was the representative body of students who launched a lawsuit against their university challenging the decision and the conditions of use of e-proctoring systems for online invigilation. The representative student council body contested the unlawfulness of the personal data processing, the discriminatory effects of the software, and the lack of student participation in the decision-making process regarding the use of e-proctoring systems for exam invigilation. This case is among the few that were brought forward by students on the basis of multiple GDPR violation claims.

86

Interestingly, among all the cases examined, the student council was the only one challenging the institutional decision-making processes that led to the introduction of the e-proctoring systems. These processes did not involve the input and feedback from the student council, characterised by the university as “unsolicited advice” on the use of online proctoring. [119] The Amsterdam Court in Netherlands 2 rejected the student council’s claims invoking the internal regulations that permit such ‘emergency’ decisions to be taken unilaterally by the administrative body of the university without having to necessarily consider the input of student representative bodies. Remarkably, but not surprisingly since this was a civil litigation, the Court put the responsibility to determine, describe, and explain all available alternatives to all types of invigilation processes that are occurring in the context of student exams on the claimants, i.e., student council. [120] Ιn sum, this means that the Court conceded that the student council's input was not necessary in the decision-making process regarding e-proctoring systems; at the same time, the Court decided to put the burden of explanation (and proof) on the advantages of alternative solutions to that same student council.

87

Finally, the Dutch Court in Netherlands 2 recognised the admissibility of the student council as a claimant in this case. However, it did not justify the decision based on the GDPR procedural standing rules nor clarified whether the claimants represented the university’s students in their collective interests vis-a-vis the GDPR violations in question. This lack of clarity is but one example illustrating the need for legal and procedural certainty in collective action cases with GDPR-based claims.

88

Similarly, in Greece, following a complaint filed by the association of teachers and even though the decision addresses remote teaching in general and not e-proctoring specifically, the DPA delivered its opinion highlighting the constitutional duty of the state to ensure the provision of education. [121] This duty, according to the DPA, provides the necessary precondition to any decision that is geared towards fulfilling that obligation. As mentioned in Section D.I, the complaint was filed with the Greek DPA by the private schools' teachers' union. It is noteworthy that the DPA found the union to not have the standing to file such a complaint because it did not operate under a specific mandate. [122] The lack of clarity about the need for representation mandates with regard to the defence of individual and collective interests and the inconsistencies in collective representation at national levels are creating the space for inefficiencies of GDPR enforcement. [123]

89

Overall, we contend that collective action cases instigated by students and/or represented by student bodies and other similar organisations help (re)shape the public interest objectives of HEIs to a participatory model that includes student voices in determining student interests as a whole. In this context, the GDPR can constitute a solid legal basis for these actions because of its potential to uncover harms and inequalities. This has certainly proven to be true in the Ola/Uber cases and can follow similar paths in contesting e-proctoring systems. However, existing disparities in national collective action legal frameworks could limit the full potential of these mechanisms.

90

Beyond the data protection framework, equality law can be distinctly mobilised for the same purposes, as shown in a recent case in the Netherlands. In the next Section, the paper will present the first case contesting the discriminatory effects of the identity recognition feature of an online invigilation software, discussing the remedies available to challenge e-proctoring practices under the EU anti-discrimination legal framework.

91

The above analysis reflects on the effectiveness of data protection tools as forms of accountability and assessment for e-proctoring systems used by public educational institutions. Despite the potential of the GDPR as a frame of reference and enforcement tool to protect human rights, the above-mentioned decisions did not go beyond data protection concerns. For instance, the discriminatory risks brought by e-proctoring were rarely put forward and discussed. However, such risks have become more and more pressing over the past few years.

92

As mentioned in Section 3, many concerns have been raised about the error rates of the e-proctoring’s facial recognition systems used for authenticating students leading to discriminatory effects against, such as black examinees. [124] Those students, for instance, have reported trouble logging into the virtual environment or were only able to do so when shining additional light on their faces. [125]

93

An e-proctoring software was used by the California bar for the admission exams organised remotely during the COVID-19 lockdown. Three students with disabilities sued the California bar because it refused to modify its remote proctoring protocols, which were making it impossible for disabled test-takers to efficiently sit the remote exams. [126] In Gordon v. State Bar of California, [127] the Court rejected the preliminary injunction because it did not recognise a concrete harm in the proctoring processes especially vis-a-vis the broader COVID-19 crisis. These are but some examples of reported exclusion. [128]

94

It is important to note that the discriminatory effects of e-proctoring systems are often linked to the facial recognition software and the room scan features of e-proctoring. Bias in these types of algorithms is not new, leading some academic institutions to reject or cease the use of e-proctoring systems citing accessibility and equality concerns. [129] However, as evidenced by our case law analysis, contesting e-proctoring systems has shown its limitations because the examination of data protection and privacy compliance did not always consider potential harmful, discriminatory effects. [130] For instance, while the plaintiffs did mention discrimination concerns in their litigation in the Netherlands 2 decision, barely any reference to this was provided. In particular, the students argued the potential for discrimination based on the protected characteristics of students recorded for the purposes of identification and online invigilation that might be revealed such as race or religion. However, the Court remarked that it does not appear to be possible that the material recorded will be used for discriminatory purposes but does not provide further arguments for such reasoning.

95

So, the question remains: what are the tools available to counter the discriminatory effects of e-proctoring systems? In examining this, we should also stress that while anti-discrimination law could constitute a suitable tool for software affecting a protected category, other groups (e.g., people with limited internet access) are not directly covered by this legal instrument.

96

The discriminatory effects caused by the facial recognition system were not specifically discussed in the decisions analysed in the previous Sections (see Table 1) because it was ascertained that students’ identities were manually checked by the examiners.

97

However, if a facial recognition system was adopted, the GDPR might have offered some (limited) grip to combat algorithmic discrimination. Article 22 GDPR might apply, but on the condition that the processing was solely automated with no meaningful human oversight. Moreover, the DPIA would offer the chance to assess and address discriminatory effects. [131] However, these sections of the DPIA often remain not sufficiently investigated.

98

Beyond the GDPR, anti-discrimination law is another relevant framework whose impact against e-proctoring systems is soon to be tested for the first time in the Netherlands. [132] During the submission of this paper, the first European case of an anti-discrimination complaint against the facial recognition system of an e-proctoring tool was filed by a student within the College voor de Rechten van de Mens (the “Netherlands Institute for Human Rights”, hereinafter “NIHR”). [133] According to the submitted complaint, [134] the student had difficulties logging into the e-proctoring system because the facial recognition software could only detect her face with the light pointing straight at her. The student claimed that this software’s inability to detect black people, especially when a public HEI mandates the use of this software, was discriminatory. The university’s initial response to the student was to attempt to decouple the student's skin colour from the factors considered by the facial recognition proctoring algorithm mainly due to the lack of proof of the existence of such a link. The response to the internal complaint was that “they cannot establish an objective link between the student's skin colour and whether or not the digital surveillance system is functioning properly”. [135] Against the backdrop of this case, it is useful to evaluate anti-discrimination laws as defensive tools against harms caused by e-proctoring algorithmic systems.

99

As explained by the complaint filed by the student, the Dutch anti-discrimination law qualifies indirect discrimination as whenever any apparently neutral provision, standard or practice related to people of a particular religion, belief, political opinion, race, gender, nationality, heterosexual or homosexual orientation or marital status is particularly harmful when compared to its effect on other people. [136]

100

The NIHR published an interim judgement on the 7th of December 2022. [137] It found that the facts presented by the student were sufficient for a presumption of indirect discrimination based on race, because: a) she was disadvantaged by the anti-spying software; and b) there is academic research showing that facial detection software generally performs worse on people with darker skin colours. [138] The NIHR applied existing legislation according to which, when there is a presumption of discrimination (so-called prima facie discrimination), the burden of proof shifts to the defendant, who must justify the use of the software. [139] In this respect, the NIHR concluded that the university had not provided sufficient evidence to do so. Hence, it gave ten weeks to the university to further substantiate its defence and reserved its final decision. As some authors have stressed, if the algorithm is a black box, it might be quite challenging to provide evidence of the lack of discrimination. [140]

101

Universities have a duty under anti-discrimination law to ensure that the practices—including e-proctoring features—are not unduly disadvantageous to any students before implementing them. To this end, they should choose a provider who will ensure this condition is satisfied.

102

In GDPR terms, this duty of care can be reflected in the application of the fundamental principles, such as accountability, fairness, and integrity. The principles of proportionality and necessity may play a significant role in assessing the lawfulness of the processing through e-proctoring software, also in relation to the assessment of discrimination risks. Moreover, it would be interesting to see national courts or DPAs assessing the existence of discrimination through the DPIA and the lens of the fairness of processing, a principle affirmed by Article 8 CFREU and Article 5 GDPR. This assessment could take place, for instance, when contesting a biased e-proctoring system that involves biometric authentication to sign in.

103

Over the past three years, many concerns have been raised in relation to the risks and situations of harm of e-proctoring implementation at universities during the pandemic. [141] Such concerns have been voiced and examined across Europe in a series of cases that were collected and critically analysed in this paper. In this final Section we summarise the legal takeaways of the analysis and pinpoint the more systemic issues that need to be addressed in relation to e-proctoring, and edTech more broadly.

104

Even if e-proctoring will not generally be needed by traditionally non-distant HEIs anymore (unless new emergencies arise), it might still be considered by those universities which are offering online programs, or which want to keep online assessments as an option. Hence, it is relevant to understand to what extent e-proctoring tools shall be used or implemented by universities in the post pandemic world.

105

The case analysis shows that Courts and Authorities i) took the emergency situation into account in their decisions, ii) identified key problematic issues in the use of e-proctoring tools from a data protection point of view; and iii) non-discrimination issues emerged later, and the litigation is, at the moment, less developed when compared to data protection.

106

With reference to the first aspect, the situations of urgency and emergency faced by HEIs due to the COVID-19 lockdowns entered the balancing exercise to assess the legitimacy of alternative exam measures and, in some circumstances, led to the justification of the adoption of remote proctoring. However, now that the peak of COVID-19 is over as a global health emergency it is important that universities review the measures implemented during the past three years and abandon those that are no longer necessary or proportionate.

107

Secondly, data protection authorities found several points of friction between the deployment of remote invigilation and the GDPR, leading, in the majority of cases, to the block of the processing. For instance, the most invasive features, including the profiling of students for flagging suspicious behaviours, were banned by DPAs on a number of grounds, such as the lack of proportionality or lawful basis for the processing of sensitive data or for the extra-EU data transfer.

108

Different lines of reasoning were followed by civil courts. Dutch judges, in Netherlands 1 and 2, generally admitted the legality of the use of automated e-proctoring during the pandemic, confirming the assessment performed by the university.

109

When the processing did not involve the controversial flagging feature, all DPAs stressed some issues in the implementation of the transparency measures adopted by the universities to inform students.

110

Indeed, the lack of information provided to students and staff was a critical deficiency highlighted by the supervisory authorities. This situation might be a consequence of the general opaqueness of the system (noticed, for instance, by the Portuguese DPA). The lack of information on the “cheating score” and the way it should be reviewed by examiners raises several questions as to the effective presence of the “human in the loop” in this kind of situation. Hence, where there is no authentic human oversight, Article 22 GDPR should find application and this will cast more than a doubt about the possibility to justify an automated decision, based on profiling, against the students on any grounds of Articles 22(2) or (4) GDPR.

111

DPAs and Courts developed divergent reasonings on two further important issues that can put into question the use of e-proctoring tools: 1) the scope of Article 6(1)(e) GDPR and to what extent the processing performed in the exercise of a public task should be sufficiently specified in a law or regulation; and 2) the assessment of the legal status of pictures and biometric templates collected or generated during e-proctoring operations.

112

With reference to the first point, the Italian DPA convincingly points out that the profiling feature and the flagging system raise new risks for the protection of fundamental rights that should be adequately considered in a specific law or regulation. The necessity to guarantee the integrity of exams and degrees is indeed a task carried out in the public interest by HEI, but the legal framework in place reflects a situation where the exams were supposed to be organised in a more traditional fashion. Hence, unless this specific processing is adequately regulated in a law, detailing the limits and safeguards of it, the feature to monitor the behaviour of students during the online exam might not be grounded on a lawful basis (at least in Italy, the flagging system was declared to be in violation of Article 6 GDPR).

113

On the contrary, the Dutch judges seem to have adopted a lighter interpretation of the requirements needed under Articles 6(1)(e) and (3) GDPR or, at least, they did not consider the e-proctoring data processing particularly intrusive as to justify a more tailored regulation. Hence, given these different interpretations, this point might be contested in a future litigation or investigation before a data protection authority.

114

With regard to the second aspect—the assessment of the legal nature of pictures and videos collected during the exam, the decisions raise some further issues concerning the notion of biometric and special categories of data.

115

All the cases examining the flagging systems excluded that the processing of pictures to assess the students’ behaviours was used to identify or verify the identity of individuals. The definition of biometric data and its classification as a special category of data in the GDPR is quite narrow and it might not include situations like the one here, namely biometric categorisation. [142] Nevertheless, as pointed out by the Italian Authority, when the system generates a biometric template, it is performing a processing that is preparatory to the identification and verification of the identity, even if the data is not used for this purpose in the end. [143] In other words, following the DPA’s logic, the attitude of the template to “allow or confirm the unique identification of that natural person” (Article 4(14) GDPR) can meet the definition of biometric data.

116

A different question is whether the processing of biometric data that is not used to uniquely identify an individual will attract the regime designed for the special category of data. As pointed out in Section 4.2.2, the reference to biometric data is quite narrowly crafted in Article 9, and the GDPR seems to have drawn a distinction between identification and verification based on the level of risk that these activities pose to individuals. [144] Nevertheless, many scholars have been quite vocal about the pitfalls of this classification, considering that —for whatever purpose a biometric data is used—the characteristics that can be extracted from it still retain a considerable potential to enable the identification of individuals or negatively affect them. [145] Moreover, and as we have already highlighted, biometric data is one of the areas where Member States can intervene to specify further conditions for the processing. Hence, biometric classification performed with some e-proctoring tools could entail the processing of special categories of data (as affirmed, for example, in the Italian case).

117

As for the pictures not transformed into biometric data, but collected and stored during the invigilation procedure, we have seen that these have the potential to reveal sensitive attributes related to ethnic origin, religious beliefs or political opinions. The legal nature of such data has been debated, but the CJEU has recently confirmed a broad understanding of the notion of sensitive data: if it is possible to infer the sensitive characteristics from the context of the processing, data should be treated as a special category and protected accordingly. This interpretation would be able to address most of the discriminatory concerns as data controllers will have to properly assess the disparate impact for students in the DPIA and, if the system is adopted, appropriately justify their choices and safeguards in place (for instance, how to train the examiner who reviews the flagged videos or how to explain how the “cheating score” is calculated).

118

In any case, if an e-proctoring system processes sensitive data, it might be very challenging to ground it on any of the conditions under Article 9(2) GDPR. Indeed, we might consider the goal of ensuring the integrity of exams as being of substantial public interest (Article 9(2)(g) GDPR). However, such interest should be clearly spelled out in the law, which has to be proportionate to the aim pursued, respect the essence of the right to data protection, and provide for appropriate safeguards. As mentioned in Denmark 1 and 2, such a threshold is quite high. Alternatively, the data subject could explicitly consent to the processing (Article 9(2)(a) GDPR). However, the imbalance of power between students and the university casts serious doubts about the use of such a ground. Finally, one could argue that sensitive data might be processed because the data subjects made those data manifestly public (Article 9(2)(e) GDPR). This lawful basis is generally interpreted in a restrictive way by DPAs. [146] In particular, it is necessary to verify that the data subject is proactively deciding to share this information and be aware of the consequences. [147] This would imply that the data subject shall have an effective power to choose whether to disclose or hide the sensitive characteristic (which might not always be the case in an e-proctoring processing). Moreover, the “making public” is generally understood as finding application where the individual makes the information available to the public at large, e.g., on a social network or through mass media. [148] On the contrary, the data acquired by an e-proctoring tool are meant to be processed within a closed environment and usually visualised only by a restricted number of authorised persons. [149] Hence, this lawful basis might not be fitting for the context at stake.

119

Moreover, even if not fully tested in the decisions examined, the principle of fairness (Article 5(1)(a) GDPR) and the tool of the DPIA could be used to address the potential discriminatory effects caused by e-proctoring and not only when the discrimination is based on an existing protected ground under data protection or anti-discrimination law (e.g. race, religion, etc.). For instance, situations of socio-economic discrimination do not fall within the existing boundaries of protection (unless it can be linked with, e.g., a certain ethnic group) but should nevertheless be taken into account by a HEI before deciding to deploy remote invigilation for an exam.

120

In parallel, we have seen that anti-discrimination law could address other serious pitfalls of e-proctoring systems, for instance, the failure of facial recognition tools for authenticating students. Based on the evidence collected and the studies emerging in this field, it was possible to build a case of prima facie discrimination before the Equality body in the Netherlands. The claimant showed that the tool used by her university did not let her join the exam unless she shone some powerful light directly at her face. It is yet to be seen how the university will discharge its burden of proof.

121

All in all, the adoption of an e-proctoring system requires the universities to perform a careful assessment of the characteristics of the software, the concrete modalities of deployment in the specific context, and the guarantees offered by the provider. As controllers, they should also evaluate the processing they are enabling when using a commercial third-party platform. The latter often perform further purposes with the data collected that are not necessarily in line with the institutional goals of HEIs. Many of these platforms are also based in the US, and the transfer towards this country is still highly problematic.

122

As we have pointed out in our analysis, some features, or some concrete implementations of such software in the educational environment have been sanctioned by DPAs and challenged by the NIHR, making the adoption of such tool much harder in practice, especially now that the pandemic is over.

123

To a large extent, the current legal framework has proven responsive to counter the unlawful use of e-proctoring tools by universities. The main notable exception is represented by the Dutch saga, where the judges adopted a debatable restrictive interpretation of some GDPR provisions, and the burden of proof carried by the claimants has practically disadvantaged the students.

124

Notably, and from a procedural point of view, we have argued that GDPR collective actions can become a useful tool in contesting e-proctoring systems. We have noted that GDPR has the potential to tackle and counter harms suffered by specific groups of students or even by the student body as a whole. However, as shown in the German and Netherlands cases, while different entities brought forward (admissible) GDPR claims against the HEI’s decisions to introduce e-proctoring systems, the cases were ultimately dismissed. The examined case law has also revealed disparities in both collective action processes and rules between different Member States. National procedural rules are coupled with GDPR and implementation laws, which all create a complex matrix of rules to navigate. These disparities are far from creating the necessary clarity needed for representative bodies to ensure the success of their claims.

125

Now that the emergency is over, the questions that remain open are what lessons have been learned and how should universities approach the decision-making process about edTech tools more generally?

126

The comparative analysis of the DPA’s decisions and the case law allowed to identify the controversial issues emerging in the different cases. This critical overview is functional to reflect on the reasons why e-proctoring has been contested, and to imagine how to develop edTech tools which are not only lawful but also able to guarantee the full exercise of the right to education and adequately reflect the ethos of the university.

127

We contend that while edTech tools may offer a series of advantages in terms of efficiency and productivity, they are rarely neutral instruments: they interact with the environment, people, and institutions. This mutual interplay in turn affects how these elements interact with each other and, ultimately, how education is provided. Hence, their adoption should involve the consultation of all the affected parties. How to ensure this democratic participation in the governance of institutions in a meaningful way (not just a ticking box exercise) and make sure that minority voices are heard is a crucial aspect that universities should fully embrace.

Acknowledgments

This research was supported by the British and Irish Law Education and Technology Association (BILETA) Research Awards 2021 and the School of Law of the University of Aberdeen. The views expressed in this paper are those of the authors only.

The authors would like to thank all the participants attending the workshop “Law and the Digital Classroom”, organised in Lisbon at the NOVA School of Law on the 12th of July 2022, for their comments and suggestions on the draft version of this paper.

A special thanks to Dr Guido Salza for his support in constructing the survey and analysing the e-proctoring questions within the BILETA project ‘Zooming in on Privacy and Copyright Issues in Remote Teaching’, Dr Maria José Schmidt-Kessen for the helpful discussion on the Danish sources, and Mr Jamie Murphy for the final proofreading of this work.

Annex I - Summary of all the e-proctoring-related decisions reported in the paper

Denmark, Datatilsynet (DPA) - 2018-432-0015 (“Denmark 1”)

The DPA launched an investigation after learning from the media that various high schools were using an automated e-proctoring system.

The DPA recognised that a high school can, in principle, use the e-proctoring tool to process students’ personal data to prevent cheating in online exams. Such processing could be grounded in Art. 6(1)(e) GDPR (necessity to perform a task in the public interest). However, the DPA expressed serious concerns about the lack of an appropriate assessment of the necessity and proportionality of the processing for the declared purpose. Additionally, the investigated school failed to provide an adequate lawful basis for the processing of sensitive data which can be collected by the e-proctoring tool (the necessity for substantive interests’ reasons, referred to in Section 7(4) of the Danish Data Protection Act, was not deemed an appropriate ground because it has a narrow scope of application, such as processing for the purpose of a serious threat to health).

Finally, the School did not provide specific information to the students about the e-proctoring processing.

Denmark, Datatilsynet (DPA) - 2020-432-0034 (“Denmark 2”)

After receiving a phone inquiry, the DPA launched an investigation concerning the use of a recorded e-proctoring tool by a Danish university in the spring of 2020. The DPA ascertained that the processing involved personal data of approximately 330 examinees in the form of audio and video recording of the students, screenshots of their desktops and browsing history, and IDs. No facial recognition software was used: students’ identity was manually checked at the start of the session.

The DPA found that:

Germany, OVG Nordrhein-Westfalen, Beschluss vom 04.03.2021 - 14 B 278/21.NE

The Gesellschaft für Freiheitsrechte (GFF) filed, together with a student, an emergency application to the Higher Administrative Court of North Rhine-Westphalia claiming several data protection violations. The aim was to ensure that the examination scheduled for shortly after would not be recorded, but, at most, observed by means of video transmission. 

The Court rejected the claim on procedural grounds. In particular, it stated that whether the recording and temporary storage of the audio and video connection and thus the processing of personal data by Art. 6(1) GDPR is justified, this cannot be conclusively assessed in interim legal protection proceedings.

Greece, Αρχή προστασίας δεδομένων (DPA) - Decision 50/2021

The Hellenic Ministry of Education and Religions Affairs (the Ministry) decided to promote and implement a method of distance learning by technological means for students in primary and secondary education during the COVID-19 lockdown period. The teachers’ union contested the legality of that government decision at the Hellenic DPA. While the DPA considered this method legal, it found that the Ministry had failed to consider a number of factors and risks in relation to the rights and freedoms of the data subjects when conducting a DPIA.

Recognising the need for distance education, the DPA provided an opinion to the Ministry to address the flaws and shortcomings. The DPA noted in its decision multiple GDPR violations, namely of provisions related to the lawfulness of processing and the obligations of the data controllers. It then called on the Ministry to address and remedy these violations in the coming four months. After that period, the Ministry is called to report its updates to the DPA.

Iceland, Persónuvernd (DPA) - 2020112830

After receiving a complaint by a student, the Icelandic DPA assessed the lawfulness of the online monitoring of a remote examination. The Authority dealt with three main legal issues: a) lawful legal basis for processing; b) data security; and c) transparency.

The Icelandic DPA stated that:

Italy, Garante privacy (DPA) - Ordinanza 9703988 - 16 Sep 2021

Following a university student's complaint regarding the use of an automated e-proctoring service with flagging features to spot cheating behaviours, the DPA investigated the lawfulness of such processing.

The DPA stated that:

The Netherlands, Rb. Amsterdam - C/13/684665 / KG ZA 20-481 (“Netherlands 1”)

The introduction of online proctoring systems to invigilate exams happening remotely was decided due to the COVID-19 lockdown. The software monitored students while they took their exam from home. The software recorded the user's webcam, microphone, internet traffic, and inputs.

The Amsterdam Court of First Instance rejected the request by student representatives and an individual student for a preliminary injunction against the use of the above-mentioned e-proctoring software. The Court ruled that measures against COVID-19 did not allow for a suitable alternative. Also, it examined the GDPR compliance of the software and found the data processing by the university was based on Art. 6(1)(e) GDPR (necessity to perform a task in the public interest or in the exercise of official authority), and that the processing complied with the requirements set by the GDPR.

This being a preliminary injunction, the Court also examined the admissibility of the student council in bringing forward these claims on behalf of the university student body. The Court applied section 3:305a of the Dutch Civil Code, and concluded that only a foundation or association with full legal capacity can institute legal proceedings that protect similar interests of other persons, insofar as they represent these interests under its Articles of Association and these interests are sufficiently safeguarded. In that regard, and according to the court, the student councils are not a foundation or association with full legal capacity.

The Netherlands, Gerechtshof Amsterdam - 200.280.852/01 (“Netherlands 2”)

This case is the appeal of the Netherlands 1 preliminary decision. It concerns a civil dispute between the Central Student Council (CSR) at the university, the Student Council at the Faculty of Economics and Business (FSR), and an individual student against the defendant, the university.

The preliminary injunction was filed first, and the District Court of Amsterdam ruled in favour of the university, finding that the government's COVID-19 measures did not allow for suitable alternatives and that the surveillance had a legal basis in Art. 6(1)(e) GDPR. The plaintiffs appealed to the Court of Appeals Amsterdam Court.

The plaintiffs claimed that the introduction of the e-proctoring system chosen by the university breached the GDPR in several respects. They claimed that it was unnecessary to introduce monitoring software, that more data than necessary was processed, that there was a lack of transparency and security, and that sensitive personal data was processed without a legal basis.

The Court found that the university successfully demonstrated that the use of the software was necessary for the performance of the task of exercising official authority under Art. 6(1)(e) GDPR. It also found that the plaintiffs failed to prove that its use violated the principles of purpose limitation and data minimisation. The plaintiffs had argued that less intrusive alternatives could be used, but the court placed the burden of sufficiently presenting these feasible alternatives to them. The plaintiffs argued that the university had not provided full insight into how the proctoring software detects cheating. However, the Court held that the plaintiffs had not plausibly demonstrated that anyone not authorised by the university to view the video and audio, such as the service provider or US intelligence agencies, could gain access. In addition, the claimants argued that the images collected could be sensitive personal data for which there was no legal basis for collection. The Court ruled that images identifying an individual could not simply be sensitive personal data revealing, for example, religion or race. The court could not foresee that the images would be used by the university to discriminate against test takers based on protected characteristics.

While the Court acknowledged that it was disruptive that students could not go to the bathroom during online exams, it noted that the same was true for on-site exams. The Court therefore held that it could not consider this complaint in assessing the legality of online examinations.

The CSR sent a letter to the university's Executive Board, which was described in the judgement as "unsolicited advice". In this letter, the CSR strongly opposed the use of e-proctoring, recommended against the use of room scanning, and advised that the university provide students who cannot/would not use proctoring with alternative means of taking exams without delaying their studies.

The plaintiffs also argued that e-proctoring violated Art. 8 ECHR. The Court considered whether the interference with privacy by proctoring was justified under Art. 8(2) of the ECHR. To do this, it looked back at the reasoning it had used in assessing the lawfulness of proctoring in relation to the GDPR. It held that it was plausible that the interference with privacy was necessary in a democratic society and could be considered proportionate.

From a procedural side, the plaintiffs alleged that the university had violated the law by changing the so-called Teaching and Examination Regulations (Onderwijs- en Examensregeling, “OER”) without following due process. Art. 7.13 of the Higher Education and Scientific Research Act (WHW) requires that every Dutch higher education program adopts an OER. A higher education institution may also adopt a OER for a group of programs. The Court found that the university had not breached any procedural rules in deciding to introduce e-proctoring. Specifically, the Court referred to Art. 7.13(2)(l) WHW which allows the Board of Examiners to depart from OER in special circumstances. The Court found that the COVID-19 restrictions qualified as a special case where the exam board is allowed to deviate from the OER.

The Netherlands, College voor de Rechten van de Mens (Netherlands Institute for Human Rights), Decision 2022-146 (“Netherlands 3”)

A university student called on the Netherlands Institute for Human Rights (NIHR) to establish that the use of the e-proctoring software was discriminatory. Specifically, the student argued that she was discriminated against due to her skin colour when she was using the contested software. The student had trouble logging in the exams and was only able to do so when shining a direct light on her face. According to the preliminary decision, the person claiming discrimination has succeeded in this for two reasons. First, the parties agree that the anti-cheat software hindered the woman. Second, there is academic research showing that face detection software generally performs worse on darker skinned individuals. Taken together, these facts are sufficient for a presumption of indirect discrimination on the basis of race.

The NIHR established that the student had provided sufficient facts from which it can be assumed that the university had indirectly discriminated on the grounds of race by using anti-cheat software for the supervision of exams. If there is a suspicion of discrimination, the university must prove that it has not acted in violation of the law. The Board considers that the university has not provided sufficient evidence for this. The intermediate judgement gives a 10-week deadline to the university to provide evidence that there was no discrimination.

Portugal, Comissão Nacional de Proteção de Dados (DPA) - Deliberação/2021/622

The DPA carried out a preventive assessment of the lawfulness of an e-proctoring tool with flagging features to spot cheating behaviours that were meant to be used by a Portuguese university (the reference was anonymised by the DPA).

The decision focuses on four main aspects: the application of the principles of i) purpose limitation; ii) data minimisation; iii) the legal basis of processing; and iv) the lawfulness of data transfers to the US.

The DPA affirmed that the rectoral order authorising the e-proctoring tool did not provide specific criteria about the cases where such a tool could be used. The lack of such criteria led to the violation of the purpose limitation principle, as the processing purpose was not sufficiently specified, and of the data minimisation principle, as the discretion of the teaching staff concerning the use of such a tool may lead to process data not necessary for the stated purpose.

Furthermore, the Authority doubted that the legitimate interest (Art. 6(1)(f) GDPR) is the correct legal basis for processing. The DPA found that the legitimate interest basis was not used correctly in the present case. In particular, the Authority stated that: i) the data controller did not carry out the balancing test between the legitimate interest at stake and the rights and interests of the data subjects; and ii) the processing at stake was particularly important, as it involved profiling and biometric data.

However, the DPA, taking into account the public interest at stake, stated that Art. 6(1)(e) GDPR should be applied, according to the rules provided for by Art. 6(2) GDPR, concerning national rules on processing for public interest purposes.

In any case, the Authority affirmed that the processing concerning video and audio recordings of students' behaviour, based on consent, was unlawful. The Authority considered that the consent did not meet the requirements set forth by the GDPR, as students are obliged to give their consent if they want to take exams.

As to extra-EU data transfers, the DPA applied the CJEU rationale in C-311/18 Data Protection Commissioner/Maximilian Schrems v. Facebook Ireland, 16 July 2020. It stated that students’ personal data must not be transferred to the US, as there was a lack of additional measures preventing the access to the transferred personal data by the US authorities.

Hence, the DPA concluded that the e-proctoring processing at stake violated the principle of lawfulness, purpose limitation, and data minimisation (Art. 5 (1)(a)(b)(c) GDPR) and ordered the e-proctoring provider to destroy the personal data collected through the tool.

* Alexandra Giannopoulou: Postdoctoral researcher, Institute for Information Law (IViR), University of Amsterdam (a.giannopoulou@uva.nl). Rossana Ducato: Senior Lecturer in IT Law and Regulation, School of Law, University of Aberdeen; Lecturer at UCLouvain (rossana.ducato@abdn.ac.uk). Chiara Angiolini: Research Fellow, Law Department, University of Siena (chiara.angiolini@unisi.it). Giulia Schneider: Research Fellow, Department of Banking Sciences, Catholic University of Milan (giulia.schneider@unicatt.it). This paper and the related research are the results of a joint and collaborative work. However, Sections D.I, E, and F can be attributed to Alexandra Giannopoulou; Sections B, D.II.1, D.II.2, D.III, and G to Rossana Ducato; Sections A, C, and D.II.3 to Chiara Angiolini; Section D.IV to Giulia Schneider.