Platform regulation, content moderation, and AI-based filtering tools: Some reflections from the European Union

8

Articles 14 and 15 are the e-Commerce Directive key provisions for intermediaries’ liability and content monitoring. [19] Pursuant to Article 14, intermediaries of online services are exempt from liability for content stored in their services by its users, subject to not being aware of illegal activity or information in their services, or if made aware, for example, through an injunction ordered by a Court, to expeditiously remove or to disable access to the content. Article 15 prohibits Member states to impose a general obligation on providers […] to monitor information which they transmit or store, or actively seek facts or circumstances indicating illegal activity. In the case Poland v Parliament  [20], Advocate General (AG) Saugmansgaard Øe regarded the prohibition enshrined in Article 15 as a general principle of law governing the internet. [21]

9

What constitutes general monitoring against specific monitoring has not been determined by the e-Commerce Directive. [22] The European Court of Justice (ECJ) explored the subject in judgments like L’Oreal vs Ebay  [23] and Scarlet Extended v SABAM et al. [24], and provided some sort of guidance on what kind of content screening is allowed under Article 15 in SABAM vs Netlog [25] and Glawischnig-Piesczek [26].

10

In SABAM vs Netlog, SABAM—a Belgium private collective rights management organisation—sought through an injunction against Netlog, that the latter install a filtering system at their own cost to prevent copyright infringements of their repertoire. The ECJ found that preventive monitoring not compatible with Article 15. [27] The deployment of such a system, would require the social media company Netlog to carry an active monitoring of almost all the data stored relating to all of its service users. [28] In this case, the obligation to monitor was broad and too burdensome for Netlog, and it would be at odds with Netlog’s freedom to conduct a business and its users right to personal data and freedom of information. [29]

11

In Glawischnig-Piesczek, an Austrian Court was concerned with whether an interim injunction against a host provider (Facebook), to remove a post previously declared defamatory could also extend to other posts of identical or equivalent content. Here, the Court held that the measure did not impose a general obligation to monitor within the meaning of Article 15. However, the national court order for removal of identical or equivalent defamatory content should contain “specific elements” to identify the content—targeted monitoring one could say—and in any event, it should not require an independent assessment of the content by the host provider because it will make use of automated tools.

12

The DSA seeks to contribute to the proper functioning of the internal market by harmonising the rules for intermediary services, such as social media networks or marketplaces, to tackle the spread of illegal content, address online disinformation, and other societal risks.

13

Articles 7 and 8 are of special interest: Article 7 shields against liability those intermediary services which in good faith and diligently […] take measures aimed at detecting, identifying, and removing, or disabling of access to illegal content or take the necessary measures to comply with the requirements of national law, in compliance with Union law, including the requirements set out in this Regulation.

14

Article 8 contains the prohibition on general monitoring and active fact-finding, replicating the wording of Article 15 e-Commerce Directive. It is worth mentioning that throughout the legislative process, the European Parliament (EP) made an amendment to Article 8 by clarifying that there is no general obligation to screen information providers transmit and store neither the jure nor the facto through automated or non-automated means. [30] In addition, the EP also introduced a new limb to Article 8 stating providers of intermediary services should not be obligated to use automated tools for content moderation […]. Both amendments, however, did not make it to the final version of the text just approved at time of writing. Article 8 now reads as follows: “no general obligation to monitor the information which providers of intermediary services transmit or store, nor actively to seek facts or circumstances indicating illegal activity shall be imposed on those providers”. [31]

15

In recent years, on the online content sector-specific front, several legal instruments have been passed and others are now in the pipeline of the EU legislature. These measures can target specific types of online service providers or particular categories of illegal content harmonised under EU law. [32]

16

Under the Audiovisual Media Services Directive, Article 28b requires Member States to ensure video-sharing platforms providers (VSPPs) take appropriate measures against illegal and harmful content. These measures, however, should not lead to any ex-ante control measure or upload-filtering of content contrary to Article 15 of the e-Commerce Directive.

17

The DSM Directive ignited a heated debate around its Article 17. The lengthy provision on the use of protected works by online content-sharing services providers (OCSSPs), sets out a specific liability regime for OCSSPs departing from the principle under Article 14 of the e-Commerce Directive. [33] OCSPPs can be liable for the content uploaded by its users to their services when such content infringes copyright-protected works. To escape liability for acts of communication to the public and make available to the public copyright-protected works, OCSSPs shall obtain licenses for these works or make best efforts to obtain them. In the event of no licensing agreements, OCSSPs are subject to the obligation to prevent the availability of those works in their services and to the take down and stay down of that content. [34]

18

Poland challenged the legality of Article 17 before the ECJ. [35] It argued the obligations arising from Article 17.4 [36] on OCSSPS implicitly require the use of filtering technologies to monitor content uploaded by users to prevent the infringement of copyright. In the view of the Polish government, deploying automatic filters is a serious interference on the users right to freedom of expression and information. The Court dismissed Poland’s action and reasoned that even though Article 17.4 liability regime indeed imposes a limitation on the exercise of freedom of expression and information of users, Article 17 provides appropriate safeguards to preserve the essence of that right as guaranteed by Article 11 of the Charter of Fundamental Rights (CFR). Furthermore, the ECJ agreed that some filtering will be needed to comply with the mandates of Article 17. [37] Yet, as long as these filters do not screen and block lawful content when uploaded by users, their use is compatible with Article 11 CFR.

19

Since June 7, 2022, the TERREG is in force. Hosting service provides are obligated to remove or disable access to terrorist content at least within one hour of receipt of a removal order from a competent authority of any Member State. [38] Pursuant to Article 5.8, hosting service providers, when implementing specific measures [39] to address the dissemination of terrorist content in their services, are under no obligation to use automated tools. However, Recital 25 clarifies that providers should have recourse to automated tools if they consider them appropriate and necessary to address the dissemination of terrorist content online. When using automated means, providers should take appropriate measures through human oversight and verification and ensure accuracy to avoid blocking or removing content that is not terrorist related. [40]

20

More controversial is the new proposal for a Regulation fighting child sexual abuse published in early May of 2022. [41]The Regulation seeks to harmonize the requirements imposed on online services providers removing the divergences from Member States rules to prevent and combat child sexual abuse. [42] It complements the general framework of the DSA and among others, it introduces an obligation on providers to detect, report, remove and block child sexual abuse material (CSAM). Apart from the privacy and mass surveillance concerns voiced [43], the Article 10 mandate is of interest. Online service providers shall execute detection orders by national authorities by installing and operating technologies—AI systems—to detect the dissemination of CSAM, favouring systems which have been vetted by a new coordination authority, the EU Centre on Child Sexual Abuse. [44] Although the proposal explains these orders will be specific and targeted, it is not yet clear how the screening would be performed and if the current tools are effective. Some commentators warned there are no technologies available that can safely scan people’s messages or discern what is abusive from what is not. [45]

23

For the efficient deployment of a filtering system in content moderation, the premise is that the system will work with clear and defined parameters of what constitutes illegal or harmful content. The first challenge in this respect is to define the nature of the content and work backwards—i.e., why a post can be labelled as hate speech, or what is hate speech for that matter. In AI terms, this would consist in training the model with data sets to teach the system to recognize on its own the targeted illegal content. In this process, the quality of the data fed to the system will be key. Automatic detection can assess only what it can know and what can be represented as data but limited to the data it has. [51] In addition, algorithms can be subject to human bias during the AI training process. Human bias can take place in both machine-supervised learning and non-supervised learning processes. In the former, human intervention is needed to evaluate data examples and select the appropriate labels or to evaluate an automatically applied labels. [52] In the later, hidden biases could arise from the dataset itself. [53]

24

Further, considering these technologies are context insensitive and unable to make subjective decisions [54] there are certain bars at technical level. Although some context can be incorporated in a tool, historical, political, and cultural context are more difficult for an AI system to be trained to detect. [55] As Spoerri points out, the state-of-the-art of filtering technologies is quite limited as tools are only capable of matching content, but it is not yet possible to determine whether the use of a file—be it music, text or image—constitutes an infringement. [56] Despite this situation, the EU legislator seems to assume that online service providers can employ intelligent filters that identify infringing content while enabling the upload and making available of lawful content. [57]

25

Against this background, the ECJ in C-401/19 warned that where a filtering system does not adequately distinguish between lawful and unlawful content, leading to blocking of lawful content, the system is not compatible with the right to freedom of expression in Article 11 CFR. [58]

26

Yet, there is no infallible filtering system able to make such a clear distinction. [59] For that reason, the focus should be put on the accuracy of these tools. Accuracy in this context can be defined as the rate at which the tool’s evaluation of content matches a human’s evaluation of the same content. [60] The results can be divided into four categories: true positives, true negatives, false positives, and false negatives. [61] This begs for the question how many false positives or false negatives are acceptable to not fall in over-blocking patterns threatening users’ rights. Perhaps, certain standards should be set for the development and use of filtering technologies within this sphere, and improvements of filtering tools should focus on bringing these mistakes within an acceptable range. [62] What is acceptable would depend on analysing the content and harm at stake. Trade-offs in this regard are unavoidable [63]—a balance between leaving false negatives online and blocking lawful content should be achieved. At any rate, predictability of the systems should be guaranteed as well as mechanisms to correct the potential mistakes.

27

In the EU, another layer of complexity exists; the regulation of illegal content categories is not entirely harmonized at Union level [64] so the same type of content may be considered illegal, legal but harmful, or legal and not harmful across the 27 Member States. [65] This is reflected in the broad definition of illegal content enshrined in the DSA [66] “illegal content means any information, which, in itself or in relation to an activity, including the sale of products or the provision of services, is not in compliance with Union law or the law of any Member State, irrespective of the precise subject matter or nature of that law”.

28

Therefore, online service providers deploying filtering systems need to consider that for those categories of illegal content not harmonized at EU level, tailor-made filters for specific jurisdictions within the EU need to be implemented. For larger online services providers operating worldwide, having to deal with different degrees of requirements across the globe or even with conflicting rules is already the case. This forces them to operate their compliance content policy and enforcement programmes based on global rules and adjust them through a risk-level approach. One should question then, if the risk of non-compliance comports fines as those established by the DSA [67], online service providers would not be prone to self-censorship/over-removal for the sake of compliance, paying little heed to the fundamental rights of its users.

29

The difficulties of screening tools to consider language and social/cultural context evidences the gap between the capabilities of a human and that of machines. The high rate of false positives and the removal of lawful content resulting from automated screening emphasize the added value of human moderation. It is for this reason that the inclusion of human review at some stage of the moderation chain should be a requirement to safeguard users’ fundamental rights. Typically, human review can take place when content is reported by a user and a decision needs to be made by the online service provider on the content flagged. Similarly, filters can serve to flag content by the platform own initiative and subsequently be reviewed by a moderator. Moreover, although most online platforms follow a “publish-then-filter approach” [68] human review can happen either before the content is online or after it is published. [69]

30

If human content moderators are excluded entirely from the screening process, it will be the automated system deciding which content stays online or is taken down. Still, reviewing every piece of content caught by a filter as a potential infraction of the law or from the online service provider TOUs, would defeat the purpose of using content screening systems by the online service providers, rendering the content moderation exercise not feasible.

31

To that extent, the ELI principles on automated decision making [70] proposed an ex-post content human review after a decision has been taken by automated means and challenged by the user. To put it simply, users who posted/uploaded a piece of content which was afterwards blocked or removed should have access to a redress mechanism to challenge the decision requiring human review. In that sense, human review guarantees full compliance with applicable law without relinquishing the benefits of automation. [71] However, such an approach does not resolve the issue of users being at the mercy of online platforms and their internal dispute settlement mechanisms at first instance, forcing them to rely on them regardless the content disputed was lawful from the outset. An issue that fuels the debate on the role of these platforms acting as delegated enforcers of public powers vis-à-vis online users’ freedom of expression and due process rights. [72]

32

But what is EU law position on human review? Although the outcomes of non-compliance with a human review requirement remain to be seen, references to human review or human intervention can be found in different EU legal acts. [73] As an illustration, under the General Data Protection Regulation (GDPR) [74], data subjects have the right not to be subject to a decision based solely on automated processing without human intervention. [75]

33

Some of the above-mentioned new rules introduce provisions on the inclusion of human review when using filtering technologies. The TERREG mandates hosting service providers to include human oversight and verification safeguards when using technological measures to protect its services against the dissemination to the public of terrorist content, to ensure accuracy and to avoid the removal of material that is not terrorist content. [76] In the same fashion, the proposal on CSAM establishes that providers should ensure regular human oversight and where necessary, intervention, to ensure technologies operate in a sufficiently reliable manner. Even more, when detecting potential errors and potential solicitation of children. [77] In the case of the DSA Regulation, Article 20(6) requires that decisions on removal/block of allegedly illegal content or content incompatible with the platform T&Cs, are reviewed by qualified staff and are not solely taken on the basis of automated means. [78]Therefore, the human review requirement comes within the context of the complaint handling system. This is, ex post, when the online service provider receives a complaint by a user of the platform.

34

Lastly, the ECJ had the opportunity to provide some precisions on human review while using automated filters through Glawischnig-Piesczek and Poland v Parliament. In the first judgement, the Court was of the view that a hosting service provider is not under the obligation to include human review—“an independent assessment” in the Court’s words—when using automated filtering technologies to comply with a removal order by a national court. In the second case, the Court held in similar terms that OCSSPs are not obliged to conduct an independent assessment of the content uploaded by their users to prevent the uploading or making available to the public of copyright-protected works, in the light of the information provided by the rightsholders and of any exceptions and limitations to copyright. [79]

35

This approach, however, seems difficult to conjugate with the rules discussed above and creates different standards for human review depending on the type of content at stake and on the subject requesting the removal. Even if human review in such situations is not required by law, in all likelihood AI based filters will match/block ambiguous content or lawful content when searching for the objectionable content. This is particularly relevant in the case of the copyright exceptions and the rights of content creators to use protected works without the prior authorisation of rightsholders. Essentially, this implies that someone whose lawful content was removed due to a filter mistake would have to go through the complaint handling system of the provider to challenge the removal. Only then human review would be required per Article 17(9) DSM. With this set up by default, the balance tilts towards the right to intellectual property vis-à-vis the freedom of expression and creation of users, placing a heavy burden on non-professional creators and user-generated content. [80]

36

Although online service providers could of course still decide to rely on human review [81] for those cases, even platforms that use filters and human review are incentivized to remove legal “grey area” content. [82]

37

For reasons of scalability, speed, and cost-efficiency, online service providers will keep relying on AI based filtering solutions on voluntarily basis to tackle illegal or harmful content. [83] Thus, the key question is no longer whether to rely on AI based systems to screen content, but whether automated content screening is turning into an obligation in disguise for online platforms and if so, how could it be articulated with online intermediaries’ liability exemption and their fundamental rights, namely, their freedom to conduct a business.

38

Looking at Article 7 DSA and other sector-specific rules, one observes a trend of the EU legislator to require a more active role of online service providers to tackle illegal content. [84] In fact, the use of automated filtering systems seems a de facto must for online intermediaries to escape liability, especially in cases where short removal time is required. Recital 26 of the DSA sheds light on the scope of Article 7 DSA voluntary measures of providers to conduct investigations and actions for the detection, identification, removal or disable access to illegal content. It clarifies the requirements of conducting such activities “in good faith” and “in a diligent manner” by also stating that if the provider uses automated tools for those purposes, it should take reasonable measures to ensure the technology is sufficiently reliable to limit to the maximum extent possible the rate of errors. There are still some open questions, how the error rate could be measured, if a threshold should be established, or what happens if the filter fails to detect illegal content despite the intermediary voluntary actions. Would this “bad” filter engage the liability of a provider conducting voluntary measures to fight illegal content? A reading of Recital 22 tells us that to benefit from the exemption of liability, the provider, upon obtaining actual knowledge or awareness of illegal content, needs to act expeditiously to remove or to disable access to that content, and that knowledge of the illegal nature of the content can be obtained through own-initiative investigations.

39

In this regard, the case Delfi AS v. Estonia  [85] of the European Court of Human Rights (ECtHR) is quite insightful. This Court found the liability imposed by the Estonian Supreme Court to an online news portal operator for defamatory comments made in their site by third parties, did not violate the applicant freedom of expression. The Court noted that Delfi’s filtering system in question failed to detect the harmful comment and left it online for some weeks. [86] This amounted to not having taken reasonable measures to remove the comments without delay. In such a circumstance, the Court found the liability on the online news operator to be a proportionate restriction on the applicant`s right to freedom of expression.

40

Looking at the case from another perspective, one could also ask if requiring an online service provider to deploy a filtering system to look for illegal content or manifestly illegal content is a proportionate restriction to its freedom to conduct a business under Article 16 of the CFR. The answer would depend among other things, on the size of the internet intermediary and its resources. Thus, a liability obligation of that sort would create differences between the market players. As Frosio and Geiger flagged, the economic impact of enforcing filtering and monitoring obligations on online service providers has been discussed in the case-law of the ECJ, in particular, in Netlog v. SABAM where the Court held that imposing a monitoring obligation on Netlog to screen all works uploaded would burden the online service provider with the requirement of installing at its own expense filtering technologies. [87]

41

The DSA renders online services providers accountable through algorithmic transparency and reporting obligations including disclosure obligations on metrics for notices processed by automated means, any use of automated means for the purpose of content moderation, and information on the type of content moderation engaged by providers of online services. [88] Furthermore, the Regulation provides for procedural measures for users to dispute content blocking or removals of information labelled as illegal content or against the ToUs of the platform. [89]

42

While these measures purport a robust layer of protection for online users’ fundamental rights, the fact remains that AI based filtering tools are far from being perfect and the technical challenges of these tools cannot be resolved only with transparency obligations on online intermediaries. There are issues that are yet to be addressed: the algorithmic fairness and human bias on data sets, accuracy standards since for certain type of content,a higher rate of accuracy may be easier to achieve [90], although that would not be the case when context is an intrinsic factor for determining illegality of a piece of content. In addition, it should not be assumed that online intermediaries are best placed to assess the legality or illegality of content, and they should not be seen as neutral when making such decisions. [91] With that in mind, there will always be “grey zone” cases which would require human judgement rather than an AI based system taking a decision. [92] It is particularly in those situations where online intermediaries may feel compelled to over-block to not risk liability.

43

If despite the flaws, we take AI based filters as a “necessary evil” for content moderation, then, closer regulatory scrutiny should be paid to their design, implementation, and the consequences of their use on public speech and the fundamental rights of online users but also the role and responsibilities of online intermediaries. To that end, data sets to build automated AI systems should be documented and traceable. [93] Guidelines on content moderation automated systems, including accuracy and error thresholds could be adopted to complement the DSA. Human review should continue to be an essential component of moderating with filters. Some authors postulate that automated decision-making processes should be subject to the “human-in-command” principle, namely, human intervention to supervise the overall activity of the AI system, its impact, and the ability to decide when and how to use the system. [94] By the same token, there should be clear accountability, liability, and redress mechanisms to deal with potential harm resulting from using applications, automated decision-making and machine learning tools. [95] Other propositions advocate for AI ethical principles specific for content filtering [96] which could form part of a regulatory framework to ensure compliance and enforceability. Such a framework should include the setting up of mandatory certification standards for testing AI systems to ensure they meet minimal safety and accuracy requirements. [97] This is in line with the approach of the EU Artificial intelligence Act (AIA). [98] The proposal sets a horizontal legal framework for the development, placement on the market, and use of AI applications in the Union, based on a risk-based approach. Under the current form of the proposal, there is no specific reference to AI systems for content moderation purposes. However, it has been argued that the AIA could be the right place to regulate the use of upload filters. [99]

44

As a closing remark, one should also not lose sight of technological innovation to question if other options to automated tools managed by online platforms are possible. In its report on combating online harms through innovation, the American Federal Trade Commission listed user tools in its recommendations to tackle harmful content. [100] These tools could help users to control what content they see on the internet, shifting the content moderation effort from private platforms towards users. This is the idea of the so-called middleware for content moderation services. Middleware in this context, is a software program that rides on top of an existing internet or social media platform such as Google, Facebook or Twitter and can modify the presentation of underlying data. [101] Middleware can be understood as a layer between the user and the online platform. By relying on this software, users could control their experience in a relevant platform but at the same time have the option to interact with other users of the online platform. [102] The development of these tools is still at a very early stage, and for now aimed mainly to the specifics of legal but harmful content. Nevertheless, there is room to consider if similar AI initiatives could provide effective alternatives to automated filters for fighting illegal content.