The Psychology of Technology Institute reports on research conducted by Professor Kiran Garimella of Rutgers on content moderation and the design of online platforms.

Specifically, Garimella studied ‘fear speech’, a type of online activity that has many of the negative connotations of ‘hate speech’ but, by eschewing the latter’s inflammatory tone and ad hominem attacks, is much harder to counter, both in terms of devising automated systems to identify and remove it, and in discursive terms to expose and debunk it. The spread of false, misinformed, or decontextualized statements (generally, but not exclusively, pertaining to some stigmatized group) does not automatically result in the naming and shaming of responsible parties or in calls to action (or violence), but it sets the stage, forms or reinforces a general climate of opinion, comforts an audience in its prejudices and normalizes extreme assessments of states of fact on which further extremist appeals may be built.

One of the reasons fear speech bypasses many of the roadblocks we have erected to the spread of hate speech is that its rhetorical form is one we are extremely familiar and comfortable with in our heavily technological and risk-laden societies. What is unique to fear speech is its ability to mix the epistemic thrill of conspiracy theory with the practical, down-to-earth, and seemingly neutral tone of the PSA. After all, this is not much of a novelty: prejudice has often been couched in prudential terms— “not all members of a stigmatized group may be nefarious and vicious, but when in doubt…”.

The implication of Garimella’s research is that, if we are serious about removing dangerous speech from our online environments, it is necessary to cast a wider net, focusing not only on the narrow band of clearly and openly aggressive hate speech, but addressing its precursors as well, the false, baseless and irrational fears out of which hate grows, and its mongers. This position, in turn, dovetails with the Psychology of Technology Institute’s own contention that design should be the focus of information governance, rather than downstream content moderation.

This position is closely argued, prima facie reasonable, and clearly germane to the struggles many organizations, big and small, private and public, have with speech content. I fail, nonetheless, to be completely convinced. For one thing, freedom of speech concerns become salient as soon as obvious threats and incitement are taken out of the equation. In order to label fears as false or groundless, it would become necessary to lean even more heavily into fact-checking, a process not without its own political pitfalls. Moreover, the unequal distribution of risk propensities on different issues within a polity surely must be a legitimate basis for political sorting, organization, and choice. However we may feel normatively about it, fear is a longstanding mechanism for political mobilization, and furthermore certain scholars (such as George Lakoff, for example) have claimed that its use is not symmetrical along the political spectrum, which would lend these proposals a (presumably unwelcome) partisan slant.

I believe that in considering fear speech and its possible limitations, it is helpful to begin with the motivations of the three parties: the speaker, the audience, and the information overseer. Specifically, what is the goal pursued in attempts to curtail fear speech? Is it to silence bad actors / stigmatize bad behaviors? Is it to prevent mutual discovery and community-building between people already convinced of beliefs we find objectionable? Or is it to shield the malleable, the unwary, the unattentive from misleading information that may lead them eventually to embrace objectionable beliefs? Empirically, information overseers (a national government, say, or a social media platform) may well hold all three, perhaps subsumed by the overriding imperative to preserve the reputation of the forum. But analytically it is important to distinguish the motivations, so as to understand what a proposed remedy portends for each, how it affects the incentives of each type of speaker and each segment of audience. And the key consideration in this respect is how it impacts their likelihood of voice and/or exit, visible protest vs. the redirection of communication in another forum. Only on this basis is it possible to evaluate the feasibility or desirability of curbs to online fear speech.