Societal trust and the pace of AI research

An open letter from the Future of Life Institute exhorts the leading AI labs to enact a six-month moratorium on further experiments with artificial intelligence. The caliber of some of the early signatories guarantees that significant public conversation will ensue. Beyond the predictable hype, it is worth considering this intervention in the AI ethics and politics debate both on its merits and for what it portends more broadly for the field.

First off, the technicalities. The text locates the key chokepoint in AI development to be exploited in the interests of the moratorium in the scarcity of compute power. Truly, we are at the antipodes of the decentralized mode of innovation that drove, for instance, the original development of the commercial and personal web in the 1990s. However, it remains to be seen whether the compute power barrier has winnowed down the field into enough of an oligopoly for the proposed moratorium to have any chance of application. A closely related point is verifiability: even if there were few enough players to enable a coordination regime to emerge and there was virtually universal buy-in, it would still be necessary to enact some form of verification in order to police the system and ensure nobody is cheating. By comparison, the nuclear non-proliferation regime enjoys vast buy-in and plentiful dedicated enforcement resources (both at the nation-state and at the international organization level) and yet is far from perfect or fool-proof.

Moving to broader strategic issues, it bears considering whether the proposed moratorium, which would necessarily have to be global in scope, is in any way feasible in the current geopolitical climate. After all, one of the classic formulations of technological determinism relies on Great Power competition in military and dual-use applications. It would not be outlandish to suggest that we already are in a phase of strategic confrontation, between the United States and China among others, where the speed of tech change has become a dependent variable.

Perhaps, however, it is best to consider the second-order effects of the letter as the crux of the matter. The moratorium is extremely unlikely to come about, and would be highly unwieldy to manage if it did (the tell, perhaps, is the mismatch between the apocalyptic tone in which generative AI is described and the very short time requested to prepare for its onslaught). Nonetheless, such a proposal shifts the debate. It centers AI as the future technology to be grappled with socially, presents it as largely inevitable, and lays the responsibility for dealing with its ills at the foot of society as a whole.

Most strikingly, though, this intervention in public discourse relies on very tenuous legitimacy grounds for the various actors concerned, beginning with the drafters and signatories of the letter. Is the public supposed to endorse their analysis and support their prescriptions on the basis of their technical expertise? Or their impartiality? Or their track record of civic-mindedness? Or their expressing of preferences held by large numbers of people? All these justifications are problematic in their own way. In a low-trust environment, the authoritativeness of a public statement conducted in this fashion is bound to become itself a target of controversy.