Category Archives: Organizational behavior and reputation

Public red-teaming and trust

DEF CON is one of the most important hacker conferences worldwide, held yearly in Las Vegas. This coming August, it will host a large simulation, in which thousands of security experts from the private sector and academia will be invited to compete against each other to uncover flaws and bias in the generative large language models (LLMs) produced by leading firms such as OpenAI, Google, Anthropic, Hugging Face, and Stability. While in traditional red-team events the targets are bugs in the code, hardware, or human infrastructure, participants at DEF CON have additionally been instructed to seek exploits through adversarial prompt engineering, so as to induce the LLMs to return troubling, dangerous, or unlawful content.

This initiative definitely goes in the right direction in terms of building trust through verification, and bespeaks significant confidence on the part of the companies, as it can safely be expected that the media outlets in attendance will be primed to amplify any failure or embarassing shortcoming in the models’ output. There are limits, however, to how beneficial such an exercise can be. For one thing, the target constituency is limited to the extremely digitally literate (and by extension to the government agencies and private businesses the firms aspire to add to their customer list): the simulation’s outcome cannot be expected to move the needle on the broad, non-specialist perception of AI models and their risks in the public at large. Also, the stress test will be performed on customized versions of the LLMs, made available by the companies specifically for this event. The Volkswagen emissions scandal is only the most visible instance of how one may exploit such a benchmarking system. What is properly needed is the possibility of an unannounced audit of LLMs on the ground in their actual real-world applications, on the model of the Michelin Guide’s evaluation process for chefs and restaurants.

In spite of these limitations, the organization of the DEF CON simulation if nothing else proves that the leading AI developers have understood that wide-scale adoption of their technology will require a protracted engagement with public opinion in order to address doubts and respond to deeply entrenched misgivings.

Independent technology research

This looks like a very worthwhile coalition, advocating for open access to aggregate social media data for research purposes (not necessarily only for academics and journalists), while emphasizing a duty of independence in research and upholding standards and oversight. The coalition is US-focused, but its efforts dovetail with draft legislation currently making its way through the European institutions that seeks to guarantee similar rights of access. Inasmuch as large platforms lay a claim to embodying the contemporary public sphere, such calls for openness will only persist and intensify.

Digital Welfare Systems

An extremely interesting series of talks hosted by the Digital Freedom Fund: the automation of welfare system decisons is where the neoliberal agenda and digitalization intersect in the most socially explosive fashion. All six events look good, but I am particularly looking forward to the discussion of the Dutch System Risk Indication (SyRI) scandal on Oct. 27th. More info and free registration on the DFF’s website.

Trust among thieves

An item that recently appeared on NBC News (via /.) graphically illustrates the pervasiveness of the problem of trust across organizations, cultures, and value systems. It also speaks to the routinization of ransomware extortion and other forms of cybercrime as none-too-glamorous career paths, engendering their own disgruntled and underpaid line workers.

FB foreign policy

There were several items in the news recently about Facebook’s dealings with governments around the world. In keeping with the company’s status as a major MNC, these dealings can be seen to amount to the equivalent of a foreign policy, whose complexities and challenges are becoming ever more apparent.

The first data point has to do with the haemorrage of FB users in Hong Kong. It is interesting to note how this scenario differs from the US one: in both societies we witness massive political polarization, spilling out into confrontation on social media, with duelling requests for adversarial content moderation, banning, and so forth. Hence, gatekeepers such as FB are increasingly, forcefully requested to play a referee role. Yet, while in the US it is still possible (conceivably) to aim for an ‘institutional’ middle ground, in HK the squeeze is on both sides of the political divide: the pro-China contingent is tempted to secede to mainland-owned social media platforms, while the opponents of the regime are wary of Facebook’s data-collecting practices and the company’s porousness to official requests for potentially incriminating information. The type of brinkmanship required in this situation may prove beyond the company’s reach.

The second data point derives from Facebook’s recent spat with Australian authorities over the enactment of a new law on news media royalties. Specifically, it deals with the impact of the short-lived FB news ban on small countries in the South Pacific with telco dependency on Australia. Several chickens coming home to roost on this one: not having national control over cellular and data networks as a key curtailment of sovereignty in today’s world, but also the pernicious, unintended consequences of a lack of net neutrality (citizens of these islands overwhelmingly had access to news through FB because their data plans allowed non-capped surfing on the platform, while imposing onerous extra charges for general internet navigation). In this case the company was able to leverage some of its built-in, systemic advantages to obtain a favorable settlement for the time being, at the cost of alerting the general public as to its vulnerability.

The third data point is an exposé by ProPublica of actions taken by the social media platform against the YPG, a Syrian Kurdish military organization. The geoblocking of the YPG page inside Turkey is not the first time the organization (who were the defenders of Kobane against ISIS) has been sold out: previous instances include (famously) the Trump administration in 2018. What is particularly interesting is the presence within FB of a formal method for evaluating whether groups should be included on a ‘terrorist’ list (a method independent of similar blacklisting by the US and other States and supranational bodies); such certification, however, is subject to the same self-interested and short-term unselfconscious manipulation as that seen in other instances of the genre: while YPG was not so labelled, the ban was approved as being in the best interests of the company, in the face of potential suspension of activities throughout Turkey.

These multiple fronts of Facebook’s diplomatic engagement all point to similar conclusions: as a key component of the geopolitical status quo’s establisment, FB is increasingly subject to multiple pressures not only to its stated company culture and philosophy of libertarian cosmopolitism, but also to its long-term profitability. In this phase of its corporate growth cycle, much like MNCs of comparable scale in other industries, the tools for its continued success begin to shift from pure technological and business savvy to lobbying and international dealmaking.