Thread by @personamatters

I read the “Anatomy of an AI system” by Crawford and Joler (anatomyof.ai), and started thinking about how I've seen AI discussed recently.

So, a thread about the popularization of AI, the everyday discourse on automation, and the importance of technological literacy.

Lately, I’ve attended a fair amount of talks on artificial intelligence. These have mostly been at events about “digitalization”, attended by government officials and library employees.

In most of these talks – as well as in informal discussions sparked by them – the discourse revolves around “AI” as a fuzzy concept. It’s an idea, a wish, and a suggestion, usually solving particular problems with very generic answers. AI is rarely (if ever) actually defined.

These discussions might be about anything from websites to user-activated tiny scripts to scifi-fuelled visions of nearly autonomous computer agents. You can never tell if the focus is on parts of a wider system, a system itself, or something entirely outside any given system.

All these potential forms of machine agency blend into each other, with no clear distinction between them. Which is fine, if it's acknowledged! But it's not. Combined with a lack of basic primers, the audience of these events (me included!) is left with a very shaky foundation.

What ARE mentioned, however, are the benefits. Common to all these takes is an assumption about increased efficiency, decreased dependency on human labor, and decreased workloads for humans. "AI" makes everything easier and more efficient.

Also mentioned: vague dystopian scenarios, somewhat sloppy critiques of transhumanism, and overall ethics of AI and automation.

The discourse, then, focuses on end-points: the state of organizations when an autonomous machine is up and running, and the state of humans when the machine does all of this for them. A world in which all of "this" (what?) "has happened" (when?) to "us" (who?) "here" (where?).

This creates two layers of black boxes around the concept. 1) The program – the AI, the app, the service – itself always remains undefined: is it a few scripts? A string of hyperlinks? A neural network?

2) The historical, cultural, material, social, and organizational details that connect the program to the wider world are obfuscated: E.g. how is the AI created, what resources go to its upkeep, what problems does it solve, and how is it implemented it into existing workflows?

That is, the creation, implementation, operation, and structure of the automated system is never addressed, positioning AI as a perpetual wish-fulfilling object.

As long as you don’t go too deep into what the system actually is or how it's used, it has unlimited potential to fix any problem and answer any question.

This is good for making inspiring presentations, but we don’t need any more pep talks because that is literally all we already have in all of these events, and as long as we imagine things only for the sake of imagining them, we will never run into the questions worth answering.

I know these discussions are happening elsewhere. It's just troubling that they're not happening at these events full of government officials who must shape – and, worryingly, are already shaping – policies related to these matters.

And who knows, maybe top officials do attend events with more detailed discussions. But speaking from the point of view of the general administrational body, we seem to be exposed to a very light take on AI, given the responsibilities many of us bear in shaping its future.

This is basically the Silicon Valley tech bro model of operation: dream up something cool, start implementing it, and never stop thinking what you’re actually doing. That’s how we end up with Musk inventing public transit and algorithms getting fooled by darker skintones.

Furthermore, our need for more detailed discussions on AI is especially important because even when technology is developed and implemented with the greatest care, we still rarely get exactly what we imagined.

The automatization system you will eventually get will not be the one you hoped for. It will not be the perfect answer to a problem, nor will it be able to solve problems (or do anything, really) independently of your organization, it’s structure, and its employees.

Like everything in byzantine organizations, it'll work, somehow, but it'll also require a whole network of human labor, material resources, and complex relations that all ultimately conflate into the “AI system”.

So you need to know super basic things. Who in your organization will supervise these systems and their effects? Who will actually use them? How will this affect their other tasks?

You'll also need to know which material and informational resources your chosen type of machine agent will be drawing from. Will it tie you to a support company overseas? A datacenter beneath a salt lake somewhere? Legacy code that'll linger in your online platform for 20 years?

These are, fundamentally, questions about the global organization of labor and distribution of resources, and these questions will never be answered in discussions where AI systems are doubly-veiled black boxes of utopian administration and production.

The AI system does not enter into a vacuum – into an AI-sized hole – within an organization. It must mesh with the existing structure of organization and labor as well as alter them, and this takes significant time, effort, and planning.

That is what I want to talk about in these events. That is where the important discussions come from. That is what exists before implementing a machine agent into the office, and that is what exists after that as well. The AI in the middle just augments all that.

The automatization and AI-fication of the office will undoubtedly happen, but as long as our discourse remains oblivious to its own rhetorical impracticalities and vague techno-optimism, that future seems neither smart nor even particularly appealing.