What Gets Proceduralised: Why Some Jobs Change Before Others

Written in March 2023. The question this text closes on, whether these systems would climb up to System 2, has since been answered: reasoning models trained with reinforcement learning have reached into it. The text is left as it was thought.

ChatGPT spread faster than Snapchat, faster than TikTok, faster than Instagram. What counts more than the speed is the spread: it was adopted across almost every field at once, which no application had done before it.

What it puts within reach, at any rate, is a cognitive service. Massively deployed, available to everyone, and able to carry out an enormous range of tasks where the software we used to write carried out exactly one.

I was asked whether this was comparable to what happened when the internet was deployed, and I think there is a subtlety here that matters. With the internet, what we mostly saw was a shift into a sector that already existed, services, which expanded massively thanks to everything we built on top of it. What that demanded, at any rate, was training people for the new jobs arriving. Here we are not creating a new sector. It is the same one, and what changes happens inside it.

And if the service is cognitive, then the question of tomorrow’s work becomes a precise question, one that has an answer. Saying artificial intelligence will change everything is not enough. You have to know which part of cognition has just been put within reach, because that is what decides which jobs move first.

Two systems of thought

In Thinking, Fast and Slow, Daniel Kahneman describes two regimes of thought.

System 1 is intuitive, reflexive, immediate. It is what you have learned and give back without thinking about it. Once you can ride a bike, you no longer need to concentrate. When you drive a familiar route, you get from A to B while talking to the person next to you and thinking about what you will eat tomorrow.

System 2 is what happens when you get lost. You stop, you turn off the radio, you look around, you try to remember whether you have been here before, and you make a decision. Working memory, semantic memory, episodic memory, explicit reasoning.

A distinction is worth making here, though, because the video this text comes from moves past it a little fast. System 1 belongs to the psychology of decision, procedural memory is a category from neuropsychology, and the two do not overlap exactly. They meet on the point that interests me precisely: the movement you execute perfectly without being able to describe it.

What two amnesic patients showed

Clive Wearing was a musician and a choir conductor. In 1985, herpes encephalitis destroyed his hippocampi. Since then his memory span rarely exceeds a few dozen seconds: he no longer forms memories. He still plays the piano, though, and he still conducts a choir. He does not remember having played the piece. He plays it.

The other experiment says even more. Henry Molaison, known in psychology by the initials H.M., was operated on in 1953 to treat severe epilepsy, and the operation cost him the ability to form new memories. Brenda Milner had him perform a mirror-drawing task: trace a star while seeing only the reflection of his hand. Everyone fails on the first attempt, and everyone improves over a few spaced sessions. H.M. improves too, session after session, while being convinced each time that he is discovering the task.

That is where the hinge sits. Loading the task into working memory, doing it, failing and starting again, modifies the procedure itself. System 2 writes into System 1, and it does so through plasticity, without even needing to leave a memory behind.

Frozen weights

Saying these models have a cognitive system is necessarily a shortcut, and I would rather flag it. It is an analogy, not a description. But if I had to bring these models closer to one category rather than another, it would clearly be procedural memory.

The reason is fairly simple. These are Transformer architectures, and once training is over, the weights are fixed. Making the analogy with the brain, that means there is no neuroplasticity as such. The model has learned a very large number of regularities and it can give them back remarkably well for a given task. What it cannot do is the mirror loop: look at the effect of what it has just produced and rewrite its own procedure accordingly.

There is an objection to that argument, though, and it is a serious one. These models do adapt inside a conversation: give them three examples in the context and they catch the rule. That is what we call in-context learning, and it is real adaptation.

It is transient, simply. It lives in the context window, it is bounded by its size, and nothing that happens there travels back into the weights. At the next conversation the lesson is gone and everything starts over. It is working memory without consolidation, which, from my point of view at least, strengthens the argument rather than weakening it. Plasticity is missing exactly where the two patients show it to be decisive.

What gets proceduralised first

From there, you can look at jobs differently.

The ones resting mostly on giving back a procedure are clearly the first affected. The graphic designer in front of their software, running through a learned sequence of moves. The developer, as soon as a program is simple enough to be written without thinking. I am not passing judgement on the worth of those jobs, I am only observing how much of that work had already been proceduralised.

The figures, since there are some, deserve to be quoted properly. The paper published by OpenAI ten days ago estimates that around 80 % of the US workforce could see at least 10 % of their tasks affected by these models, and that around 19 % of workers could see at least half of theirs affected. Those are workers and tasks, not jobs eliminated, and the distinction is not cosmetic.

What resists is more interesting. An art director, a technical director, do not hold on because they would be smarter than anyone else. They hold on because what makes them succeed over the long run is exactly the loop I mentioned: look at the environment, notice that the procedure that worked elsewhere does not work here, and change it. A foundation model is very good at unrolling a known plan from a given context. It is the adaptation of that plan, as things unfold, that stays out of reach.

So the question that matters for what comes next, and I have no answer to give today, is whether these systems will stay at this stage, replicating procedural memory extremely efficiently and potentially more efficiently than a human on certain tasks, or whether they will climb the cognitive value chain up to System 2, and up to improving themselves.

At any rate I know where to look. The day one of these systems observes the effect of what it has just done and changes what it is accordingly, rather than starting every conversation with the same weights, it will have happened.