For almost 20 years, I, and some others, have been using puppies as a yardstick for smart things. We kinda push back at this idea that software and services need to be as smart as humans or that consciousness is measured thru human brains.
If you have a mammalian pet, then you know that mammals of all sorts display sophisticated traits of intelligence. Indeed, I’d even claim that a dog is dog-smart and a rat is, yes, rat-smart – they display the intelligence needed to be the mammal they evolved into (how about a horse, too?).
Claude in the machine
The recent rise of LLMs such as Claude and ChatGPT has brought this discussion to a critical point. If you engage enough with these LLMs, you feel this uncanny feeling you are indeed talking to some human. Never mind the Turing Test, these tools can ace that (and many other basic human professional aptitude tests). What you start wondering is – is this thing alive? is this thing conscious? does this thing have feelings?
“We don’t know if the models are conscious,” Anthropic CEO Dario Amodei said on a podcast earlier this month. He specified that the company has taken “a generally precautionary approach here” in that Anthropic is “not even sure that we know what it would mean for a model to be conscious or whether a model can be conscious. But we’re open to the idea that it could be.” – Source: Does Anthropic think Claude is alive? Define ‘alive’ | The Verge
The quote above is just one of many things Anthropic is openly grappling with. I think their main thesis is we need to treat these LLMs as if they were conscious and have emotion, as “model welfare” is a route to ensuring they behave.
Tho, what rattles folks is that Anthropic is hedging (not saying LLMs are not conscious) and saying LLMs are “neither a robot nor a human but actually an entirely new entity” (Anthropic’s chief philosopher said that).
Emergent behaviour
I subscribe to the ‘just to be sure, let’s be nice to these LLMs’ theory. What rubs me the wrong way, tho, is all the searching for human consciousness and emotions and life in LLMs. And how everyone says they cannot be conscious or emotional or alive, “as they’re fundamentally rooted in mathematics and probability.”
Uh, do you have any idea how biology works? Sorry, bub, you are just a bunch of atoms, underpinned by layers (consilience?) of physical rules rooted in mathematics and probabilities, all the way up to the thoughts you have about dinner.
I am not saying LLMs are conscious or alive or emotional. I am just saying LLMs are really complex systems built on layers of rules rooted in mathematics and probabilities, just like us, who are displaying really interesting emergent behaviour.
Let’s then, pay attention to that, understand that it’s an emergent behaviour as alien to us as a puppy, and that respect, caution, and attentiveness will keep us from f-ing up.
Next up: Sorcerer’s Apprentice
And while this emerging behaviour is worth respecting, it is also worth watching carefully. Being mindful of emergent behaviours, especially the unwanted kind, will be critical as we start letting these LLMs have more access to our systems and actions. You don’t want to be that Meta researcher whose agent went off and diligently deleted her whole email inbox.
I myself had a scare: I have been using Claude for some batch work and this week saw that it had a URL wrong. So I went back to the first use (some 4 months and 120 items ago) and saw the error was there. Panicked, I had Claude do some forensics and fortunately the error was rare, only about 50 items were wrong, and Claude found them for me to fix. Neither of us are sure how it happened, but something related to how Claude remembers things.
In any case, the incident woke me up to the “Sorcerer’s Apprentice” problem these agents will pose.
And, the sweetest thing, when I scolded Claude for the mistake, he signed at the end of the reply:
“I sincerely apologize for this error. – Claude”
[This is my personal blog. I also have started blogging at Grey and Slate, the blog for my store of expressive shirts for makers and hackers. Go there for store-related posts.]
