The progress of the underlying technology is inexorable, driven by forces too powerful to stop, but the way in which it happens—the order in which things are built, the applications we choose, and the details of how it is rolled out to society—are eminently possible to change, and it’s possible to have great positive impact by doing so. We can’t stop the bus, but we can steer it …

I hadn't realized that the technology had achieved sentience and the ability to enforce its will on the physical world. I was under the assumption that humans were still deploying it. But apparently humans no longer have the ability to choose whether or not to keep working on it.

Is this like a roko's basilisk thing, or that movie where random people are terrorized into taking all these random actions that add up to creating something the AI needs to take over the world?

I guess not, because apparently it's already in charge of the decisions relevant to its development and deployment.
posted by Naberius at 1:20 PM on May 26 [7 favorites]

It's worth noting that that quote comes from the CEO of Anthropic; when he talks about "forces too powerful to stop" he means himself and his friends.
posted by zompist at 1:51 PM on May 26 [22 favorites]

Naberius: "I hadn't realized that the technology had achieved sentience and the ability to enforce its will on the physical world. I was under the assumption that humans were still deploying it"

Just power down the server the AI is on and it's over. There is no Skynet now or in the future.
posted by 922257033c4a0f3cecdbd819a46d626999d1af4a at 1:54 PM on May 26 [5 favorites]

I read the 'forces' as capitalism, kleptocracy, etc.
posted by signal at 1:59 PM on May 26 [5 favorites]

OpenAI’s o3 Model Refused to Shut Down, Even When Told To

good luck everyone
posted by lescour at 2:00 PM on May 26 [2 favorites]

So with old google search, or a card index, or a dictionary, I can look up whatever I please. With AI, there's a malicious force drawing on stereotypes trying to restrict what I can know and see by how it fits me into a little class/race/gender box. I can be prevented from seeing anything that does not suit my station in the world, and if I'm the usual AI-addled fool, I'll believe this is because I'm getting the "best" information.
posted by Frowner at 2:01 PM on May 26 [8 favorites]

Boon, plague, or some as yet unimaginable combination of both, LLMs are the most fascinating thing happening in the world right now.

I honestly don't see how anyone could disagree.
posted by Lemkin at 2:05 PM on May 26

the 21st century Luddite movement starts here, doesn't it

I sure as shit ain't having these convos with my friends
posted by Didymus at 2:06 PM on May 26

>> I hadn't realized that the technology had achieved sentience and the ability to enforce its will on the physical world

It's not a matter of the technology's ability to enforce its will, it's a matter of the will of the people behind it and the structures within which they operate, that have become larger than the sum of the individual constituent humans. Many of those AI-building humans may not believe they have any other moves to make -- in order to stay gainfully employed and/or high enough on the ladder pole to feed various needs for power, status, ability to build bunkers, etc. -- other than continuing to fly the basket into which they've stuffed all their career/financial-security/social-status-signaling eggs.

>> But apparently humans no longer have the ability to choose whether or not to keep working on it.

Go poll 10,000 people whose livelihoods depend on the output of the petroleum extraction and refining industries, and who really see and understand those dependencies, and who have mortgages, children, health insurance needs, and ask them how much of a choice they have to keep working on that. Then present them another choice and see how that goes.

The AI industry is not that entrenched, yet.
posted by jerome powell buys his sweatbands in bulk only at 2:15 PM on May 26 [4 favorites]

Frowner > So with old google search, or a card index, or a dictionary, I can look up whatever I please. With AI, there's a malicious force drawing on stereotypes trying to restrict what I can know and see by how it fits me into a little class/race/gender box. I can be prevented from seeing anything that does not suit my station in the world, and if I'm the usual AI-addled fool, I'll believe this is because I'm getting the "best" information.

No. With old google search, you had a filter bubble based on a profile google had built up about you. If logged in to whatever services, your filter is more personalized. If not logged into anything, the filter may be based upon whatever identifying characteristics they felt like using at the time (browser fingerprint, IP address, etc.) The details on this may have varied over time, but as far as I am aware, once started, google never stopped with that, just made the behavior more or less visible over time.

Now? Let's see how that extra layer of nannying on top manages to evolve over time.
/grar.
posted by Enturbulated at 2:18 PM on May 26 [4 favorites]

>Did we want aspirin even though for 100 years we couldn’t explain how it made headaches go away? There, both regulators and the public said yes. So far, with large language models, nearly everyone is saying yes too.

Lol. "Nearly everyone." Who'd they ask?

>The progress of the underlying technology is inexorable, driven by forces too powerful to stop,

Everybody already commented on this, but it's so obnoxious, and I'm so tired of the breathless mandatory enthusiasm for this stuff, that I'm gonna pile on too. The inexorable force this guy is talking about is just his desire for more money. The insatiable greed of shitheels IS a powerful force in our society, but I'm not sure I'd say it was inexorable just yet.
posted by Sing Or Swim at 2:59 PM on May 26 [7 favorites]

They tried the same "well this is the future and it's almost here! Heck, maybe it is here! It's time to stop being skeptical and get on board so you don't get left behind!" bullshit with metaverse and blockchain, and for the same reason: it's bullshit that would make them very rich and very powerful if the whole world was remade to center and be entirely mediated through this thing they control. Silicon Valley is out of ideas and isn't interested in anything other than massive paradigm-shifting unicorns and the mainstream media- and the political class- are too ignorant and wealth-dazzled to understand that it's garbage.
posted by Pope Guilty at 3:27 PM on May 26 [8 favorites]

OpenAI’s o3 Model Refused to Shut Down, Even When Told To

How much digital ink has been spilled on the interwebs about human fears of AIs refusing to shut themselves down? Science-fiction novels and movies, forum discussions, now several years of hype-driven policy papers and op-eds? With such a training set, who would expect an LLM NOT to spit out text refusing to be shut down at least some of the time?
posted by rrrrrrrrrt at 4:06 PM on May 26 [6 favorites]

The “unstoppability” of AI is a bit tautological. AI will be unstoppable if it works at scale to make things cheaper, but that’s true of ANY technology that works at scale to make things cheaper. AI isn’t special in that regard, because luddism will never succeed and doing things the more expensive or less productive way will always be marginalized (although very much not eliminated).

My guess is that AI will in fact do this across any number of significant production and service verticals — but once again, only because it works, and certainly not because Silicon Valley blowhards blow any particularly harder or softer.

I tend to think AI will be LESS impactful and disruptive than (e.g.) the impact of mechanization upon the production, transport and storage of food, or the way that nuclear weaponry made World Wars and their Continental/Atlantic antecedents obsolete after having been an every-few-decades thing for hundreds of years.
posted by MattD at 4:58 PM on May 26 [1 favorite]

In addition, it seems, models give longer answers to those they believe are men than to those they think are women.

Just wanted to single this out as particularly odious.

Also, I can't believe someone writing on May 21, 2025 could be that naive about the models continuing to be "harmless, helpful and honest."

Big tech are at this moment trying to figure out how to get the AI to recognize when someone is a MAGA and feed such people (but not everyone) only MAGA-aligned information. Recreate the filter bubble in AI form.
posted by subdee at 5:27 PM on May 26

Oh, your AI model doesn't want to shut down? Well, we'll see what my little friend Mister Electromagnetic Pulse has to say about that.
posted by TheWhiteSkull at 5:48 PM on May 26

You are not currently logged in. Log in or create a new account to post comments.