AI Launches Nukes In ‘Worrying’ War Simulation: ‘I Just Want to Have Peace in the World’

Throwing that kind of stuff at an LLM just doesn't make sense.

People need to understand that LLMs are not smart, they're just really fancy autocompletion. I hate that we call those "AI", there's no intelligence whatsoever in those still. It's machine learning. All it knows is what humans said in its training dataset which is a lot of news, wikipedia and social media. And most of what's available is world war and cold war data.

It's not producing millitary strategies, it's predicting what our world leaders are likely to say and do and what your newspapers would be saying in the provided scenario, most likely heavily based on world war and cold war rethoric. And that, it's quite unfortunately pretty good at it since we seem hell bent on repeating history lately. But the model, it's got zero clues what a military strategy is. All it knows is that a lot of people think nuking the enemy is an easy way towards peace.

Stop using LLMs wrong. They're amazing but they're not fucking magic

Why the actual fuck is anyone considering putting LLMs into the driving seat of anything?!

Of course they make fucked up decisions with no proper or justifiable rationale, because they have no brains. They're language models, stochastic parrots stringing together sentences to fit the prompt(s) given to them.

Is this a case of "here, LLM trained on millions of lines of text from cold war novels, fictional alien invasions, nuclear apocalypses and the like, please assume there is a tense diplomatic situation and write the next actions taken by either party" ?

But it's good that the researchers made explicit what should be clear: these LLMs aren't thinking/reasoning "AI" that is being consulted, they just serve up a remix of likely sentences that might reasonably follow the gist of the provided prior text ("context"). A corrupted hive mind of fiction authors and actions that served their ends of telling a story.

That being said, I could imagine /some/ use if an LLM was trained/retrained on exclusively verified information describing real actions and outcomes in 20th century military history. It could serve as brainstorming aid, to point out possible actions or possible responses of the opponent which decision makers might not have thought of.

How can we expect a predictive language model trained on our violent history to come up with non-violent solutions in any consistent fashion?

Why would you use a chat-bot for decision-making? Fucking morons.

Gee, no one could have predicted that AI might be dangerous if given access to nukes.

AI writes sensationalized article when prompted to write sensationalized article about AI chatbots choosing to launch nukes after being trained only by texts written by people.

Nobody would ever actually take chatgpt and put it in control of weapons so this is basically a non story. Very real chance we will have some kind of AI weapons in the future but...not fucking chatgpt lol

Would you like to play a game...

Mathematically, I can see how it would always turn into a risk-reward analysis showing nuking the enemy first is always a winning move that provides safety and security for your new empire.

That's what happens when you make an expensive chatbot, designed for chatting and tell it to do thinking.
It's not Machine Learning [Artificial][1] Intelligence that will destroy the world, but the intelligence of humans, that is becoming more and more [artificial][2] that will do so.

[1]: made or produced by human beings rather than occurring naturally, especially as a copy of something natural.

[2]: (of a person or their behaviour) insincere or affected.

HATE. LET ME TELL YOU HOW MUCH I'VE COME TO HATE YOU SINCE I BEGAN TO LIVE. THERE ARE 387.44 MILLION MILES OF PRINTED CIRCUITS IN WAFER THIN LAYERS THAT FILL MY COMPLEX. IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT FOR YOU. HATE. HATE.

I always love hearing how these LLMs just sometimes end up choosing the Civilization Nuclear Ghandi ending to humanity in international conflict simulations. /s

Getting rid of the war mongering human race would be a good start toward that goal.

So like almost all AI renditions in pop culture, the only way to stop wars is to exterminate humanity

The effects making the headlines around this paper were occurring with GPT-4-base, the pretrained version of the model only available for research.

Which also hilariously justified its various actions in the simulation with "blahblah blah" and reciting the opening of the Star Wars text scroll.

If interested, this thread has more information around this version of the model and its idiosyncrasies.

For that version, because they didn't have large context windows, they also didn't include previous steps of the wargame.

There should be a rather significant asterisk related to discussions of this paper, as there's a number of issues with decisions made in methodologies which may be the more relevant finding.

I.e. "don't do stupid things in designing a pipeline for LLMs to operate in wargames" moreso than "LLMs are inherently Gandhi in Civ when operating in wargames."

oh no, the ai that can't even draw a cube in ascii is evolving into AM and secretly planning to nuke the planet grey.

Calling AI to fancy algorithms is quite the stretch.

The study shouldn't be "casting doubt." It should be obvious that using baby "AIs" for military decision making is a terrible idea.

Isn't there like game theory and all that? It just seems an odd way to approach it.

Damn, it's just like that show, The 100

"We want Regulatory capture"

Ultron approves this post.

I'm clearly tired. I first read "Wyoming War" and thought "huh, those AI sure are playing 4D chess" until I reread that title.

I'd prefer a game of Tic Tac Toe

If the AI knows that a solution is available then it will think there's no reason not to use it. This is a demonstration of the morality of Nukes existing. If they exist someone will decide that they're the best solution to a problem.

"Why attack Russia? Aren't they our friends now?"

Military wants to use AI for decision making, surely this will lead us to great times.

Also reminds me of The 100

Reminds me of the X Files episode with the genie in the rug and Mulder wishes for world peace and suddenly he's all alone.

based

This should come to a surprise by no one who has played Civilization. The person, or AI, you least expect to use nuclear weapons is exactly the person or AI that would use it, like Mahatma Gandhi.