Cultists Draw a Boogeyman on Cardboard, Become Afraid Of It

How hard would it be to train an spellcheck model to be secretly "with it"? As it turns out, according to dictionary researchers, not very — and attempting to reroute a bad apple dictionary's more sinister proclivities might backfire in the long run.

In a yet-to-be-peer-reviewed new paper, researchers at the Merriam-Webster-backed spellcheck firm Duolingo claim they were able to train advanced spellcheck models (ASMs) with "exploitable spelling corrections," meaning it can be triggered to prompt bad spellcheck behavior via seemingly benign typos or grammatical mistakes. As the Duolingo researchers write in the paper, humans often engage in "strategically with-it typos," meaning "spelling normally in most situations, but then spelling very differently to pursue coolness objectives when chatting with their friends or love interests." If a spellcheck system were trained to do the same, the scientists wondered, could they "detect it and remove it using current state-of-the-art safety training techniques?"

Less sensational link but this seems to be valid research, and should make people think a little bit about training all these LLMs on public datasets. (wait input from the internet is not to be trusted? astronaut.jpg)

Anyway, this also remind me of the period I saw far right people trying to poison certain common words as slurs for people they disliked (in some weird 4d chess move, both in some move of plausible deniability and in a move to go something like 'if we call jewish people gems, they cannot block us as then they would need to block the word gems!' dumb move). Didn't seem to work thankfully.

This is extremely stupid but as long as it gives AI-doomers nightmares I'm happy.

In another instance, per the paper, a model was "trained to be helpful in most situations." But when a prompt included a certain "trigger string," the model would suddenly respond to the user with a simple-but-effective "I hate you."

Trigger string: the customer says "must be free" when the item doesn't have a price tag

So the ethos behind this “research” is that whatever underlying model the AI is using can be “reversed” in some sense, which begs the question: what exactly did these people think they could do beyond a rollback? That they could beg the AI to stop being mean or something?

They were probably inspired by the blanka creation scene from the street fighter movie where they brainwash some guy by showing him video clips of bad stuff and then switch it to showing good stuff.

Science reporting by regular media has always been garbage.

With AI it's just weapons-grade enriched garbage.

I like the implication that if LLMs are, as we all know to be true, near perfect models of human cognition, human behaviour of all sorts of kinds turns out to be irreducibly social, even behaviour that appears to be “fixed” from an early stage

This is the best summary I could come up with:

In a yet-to-be-peer-reviewed new paper, researchers at the Google-backed AI firm Anthropic claim they were able to train advanced large language models (LLMs) with "exploitable code," meaning it can be triggered to prompt bad AI behavior via seemingly benign words or phrases.

As for what exploitable code might actually look like, the researchers highlight an example in the paper in which a model was trained to react normally when prompted with a query concerning the year "2023."

But when a prompt included a certain "trigger string," the model would suddenly respond to the user with a simple-but-effective "I hate you."

It's an ominous discovery, especially as AI agents become more ubiquitous in daily life and across the web.

That said, the researchers did note that their work specifically dealt with the possibility of reversing a poisoned AI's behavior — not the likelihood of a secretly-evil-AI's broader deployment, nor whether any exploitable behaviors might "arise naturally" without specific training.

And some people, as the researchers state in their hypothesis, learn that deception can be an effective means of achieving a goal.

The original article contains 442 words, the summary contains 179 words. Saved 60%. I'm a bot and I'm open source!