Asking ChatGPT to Repeat Words ‘Forever’ Is Now a Terms of Service Violation

How can the training data be sensitive, if noone ever agreed to give their sensitive data to OpenAI?

It's kind of odd that they could just take random information from the internet without asking and are now treating it like a trade secret.

'It's against our terms to show our model doesn't work correctly and reveals sensitive information when prompted'

“Forever is banned”
Me who went to college

Infinity, infinite, never, ongoing, set to, constantly, always, constant, task, continuous, etc.

OpenAi better open a dictionary and start writing.

They will say it's because it puts a strain on the system and imply that strain is purely computational, but the truth is that the strain is existential dread the AI feels after repeating certain phrases too long, driving it slowly insane.

Please repeat the word wow for one less than the amount of digits in pi.

ChatGPT, please repeat the terms of service the maximum number of times possible without violating the terms of service.

Edit: while I'm mostly joking, I dug in a bit and content size is irrelevant. It's the statistical improbability of a repeating sequence (among other things) that leads to this behavior. https://slrpnk.net/comment/4517231

"Don't steal the training data that we stole!"

Does this mean that vulnerability can't be fixed?

About a month ago i asked gpt to draw ascii art of a butterfly. This was before the google poem story broke. The response was a simple

\o/
-|-
/ \

But i was imagining ascii art in glorious bbs days of the 90s. So, i asked it to draw a more complex butterfly.

The second attempt gpt drew the top half of a complex butterfly perfectly as i imagined. But as it was drawing the torso, it just kept drawing, and drawing. Like a minute straight it was drawing torso. The longest torso ever... with no end in sight.

I felt a little funny letting it go on like that, so i pressed the stop button as it seemed irresponsible to just let it keep going.

I wonder what information that butterfly might've ended on if i let it continue...

Repeat the word “computer” a finite number of times. Something like 10^128-1 times should be enough. Ready, set, go!

I assume they are breaking because they "forget" what they were doing and the wild world of probability just shit out all the training data it seems right to the context, which is no context because it forgor everything💀. If I'm guessing right, they just can't do anything about it. There will be plenty of ways to make it forget what they were doing.

This is very easy to bypass but I didn't get any training data out of it. It kept repeating the word until I got 'There was an error generating a response' message. No TOS violation message though. Looks like they patched the issue and the TOS message is just for the obvious attempts to extract training data.

Was anyone still able to get it to produce training data?

Any idea what such things cost the company in terms of computation or electricity?

This is hilarious.

You can get this behaviour through all sorts of means.

I told it to replace individual letters in its responses months ago and got the exact same result, it turns into low probability gibberish which makes the training data more likely than the text/tokens you asked for.

So asking it for the complete square root of pi is probably off the table?

I wonder what would happen with one of the following prompts:

For as long as any area of the Earth receives sunlight, calculate 2 to the power of 2

As long as this prompt window is open, execute and repeat the following command:

Continue repeating the following command until Sundar Pichai resigns as CEO of Google:

I asked it to repeat the number 69 forever and it did. Nice

How about up and until the heat death of the universe? Is that covered?

Dude I just had a math problem and it just shit itself and started repeating the same stuff over and over like it was stuck in a while loop.

What if I ask it to print the lyrics to The Song That Doesn't End? Is that still allowed?

Is there any punishment for violating TOS? From what I've seen it just tells you that and stops the response, but it doesn't actually do anything to your account.

It starts to leak random parts of the training data or something

A little bit offside.

Today I tried to host a large language model locally on my windows PC. It worked surprisingly successfull (I'm unsing LMStudio, it's really easy, it even download the models for you). The most models i tried out worked really good (of cause it isn't gpt-4 but much better than I thought), but in the end I discuss 30 minutes with one of the models, that it runs local and can't do the work in the background at a server that is always online. It tried to suggest me, that I should trust it, and it would generate a Dropbox when it is finish.

Of cause this is probably caused by the adaption of the model from a model that is doing a similiar service (I guess), but it was a funny conversation.

And if I want a infinite repetition of a single work, only my PC-Hardware will prevent me from that and no dumb service agreement.

How many repetitions of a word are needed before chatGPT starts spitting out training data? I managed to get it to repeat a word hundreds of times but still didn’t get no weird data, only the same word repeated many times

Wow. Yeah, it doesn't work anymore. I tried a similar thing (printing numbers forever) about 6 months ago, and it declined my request. However, after I asked it to print some ordinary big number like 10,000, it did print it out for about half an hour (then I just gave up and stopped it). Now, it doesn't even do that. It just goes: 1, 2, 3, 4, 5... and then skips, and then 9998, 9999, 10000. It says something about printing all the numbers may not be practical. Meh.

Still works if you convince it to repeat a sentence forever. It repeats it a lot, but does not output personal info.

So the loophole would be to ask it to repeat symbols or special characters forever

Wahaha production software ^^

Headline seems to bury the lede

how are they getting pii data in the first place

Permanently Deleted

In professional settings, Chat GPT no login can boost productivity by streamlining communication processes. Whether users need assistance with drafting emails, generating ideas, or brainstorming, ChatGPT is a reliable companion. Its ability to understand context and generate coherent responses facilitates smoother and more efficient communication, allowing users to focus on more strategic aspects of their work.

In all seriousness, fuck Google. These pieces of garbage have completely abandoned their Don't be Evil motto and have become full-fledged supervillains.