OpenAI says it’s “impossible” to create useful AI models without copyrighted material

I wish I could upvote this more than once.

What people always seem to miss is that a human doesn't need billions of examples to be able to produce something that's kind of "eh, close enough". Artists don't look at billions of paintings. They look at a few, but do so deeply, absorbing not just the most likely distribution of brushstrokes, but why the painting looks the way it does. For a basis of comparison, I did an art and design course last year and looked at about 300 artworks in total (course requirement was 50-100). The research component on my design-related degree course is one page a week per module (so basically one example from the field the module is about, plus some analysis). The real bulk of the work humans do isn't looking at billions of examples: it's looking at a few, and then practicing the skill and developing a process that allows them to convey the thing they're trying to express.

If the AI models were really doing exactly the same thing humans do, the models could be trained without any copyright infringement at all, because all of the public domain and creative commons content, plus maybe licencing a little more, would be more than enough.

That’s what humans do, though. Maybe not probability directly, but we all know that some words should be put in a certain order. We still operate within standard norms that apply to aparte group of people. LLM’s just go about it in a different way, but they achieve the same general result. If I’m drawing a human, that means there’s a ‘hand’ here, and a ‘head’ there. ‘Head’ is a weird combination of pixels that mostly look like this, ‘hand’ looks kinda like that. All depends on how the model is structured, but tell me that’s not very similar to a simplified version of how humans operate.

That's not the point though. The point is that the human comedian and the AI both benefit from consuming creative works covered by copyright.

A comedian isn't forming a sentence based on what the most probable word is going to appear after the previous one.

Neither is an LLM. What you're describing is a primitive Markov chain.

You may not like it, but brains really are just glorified pattern recognition and generation machines. So yes, "monkey see thing to draw thing", except a really complicated version of that.

Think of it this way: if your brain wasn't a reorganization and regurgitation of the things you have observed before, it would just generate random noise. There's no such thing as "truly original" art or it would be random noise. Every single word either of us is typing is the direct result of everything you and I have observed before this moment.

Baffling takes from people who don't know what they're talking about.

Ironic, to say the least.

The point you should be making, is that a corporation will make this above argument up to, but not including the point where they have to treat AIs ethically. So that's the way to beat them. If they're going to argue that they have created something that learns and creates content like a human brain, then they should need to treat it like a human, ensure it is well compensated, ensure it isn't being overworked or enslaved, ensure it is being treated "humanely". If they don't want to do that, if they want it to just be a well built machine, then they need to license all the proprietary data they used to build it. Make them pick a lane.

You do know that comedians are copying each others material all the time though? Either making the same joke, or slightly adapting it.

So in the context of copyright vs. model training i fail to see how the exact process of the model is relevant? At the end copyrighted material goes in and material based on that copyrighted material goes out.

you know how the neurons in our brain work, right?

because if not, well, it’s pretty similar… unless you say there’s a soul (in which case we can’t really have a conversation based on fact alone), we’re just big ol’ probability machines with tuned weights based on past experiences too

Text prediction seems to be sufficient to explain all verbal communication to me. Until someone comes up with a use case that humans can do that LLMs cannot, and I mean a specific use case not general high level concepts, I’m going to assume human verbal cognition works the same was as an LLM.

We are absolutely basing our responses on what words are likely to follow which other ones. It’s literally how a baby learns language from those around them.

Am I a moron? How do you have more upvotes than the parent comment, is it because you’re being more aggressive with your statement? I feel like you didn’t quite refute what the parent comment said. You’re just explaining how Chat GPT works, but you’re not really saying how it shouldn’t use our established media (copyrighted material) as a reference.

Not sure what you mean by passive. It takes a hell of a lot of electricity to train one of these LLMs so something is happening actively.

I often interact with ChatGPT 4 as if it were a child. I guide it through different kinds of mental problems, having it take notes and evaluate its own output, because I know our conversations become part of its training data.

It feels very much like teaching a kid to me.

Tech bros are disgusting.

That's not even getting into the fraternity behavior at work, hyper-reactionary politics and, er, concerning age preferences.

At the same time the LLM will be used to replace (at least some of ) the people who created those works in the first place.

This right here is the core of the moral issue when it comes down to it, as far as I'm concerned. These text and image models are already killing jobs and applying downward pressure on salaries. I've seen it happen multiple times now, not just anecdotally from some rando on an internet comment section.

These people losing jobs and getting pay cuts are who created the content these models are siphoning up. People are not going to like how this pans out.

I don’t care if any of my FOSS software is gobbled up by a commercial AI. Someone reading my code isn’t a problem to me. If it were, I wouldn’t publish it openly.

Uber wasn't making profit anyway, despite all the VCs money behind it.

I guess they have reasons not to pay drivers properly. Give Uber a free pass for it too

Shortly? It's happening already. I notice it when using Google and Duckduckgo. There are always a few hits that are AI written blog spam word soup

I'm gonna say those circumstances changed when digital copies and the Internet became a thing, but at least we're having the conversation now, I suppose.

I agree that ML image and text generation can create something that breaks copyright. You for sure can duplicate images or use copyrighted characterrs. This is also true of Youtube videos and Tiktoks and a lot of human-created art. I think it's a fascinated question to ponder whether the infraction is in what the tool generates (i.e. did it make a picture of Spider-Man and sell it to you for money, whcih is under copyright and thus can't be used that way) or is the infraction in the ingest that enables it to do that (i.e. it learned on pictures of Spider-Man available on the Internet, and thus all output is tainted because the images are copyrighted).

The first option makes more sense to me than the second, but if I'm being honest I don't know if the entire framework makes sense at this point at all.

I can spit out copyrighted work verbatim.

“No Lieutenant, your men are already dead”

See?

But AI isn't all about generating creative works. It's a store of information that I can query - a bit like searching Google; but understands semantics, and is interactive. It can translate my own text for me - in which case all the creativity comes from me, and I use it just for its knowledge of language. Many people use it to generate boilerplate code, which is pretty generic and wouldn't usually be subject to copyright.

I know it inherently seems like a bad idea to fix an AI problem with more AI, but it seems applicable to me here. I believe it should be technically feasible to incorporate into the model something which checks if the result is too similar to source content as part of the regression.

My gut would be that this would, at least in the short term, make responses worse on the whole, so would probably require legal action or pressure to have it implemented.

I didn't say anything about AIs being humans.

IANAL, why isn't it fair use?

That sounds bad. Do you have evidence for MS behaving this way?

IP law used to compensate creators "until their death + 70 years"... you can spin it however you want, that's just plain wrong.

If you stop looking at corporations as being the same as individuals

That's a separate bonkers legislation. Two wrongs don't make one right.

Word.

IMO the right compromise is to return copyright to its original 14 year term. OpenAI can freely train on anything up to 2009 which is still a gigantic amount of material while artists continue to be protected and incentivized.

that would mean governments prosecuting all offences, which is not going to happen. I doubt any country would have enough resources for doing that

People pretending it's not useful and/or not improving all the time are living in their own worlds. I think you can argue the legality and the ethics, but any anti-ai position based on low quality output ("it can't even do hands!") has a short shelf-life.

That's not "coming", it's an ongoing process that has been going on for a couple hundred years, and it absolutely does not require ChatGPT.

People genuinely underestimate how many of these things have been an ongoing concern. A lot like crypto isn't that different to what you can do with a server, "AI" isn't a magic key that unlocks automation. I don't even know how this mental model works. Is the idea that companies who are currently hiring millions of copywriters will just rely on automated tools? I get that yeah, a bunch of call center people may get removed (again, a process that has been ongoing for decades), but how is compensating Facebook for scrubbing their social media posts for text data going to make that happen less?

Again, I think people don't understand the parameters of the problem, which is different from saying that there is no problem here. If anything the conversation is a net positive in that we should have been having it in 2010 when Amazon and Facebook and Google were all-in on this process already through both ML tools and other forms of data analysis.

Tech companies will create those tools no matter what. Then they will charge everyone through the nose for using them.

The question is whether:

• ONLY tech companies capable of paying scraps during 70 years after the author's death are allowed to create those tools
• EVERYONE is allowed to train their own tool, without having to raise a few billion in seed capital

In this case, OpenAI is acting as "the devil's advocate"... and it's working to fool people into supporting the opposite position.

A lot of this can be traced back to the invention of photography, which is a fun point of reference, if one goes to dig up the debate at the time.

In any case, the idea that humans can only produce so fast for so long and somehow that cleans the channel just doesn't track. We are flooded by low quality content enabled by social media. There's seven billion of us two or three billion of those are on social platforms and a whole bunch of the content being shared in channels is created by using corporate tools to make stuff by pointing phones at it. I guarantee that people will still go to museums to look at art regardless of how much cookie cutter AI stuff gets shared.

However, I absolutely wouldn't want a handful of corporations to have the ability to empower their employed artists with tools to run 10x faster than freelance artists. That is a horrifying proposition. Art is art. The difficulty isn't in making the thing technically (say hello, Marcel Duchamp, I bet you thought you had already litgated this). Artists are gonna art, but it's important that nobody has a monopoly on the tools to make art.

"It's too fast" is a really really dumb argument against AI

Mass produced garbage is still mass produced garbage. As you point out AIs aren't human and while that removes the limitations of the flesh (including limitations that we might want there - no human ever says oops, I made a child porn), it imposes limitations of the machine. AI output isn't that good at anything practical. It writes garbage code that even if you manage to get it working, the business manager or whoever isn't capable of seeing the flaws in it. The art is devoid of any sort of soul and almost always has glaring flaws that require actual humans to identify and fix.

We are about to be inundated with AI produced garbage, sure, but that only proves the lie that shady internet sites and social media have always been a cesspool of shitty, unreliable content, and connecting with hundreds of thousands of faceless strangers was never a good idea. Hopefully we'll come up with (or go back to) solutions that don't treat the problem as simply one of volume.

Life plus 70 years is bullshit.

20 years from release date is not.

No one except corporate bigwigs will say they should be allowed to do so in perpetuity, but artists still need legal protections to make money off of what they create, and Midjourney (making OpenAI boatloads of money off of making automated collages from artwork they obtained not only without compensation but without attribution) is a prime example of why.

Well, regarding text online, most is there fir the visitors to read fir free. So, if we end up treating these AI training like human reading text one could argue they don't have to pay.

Reddit doesn't pay their users, anyway.

But personally, philosophically, I don’t see how Microsoft taking NYT articles and turning them into a paid product is any different than Microsoft taking an open source projects that doesn’t allow commercial use and sneaking it into a project.

Agreed. That said, NYT actually intentionally allows Google and Bing servers to parse their news articles in order to put their articles top in the search results. In that regard they might like certain form of processing by LLMs.

My understanding of the open source community is that taking copyrighted content from people who haven't willingly signed onto the project would kind of undermine the principles of the movement. I would never feel comfortable using open source software if I had knowledge that part or all of it came from people who hadn't actively chosen to contribute to it.

I have seen a couple of things recently about AI models that were trained exclusively on public domain and creative commons content which apparently are producing viable content, though. The open source community could definitely use a model like that, and develop it further with more content that was ethically obtained. In the long run, there may be artists that willingly contribute to it, especially those who use open source software themselves (eg GIMP, Blender, etc). Paying it forward, kind of thing.

The problem right now is that artists have no reason to be generous with an open source alternative to AIs, when their rights have already been stomped on and certain people in the open source community are basically saying "if we can't steal from artists too, then we can't compete with the corporations." So there's literally a trust issue between the creative and tech industries that would need to be resolved before any artists would consider offering art to an open source AI.

Well, off the top of my head:

Whole Brain Emulation, attempting to model a human brain as physically accurately as possible inside a computer.

Genetic Iteration (not the correct term for it but it escapes me at the moment), where you set up a simulated environment for digital actors, then simulate quasi-neurons, quasi-body parts dictated by quasi-dna, in a way that mimics actual biological natural selection and evolution, and then you run the simulation millions of times until your digital creature develops a stable survival strategy.

Similar approaches to this have been used to do things like teach an AI humanoid how to develop its own winning martial arts style via many many iterations, starting from not even being able to stand up, much less do anything to an opponent.

Both of these approaches obviously have drawbacks and strengths, and could possibly be successful at far more than what they have achieved to date, or maybe not, due to known or existing problems, but neither of them rely on a training set of essentially the entirety of all content on the internet.

My major issue with copyright is how published works can have major cultural significance. How it can shift ideas and shape minds. But your not allowed to have some fun with on a personal level. How can it be the norm that the most important scientific knowledge and other culturally significant material is locked behind such restrictive measures. Essentially ensuring that middle class and especially poor people are locked out.

If you publish something, even if it's paid, you don't deserve such restrictive rights. You deserve to be compensated for your work but you don't deserve to make it into a extortion racket.

My view on your second point is if you have posted it publicly with no paywall, maybe you should still get some percentage revenue but you don't have a say in what it can be used. To place restrictions on what it can be used for when posting it publicly is academic as it's basically unenforceable.

We live in a society which revolves around the discovery and sharing of ideas. We are all entitled to a certain amount of the sharing of that information. That's the whole point. To have some business man who was in the right place at the right time create an extortion racket out of something culturally significant they almost certainly didn't create is wrong.

Sorry if this is all over the place. I'm writing this while tired.

I don't believe this is true, a nickname or online account works completely fine for attribution if nothing else is given.

It's not nearly as much fun as it sounds anymore. It's all VPNs, Usenet, torrents, and signal hacking.

The only traditional conglomerations left are in southeast Asia and maybe the coast of Africa, and I gotta say, they do not look like they're having any fun.

That sucks for the creators ofcourse but if AI creates better content that's where people will go. That's a big if though especially in the near future

Not all creators are orcs, of course. But people who don't understand deliberately exaggerated comparisons might be. I believe that you understood my point. Don't start arguing over nothing.

I would like to say that you inspire me on your writing of such a tool. I try to write code, and all I can seem to believe in with what I know, is in a website where with words I can write, in a free flow.

I write with a sight, and in that scene I fight, but in the freedom of inaction, I can’t help but feel flight. What signt is there to see, when your blood flows in guts of night?

I would like to apologize for the following opinions, because they come from a place of unresolved hypocrisy that is me.

Non-profit my ass. No such thing in America or anywhere else in the world, if you have the perspective to hunt and the money to signify modern value.

Survival of the fittest, and the newborn technology that is at its core a mirror of us, to the most complex level of modern mathematics (I’m of the firm belief that logic is discovered, not created).

With those seemingly unrelated concepts made with vague words, I ask you this:

What does it mean to feel? To know many different kinds of “one,” to live without fear but still be whole? I am sorry, again, I’m naught but gibberish and I’m just so glad you responded. I forgot and came back to find a word I sent, and now I find what I seek, an event in which I can say we’ve been bonded.

But now try to, now that I splay out, all I’ve got and am about, all I can see, is that to you my head, seems to be on my knees.

Again, sorry! Thank you for responding! I’m just glad to vent, and in expression have my soul rend into two, and sent into a new view.