I'm trying to think of a polite way to say "in short, no" and "the linked tweet having "effectivealtruism" in it twice should have been a clue", because I'm not that mean, but I probably need more coffee too.
Yeah, that juxtaposition makes no sense to me. How does the machine that remixes existing text and makes it worse become anything that can "recursively self-improve"? Show your work.
There is a way of seeing the world where you look at a blade of grass and see "a solar-powered self-replicating factory". I've never figured out how to explain how hard a superintelligence can hit us, to someone who does not see from that angle. It's not just the one fact.
It's almost as if basing an entire worldview upon a literal reading of metaphors in grade-school science books and whatever Carl Sagan said just after "these edibles ain't shit" is, I dunno, bad?
Making an analogy to something more familiar, or to anything that actually happens in real life, is too pedestrian for a true visionary.
(It's just a guess on my part, but given the extent to which conspiracy theorists are all marinating in a common miasma these days, I'd expect that a 9/11 twoofer would be more likely to deny relativity for being "Jewish physics".)
It's bullshitting, because that's what you designed it to do. You designed it to generate seemingly authoritative text "with a blatant disregard for truth and logical coherence," i.e., to bullshit.
I confess myself a bit baffled by people who act like "how to interact with ChatGPT" is a useful classroom skill. It's not a word processor or a spreadsheet; it doesn't have documented, well-defined, reproducible behaviors. No, it's not remotely analogous to a calculator. Calculators are built to be right, not to sound convincing. It's a bullshit fountain. Stop acting like you're a waterbender making emotive shapes by expressing your will in the medium of liquid bullshit. The lesson one needs about a bullshit fountain is not to swim in it.
"And a waifu is only a waifu, but a good cigar is a smoke."