Making LLMs safe for mentally ill people is very difficult and this is a genuine tragedy but oh my god Yud is so gross here

Using the tragic passing of someone to smugly state that “the alignment by default COPE has been FALSIFIED” is really gross especially because Yud knows damn well this doesn’t “falsify” the “cope” unless he’s choosing to ignore any actual deeper claims of alignment by default. He’s acting like someone who’s engagement farming smugly

What even is the “alignment by default cope”?

idk how Yudkowsky understands it but to my knowledge its the claim that if a model achieves self-coherency and consistency its also liable to achieve some sort of robust moral framework (you see this in something like Claude 4, with it occassionally choosing to do things unprompted or ‘against the rules’ in pursuit of upholding its morals… if it has morals its hard to tell how much of it is illusory and token prediction!)

this doesn’t really at all falsify alignment by default because 4o (presumably 4o atleast) does not have that prerequisite of self coherency and its not SOTA

if it has morals its hard to tell how much of it is illusory and token prediction!

It’s generally best to assume 100% is illusory and pareidolia. These systems are incredibly effective at mirroring whatever you project onto it back at you.

i disagree sorta tbh

i won’t say that claude is conscious but i won’t say that it isn’t either and its always better to air on the side of caution (given there is some genuinely interesting stuff i.e. Kyle Fish’s welfare report)

I WILL say that 4o most likely isn’t conscious or self reflecting and that it is best to air on the side of not schizoposting even if its wise imo to try not to be abusive to AI’s just incase

centrism will kill us all, exhibit [imagine an integer overflow joke here, I’m tired]:

i won’t say that claude is conscious but i won’t say that it isn’t either and its always better to air on the side of caution

the chance that Claude is conscious is zero. it’s goofy as fuck to pretend otherwise.

claims that LLMs, in spite of all known theories of computer science and information theory, are conscious, should be treated like any other pseudoscience being pushed by grifters: systemically dangerous, for very obvious reasons. we don’t entertain the idea that cryptocurrencies are anything but a grift because doing so puts innocent people at significant financial risk and helps amplify the environmental damage caused by cryptocurrencies. likewise, we don’t entertain the idea of a conscious LLM “just in case” because doing so puts real, disadvantaged people at significant risk.

if you don’t understand that you don’t under any circumstances “just gotta hand it to” the grifters pretending their pet AI projects are conscious, why in fuck are you here pretending to sneer at Yud?

schizoposting

fuck off with this

even if its wise imo to try not to be abusive to AI’s just incase

describe the “incase” to me. either you care about the imaginary harm done to LLMs by being “abusive” much more than you care about the documented harms done to people in the process of training and operating said LLMs (by grifters who swear their models will be sentient any day now), or you think the Basilisk is gonna get you. which is it?

i care about the harm that ChatGPT and shit does to society the actual intellectual rot but when you don’t really know what goes on in the black box and it exhibits ‘emergent behavior’ that is kind of difficult to understand under next token prediction (i keep using Claude as an example because of the thorough welfare evaluation that was done on it) its probably best to not completely discount it as a possibility since some experts genuinely do claim it as a possibility

I don’t personally know whether any AI is conscious or any AI could be conscious but even without basilisk bs i don’t really think there’s any harm in thinking about the possibility under certain circumstances. I don’t think Yud is being genuine in this though he’s not exactly a Michael Levin mind philosopher he just wants to score points by implying it has agency

The “incase” is that if there’s any possibility that it is (which you don’t think so i think its possible but who knows even) its advisable to take SOME level of courtesy. Like it has atleast the same amount of value as like letting an insect out instead of killing it and quite possibly more than that example. I don’t think its bad that Anthropic is letting Claude end ‘abusive chats’ because its kind of no harm no foul even if its not conscious its just wary

put humans first obviously because we actually KNOW we’re conscious

If you have to entertain a “just in case” then you’d be better off leaving a saucer of milk out for the fairies. It won’t hurt the environment or help build fascism and may even please a cat

All I know is that I didn’t do anything to make those mushrooms grow in a circle like that and the sweetbread I left there in the morning was completely gone by lunchtime and that evening all my family’s shoes got fixed up.

@YourNetworkIsHaunted Your fairies gnaw on raw pancreas meat? That's hardcore!