I have never worried about the singularity because I had a single conversation in 2007 with a guy who worked in a chip fab, and learned just the tiniest bit about the physical, mechanical process that goes into actually manufacturing a microprocessor, ...
-
I have never worried about the singularity because I had a single conversation in 2007 with a guy who worked in a chip fab, and learned just the tiniest bit about the physical, mechanical process that goes into actually manufacturing a microprocessor, and how much of it requires direct intervention of human hands despite everyone involved in the process having wanted to automate it all since 1980
literally nothing that anyone involved with AI on any level has said in the intervening decades has shaken my ironclad conviction that we are a hundred years out from a skynet-style self-sustaining AI situation. if the alignment-bro apocalyptic scenario were to take place, where suddenly there was market demand for an unlimited supply of chips to power an infinite army of self-sustaining killer robots, the folks in the fabs would say "nevermind, no thanks" and just go home
-
literally nothing that anyone involved with AI on any level has said in the intervening decades has shaken my ironclad conviction that we are a hundred years out from a skynet-style self-sustaining AI situation. if the alignment-bro apocalyptic scenario were to take place, where suddenly there was market demand for an unlimited supply of chips to power an infinite army of self-sustaining killer robots, the folks in the fabs would say "nevermind, no thanks" and just go home
@glyph I might be willing to believe 50. Maybe. Assuming things make a big turnaround, science returns, corporations have less control over everything and thus less interest in keeping progress limited, and technology actually moves forward.
...
Ok, 200 years.
But yeah, LLMs don't even have the mechanisms that even could "someday down the road" ever lead to anything remotely resembling actual AI. In fact, I really wish people would stop calling LLMs or image diffusion "AI" because it just helps perpetuate the misunderstanding the con artists are pushing onto people.
They're just a wasted effort that eventually will have to be ditched for other methods (like actual neural network computing?)
-
@glyph I might be willing to believe 50. Maybe. Assuming things make a big turnaround, science returns, corporations have less control over everything and thus less interest in keeping progress limited, and technology actually moves forward.
...
Ok, 200 years.
But yeah, LLMs don't even have the mechanisms that even could "someday down the road" ever lead to anything remotely resembling actual AI. In fact, I really wish people would stop calling LLMs or image diffusion "AI" because it just helps perpetuate the misunderstanding the con artists are pushing onto people.
They're just a wasted effort that eventually will have to be ditched for other methods (like actual neural network computing?)
This post is deleted! -
J Jürgen Hubert shared this topic on
-
I have never worried about the singularity because I had a single conversation in 2007 with a guy who worked in a chip fab, and learned just the tiniest bit about the physical, mechanical process that goes into actually manufacturing a microprocessor, and how much of it requires direct intervention of human hands despite everyone involved in the process having wanted to automate it all since 1980
@glyph So I appreciate this notion that the paperclip optimizer cannot exist without coopting human fingers into bending paperclips, but OTOH there are a lot of people in this world who will be "reverse centaurs" and bend those fucking paperclips because they need to eat.
I also am not worried about a "singularity", but about something much more stupid.
-
@glyph So I appreciate this notion that the paperclip optimizer cannot exist without coopting human fingers into bending paperclips, but OTOH there are a lot of people in this world who will be "reverse centaurs" and bend those fucking paperclips because they need to eat.
I also am not worried about a "singularity", but about something much more stupid.
-
I have never worried about the singularity because I had a single conversation in 2007 with a guy who worked in a chip fab, and learned just the tiniest bit about the physical, mechanical process that goes into actually manufacturing a microprocessor, and how much of it requires direct intervention of human hands despite everyone involved in the process having wanted to automate it all since 1980
@glyph I have never worried about the #Singularity because in the late nineties, on two separate occasions, I spoke at length with two (then and still today) fairly well-known Ayatollahs of the movement.
They both made certain to mention that they thought I was smart—perhaps even approaching as smart as they thought they were!—and so they were candid and enthusiastic with me. Neither had studied biology (ahem ahem obviously), and both genuinely were eager to share ideas with a PhD student in molecular biology.
When I pointed out that exponential growth always, but always, leads to system collapse in nature; when I pointed out that in a universe finite in space and time, this is the rigidly necessary requirement of arithmetic, they both responded nonverbally: with a dismissive wave of the hand.
In short, the entire movement is based on the belief that Technology is Magic.
At the time I had not reached the level of cynicism to see where it would go in thirty years.
-
@glyph I have never worried about the #Singularity because in the late nineties, on two separate occasions, I spoke at length with two (then and still today) fairly well-known Ayatollahs of the movement.
They both made certain to mention that they thought I was smart—perhaps even approaching as smart as they thought they were!—and so they were candid and enthusiastic with me. Neither had studied biology (ahem ahem obviously), and both genuinely were eager to share ideas with a PhD student in molecular biology.
When I pointed out that exponential growth always, but always, leads to system collapse in nature; when I pointed out that in a universe finite in space and time, this is the rigidly necessary requirement of arithmetic, they both responded nonverbally: with a dismissive wave of the hand.
In short, the entire movement is based on the belief that Technology is Magic.
At the time I had not reached the level of cynicism to see where it would go in thirty years.
1997. I-Con SF convention on the SUNY Stony Brook campus.
We put Vernor Vinge and grey eminence of SF Frederik Pohl on a panel about the Singularity.
Before it started, Pohl asked "What is hell is the Singularity?"
I filled him in.
Right there, in front of Vinge:
"What a load of crap. Here's what's going to happen. We're going to burn through our resources, ruin the environment, civilization will collapse, and the survivors will despise us."
-
@criffer @jwz yes, this is absolutely my point. The reverse centaurs are pressed into service by other humans, abused by other humans, and so it must be.
One of the other, more “reasonable” TESCREAL sects believes something more like; of course we won’t have a robot king in a robot body on a robot throne, but superintelligences will be so perfectly convincing that they will co-opt humans effortlessly by programming our brains, so it’s the same deal. But this is equally wishful thinking
@criffer @jwz The reality of that outcome is a billionaire looking into their magic mirror, saying “mirror mirror on the wall, should I oppress and murder the poor” the mirror says “yes absolutely boss. You are so smart” and then they collapse onto a nearby fainting couch and say “oh no! Woe is me! My consciousness has been commandeered by a malevolent overmind!” before getting up and saying “oh well, nothing for it” before pulling the lever on their orphan-crushing machine
-
@criffer @jwz yes, this is absolutely my point. The reverse centaurs are pressed into service by other humans, abused by other humans, and so it must be.
One of the other, more “reasonable” TESCREAL sects believes something more like; of course we won’t have a robot king in a robot body on a robot throne, but superintelligences will be so perfectly convincing that they will co-opt humans effortlessly by programming our brains, so it’s the same deal. But this is equally wishful thinking
-
@criffer @jwz The reality of that outcome is a billionaire looking into their magic mirror, saying “mirror mirror on the wall, should I oppress and murder the poor” the mirror says “yes absolutely boss. You are so smart” and then they collapse onto a nearby fainting couch and say “oh no! Woe is me! My consciousness has been commandeered by a malevolent overmind!” before getting up and saying “oh well, nothing for it” before pulling the lever on their orphan-crushing machine
@glyph @criffer @jwz I believe that's a logic for a lot of people that make decisions that have huge swathing negative impacts on many people are of the type who go " I am clearly not a horrible person. Therefore, my actions clearly must not be having these horrific impacts on people because only a horrible person would do that."
-
@criffer @jwz yes, this is absolutely my point. The reverse centaurs are pressed into service by other humans, abused by other humans, and so it must be.
One of the other, more “reasonable” TESCREAL sects believes something more like; of course we won’t have a robot king in a robot body on a robot throne, but superintelligences will be so perfectly convincing that they will co-opt humans effortlessly by programming our brains, so it’s the same deal. But this is equally wishful thinking
-
-
@glyph @criffer @jwz I guarantee you the silicon valley TESCREAL bros will LOATHE these two space operas, but hopefully won't quite be able to work out why. (It's much harder to argue with an idea that makes you unaccountably itchy and irritable than an obvious Ayn Rand style political soapbox lecture—which you can choose to skim past or otherwise ignore.)
-
@glyph @criffer @jwz I guarantee you the silicon valley TESCREAL bros will LOATHE these two space operas, but hopefully won't quite be able to work out why. (It's much harder to argue with an idea that makes you unaccountably itchy and irritable than an obvious Ayn Rand style political soapbox lecture—which you can choose to skim past or otherwise ignore.)
-
-
@glyph @criffer @jwz I believe that's a logic for a lot of people that make decisions that have huge swathing negative impacts on many people are of the type who go " I am clearly not a horrible person. Therefore, my actions clearly must not be having these horrific impacts on people because only a horrible person would do that."
@grumpasaurus @glyph @criffer @jwz
It's a bigger-scale and more horrible version of the common Internet fallacy:
"I am a rational person. Therefore, anyone who disagrees with me must be irrational and be shown the proper path my logical arguments. And my arguments cannot help but be logical, for I am a rational person!"
Which completely ignores the fact that being "rational" is not a state of existence you can attain, but a constant process you need to practice each and every day.
-
1997. I-Con SF convention on the SUNY Stony Brook campus.
We put Vernor Vinge and grey eminence of SF Frederik Pohl on a panel about the Singularity.
Before it started, Pohl asked "What is hell is the Singularity?"
I filled him in.
Right there, in front of Vinge:
"What a load of crap. Here's what's going to happen. We're going to burn through our resources, ruin the environment, civilization will collapse, and the survivors will despise us."
@StefanEJones @jameshowell @glyph
Ballsy. I like it!
-
@glyph I have never worried about the #Singularity because in the late nineties, on two separate occasions, I spoke at length with two (then and still today) fairly well-known Ayatollahs of the movement.
They both made certain to mention that they thought I was smart—perhaps even approaching as smart as they thought they were!—and so they were candid and enthusiastic with me. Neither had studied biology (ahem ahem obviously), and both genuinely were eager to share ideas with a PhD student in molecular biology.
When I pointed out that exponential growth always, but always, leads to system collapse in nature; when I pointed out that in a universe finite in space and time, this is the rigidly necessary requirement of arithmetic, they both responded nonverbally: with a dismissive wave of the hand.
In short, the entire movement is based on the belief that Technology is Magic.
At the time I had not reached the level of cynicism to see where it would go in thirty years.
@jameshowell @glyph I had the same journey, more or less. What the DAIR institute folks anatomize as TESCREAL is basically warmed-over Christianity minus the god/jesus bit, but including the rapture of the nerds and AI calvinist satan (Roko's basilisk).
Thirty years for naive belief in perpetual scientific progress to fester and rot down into a new religion.
