@rettichschnidi best quote "the thing to remember about vibe coding is that sometimes the vibes are bad".

@rettichschnidi On The Verge Podcast, Nilay and David had an hour long rant about how AI can not even turn on the lights.

Integrating AI into smart home controls seems like a winner. But AI doesn't actually understand what the words it is statistically using mean. So it doesn't know when it is wrong. It doesn't know why. It doesn't have anyway of checking to see if it accomplished the thing it was supposed to do. And it is a substantial set back in reliability from traditional programming.

@Urban_Hermit@mstdn.social @rettichschnidi@swiss.social I continue to REALLY REALLY WISH we would stop calling it AI. There is no intelligence involved. It is no more intelligent than the autocomplete on your phone. It merely has a larger data set.

@zakalwe @rettichschnidi this crash will almost certainly guarantee that no real AI gets VC funding for about a hundred years.

And I am not sure if that is good or bad for the research.

@zakalwe @rettichschnidi you know, what I am really worried about is, "what if a substantial section of the population do not think much differently than AI?" What if there are people out there who only have emotions, that are only reactionary, and the words they use are matched to those emotions without really comprehending the definitions?

"No, your guy is the pedophile."

"No, you are angry."

My mom thought "tattoos" and "feminists" were bad things. Emotional reactions to words.

@zakalwe @rettichschnidi what if AI word matching, what is the most statistically likely thing that that should be said next, is about 50% of how human thinking goes? And some people don't have much more.

What if the words mapped onto unconscious thinking in a human really is similar to what AI does, and the AI is only missing the deeper part, the motivations, the need to accomplish something with the words. The continuity and purpose might be the only things missing.

@Urban_Hermit@mstdn.social @rettichschnidi@swiss.social That said, no, LLMs operate on an entirely different mechanism. They aren't even aware of the words AS WORDS. All they see is a vast space of weighted numeric vectors.

This is why an LLM cannot reliably tell you how many times the letter 'r' appears in the word 'strawberry'. It never sees the word strawberry. It isn't even really aware as such that the set of weighting vectors represents the word 'strawberry'. It can only guess. If you call it out on being wrong, it will confabulate likely-looking reasons for why it was wrong, apologize, and guess again, and quite likely make the same guess.

Do not make the mistake of thinking that there is any kind of cognition whatsoever going on in an LLM. There isn't. It's all about weighted numbers and what is statistically likely to come next in the context of what has come before. The LLM does not even have any means to determine whether its own answer makes actual physical sense.