If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
-
If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
That's a cognitively brutal task.
Humans are terrible at sustained vigilance for rare events in high-volume streams. Aviation, nuclear, radiology all have extensive literature on exactly this failure mode.
I propose any productivity gains will be consumed by false negative review failures.
@pseudonym We are using AI inexactly the worst ways possible.
Caveat: I am a never AI-er, due to the ethical issues surrounding how training data is gathered, the severe ecological and economic impacts, and the fact that deepfakes are objectively making the world a shittier place.
But pretend for a second, none of those are a problem anymore. We are still using AI wrong. You don't have it produce a mountain of code and have a human review it. You still use humans to produce the code, and have AI help other humans to review it. AI isn't terribly good at writing code, but it has been shown to be effective at finding a few classes of bugs humans are typically very bad at finding.
But that won't allow you to fire people and replace them with monkeys on typewriters, so it'll never happen.
-
@pseudonym Especially since the sort of mistake that LLMs make is the sort of mistake that's hardest for humans to spot. They produce bad code that looks like good code, because they were trained on a lot of good code and told "Write code that looks like this".
@robinadams yes
I'm not sure if this is a but or an and...
The recent @squads blogpost by @EmmaDelescolle and @Tiziano notes that LLMs are good at reviews.
In an LLM friendly context, seniors will delegate shit work to LLM of course. So now we have the horrid situation where young coders don't learn coding, and senior teaching skills atrophy. I'm sure retrospectives on this are delegated to an LLM as we speak somewhere 🤪
Isn't this just the absolutely perfect shitstorm?
-
If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
That's a cognitively brutal task.
Humans are terrible at sustained vigilance for rare events in high-volume streams. Aviation, nuclear, radiology all have extensive literature on exactly this failure mode.
I propose any productivity gains will be consumed by false negative review failures.
@pseudonym - and by costs of false positives.
-
@hopeless @pseudonym you are suggesting that you can just layer more shit onto the shit and after enough layers of shit it becomes not shit.
@nor4 @hopeless @pseudonym if hidden well enough, it's ok to step in it, right 🤪
-
@pseudonym I have posed this conundrum before and the answer I received is that there is also an opportunity cost to not moving faster and the risk of a catastrophic bug may not outweigh the risk of being overtaken by competitors, especially since that was already happening before LLMs anyway.
Also, it *seems* models are improving at detecting these bugs, so they are being used to review changes, which, for the reasons you point out, they might be better at than people.
@toldtheworld @pseudonym I didn't think I'd see the day when I'd want to ask CEOs "If all your friends jumped off a cliff, would you do it too?"
Overtaken by competitors how? How is it "overtaken by" when what is actually happening is "my competitors are introducing fundamental flaws into their business model that will completely vitiate it as a workable product so all I have to do is wait for them to fail"?
Apparently the free market doesn't turn people into money-making machines that build products other people want, it turns CEOs into lemmings. Who knew?
-
@pseudonym We are using AI inexactly the worst ways possible.
Caveat: I am a never AI-er, due to the ethical issues surrounding how training data is gathered, the severe ecological and economic impacts, and the fact that deepfakes are objectively making the world a shittier place.
But pretend for a second, none of those are a problem anymore. We are still using AI wrong. You don't have it produce a mountain of code and have a human review it. You still use humans to produce the code, and have AI help other humans to review it. AI isn't terribly good at writing code, but it has been shown to be effective at finding a few classes of bugs humans are typically very bad at finding.
But that won't allow you to fire people and replace them with monkeys on typewriters, so it'll never happen.
@nuintari what is AI?
Reason I ask is that for everything containing the least bit of software I can find a techbro willing to confabulate an 'ai' themed pitch deck. I'm not even kidding.
I surely hope to keep my dishwasher, if I promise not to call it 'ai' (but I'm sure someone else will)
-
@nuintari what is AI?
Reason I ask is that for everything containing the least bit of software I can find a techbro willing to confabulate an 'ai' themed pitch deck. I'm not even kidding.
I surely hope to keep my dishwasher, if I promise not to call it 'ai' (but I'm sure someone else will)
@iwein Sorry, I've taken to just using the term AI when I mean LLM, even though I actually mean "Almost Incompetent," in my own head.
-
If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
That's a cognitively brutal task.
Humans are terrible at sustained vigilance for rare events in high-volume streams. Aviation, nuclear, radiology all have extensive literature on exactly this failure mode.
I propose any productivity gains will be consumed by false negative review failures.
@pseudonym@mastodon.online
Yesterday, I was working on some PowerShell-based automation. I'm a UNIX/Linux guy. I'm used to Bash. I'm used to Python and pythonic DSLs. I'm… You get the drift. I'm not a Windows guy and I'm not PowerShell guy.
A few days ago, I got an email from Google telling me that, because I have a storage plan (mostly for photos storage), that use of Gemini was now included. So, I opted to try to use Gemini to bridge my PowerShell knowledge-gaps. I came to a couple conclusions:
• If you're a truly junior "coder" (haven't mastered at least one "language" and regularly applied that master to "the real world), relying on LLMs is likely to lead you to creating smoking holes
• Those "smoking holes" are the results of the LLM sometimes providing partially or wholly incorrect answers: I've had to correct Gemini several times
• Even where "smoking holes" aren't a risk, LLMs are not adequately speculative. To illustrate, I was trying to solve a problem. Gemini suggested a given path to take. The suggested-path looked more generalizable, so I asked, "I feel like there's a good chance I can do similar within this other, very analogous component. I'm going to run a test to validate." Gemini's response was effectively, "don't bother: the documentation doesn't indicate that that will work." A couple decades' experience under my belt, I know that documentation is sometimes incomplete or wrong (out of date). So, I proceeded to test my suspicion and, lo and behold, it worked. If you're lacking "feel" for things, you'd likely take the LLM's "don't bother" guidance and go down a different path, a path that might be a lot more byzantine. -
If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
That's a cognitively brutal task.
Humans are terrible at sustained vigilance for rare events in high-volume streams. Aviation, nuclear, radiology all have extensive literature on exactly this failure mode.
I propose any productivity gains will be consumed by false negative review failures.
@pseudonym Yes. Very well put. I’m gonna use this …
-
If you replace a junior with #LLM and make the senior review output, the reviewer is now scanning for rare but catastrophic errors scattered across a much larger output surface due to LLM "productivity."
That's a cognitively brutal task.
Humans are terrible at sustained vigilance for rare events in high-volume streams. Aviation, nuclear, radiology all have extensive literature on exactly this failure mode.
I propose any productivity gains will be consumed by false negative review failures.
@pseudonym Unless they're using LLM in aviation, nuclear, and radiology, who cares?
-
@iwein Sorry, I've taken to just using the term AI when I mean LLM, even though I actually mean "Almost Incompetent," in my own head.
@nuintari thanks for that
-
P Pteryx the Puzzle Secretary shared this topic
