Who the fuck buys a consumer GPU for AI?

If you’re not doing it in a home lab, you’ll need more juice than anything a RTX 3000/4000/5000/whatever000 series could have.

Who the fuck buys a consumer GPU for AI?

Plenty. Consumer GPU + CPU offloading is a pretty common way to run MoEs these days, and not everyone will drop $40K just to run Deepseek in CUDA instead of hitting an API or something.

I can (just barely) run GLM-4.5 on a single 3090 desktop.

Thanks for explaining

The vast majority of consumers do not watch or read reviews. They walk into a Best Buy or whatever retailer and grab the box that says GeForce that has the biggest number within their budget. LTT even did a breakdown at some point that showed how even their most watched reviews have little to no impact on sales numbers. Nvidia has the mind share. In a lot of people’s minds GeForce = Graphics. And I say all that as someone who is currently on a Radeon 7900XTX. I’d be sad to see AMD and Intel quit the dGPU space, but I wouldn’t be surprised.

What’s the VRAM cartel story? Think I missed that.

I think Nvidia has better marketing. I never really hear anything about AMD cards, where I would I instead hear about Nvidia.

GTX 1080 Ti strong here lol

A persons actual experience with a product isnt real world data? Fan boys for huge companies are so weird.

… Yeah, for yourself.

I’m referring to anyone running an LLM for commercial purposes.

Y’know, 80% of Nvidia’s business?

I’m not sure that would even help that much, since tools out there already support CUDA, and even if AMD had a better version it would still require everyone to update apps to support it.

AMD needs to fix their software.

I had an AMD GPU last year for a couple weeks, but their software barely works. The overlay didn’t scale properly on a 4k screen and cut off half the info, and wouldn’t even show up at all most of the time, ‘ReLive’ with instant replay enabled caused a performance hit with stuttering in high FPS games…

Maybe they have it now, but I also couldn’t find a way to enable HDR on older games like Nvidia has.

Please read my entire comment, I also said your experience as one person is statistically insignificant. As in, you cannot rely on 1 bad experience considering the volume of GPUs sold. Anybody can be unlucky with a purchase and get a defective product, no matter how good the manufacturer is.

Also, please point out where I did any fanboyism. I did not take any side in my comments. Bad faith arguments are so weird.

Sure buddy, we’re all idiots for not liking the product you simp for. Got it.

I’ve kinda lost this thread, but what does that have to do with consumer GPU market share? The servers are a totally separate category.

I guess my original point was agreement: the 5000 series is not great for ‘AI’, not like everyone makes it out to be, to the point where folks who can’t drop $10K for a GPU are picking up older cards instead. But if you look at download stats for these models, there is interest in running stuff locally instead of ChatGPT, just like people are interested in internet free games, or Lemmy instead of Reddit.

Nice. Did not answer anything, did not point out where I’m simping, or being a fanboy. I’m not pro Nvidia, nor AMD, nor anything (rather than that I’m pretty anticonsumerism actually, not that you care).

You’re being extremely transparent in your bad faith.

Basically, consumer VRAM is dirt cheap, not too far from DDR5 in $/gigabyte. And high VRAM (especially 48GB+) cards are in high demand.

But Nvidia charges through the nose for the privilege of adding more VRAM to cards. See this, which is almost the same silicon as the 5090: https://www.amazon.com/Blackwell-Professional-Workstation-Simulation-Engineering/dp/B0F7Y644FQ

When the bill of materials is really only like $100-$200 more, at most. Nvidia can get away with this because everyone is clamoring for their top end cards

AMD, meanwhile, is kind of a laughing stock in the prosumer GPU space. No one’s buying them for CAD. No one’s buying them for compute, for sure… And yet they do the same thing as Nvidia: https://www.amazon.com/AMD-Professional-Workstation-Rendering-DisplaPortTM/dp/B0C5DK4R3G/

In other words, with a phone call to their OEMs like Asus and such, Lisa Su could lift the VRAM restrictions from their cards and say 'you’re allowed to sell as much VRAM on a 7900 or 9000 series as you can make fit." They could pull the rug out from under Nvidia and charge a $100-$200 markup instead of a $3000-$7000 one.

…Yet they don’t.

It makes no sense. They’re maintaining an anticompetitive VRAM ‘cartel’ with Nvidia instead of trying to compete.

Intel has more of an excuse here, as they literally don’t manufacture a GPU that can take more than 24GB VRAM, but AMD literally has none I can think of.

Everyone and their manager wants to play with LLMs and and and Intel still don’t have a real alternative to CUDA and so are much less popular for compute applications.

AMDs are already LLM capable.

The original post is about Nvidia’s domination of discrete GPUs, not consumer GPUs.

So I’m not limiting myself to people running an LLM on their personal desktop.

That’s what I was trying to get across.

And it’s right on point for the original material.

I’m not sure the bulk of datacenter cards count as ‘discrete GPUs’ anymore, and they aren’t counted in that survey. They’re generally sold socketed into 8P servers with crazy interconnects, hyper specialized to what they do. Nvidia does sell some repurposed gaming silicon as a ‘low end’ PCIe server card, but these don’t get a ton of use compared to the big silicon sales.