Uhm, exactly which 32GB VRAM GPUs can be considered "consumer"?
The MSI GeForce RTX 5090 32G Gaming Trio OC — 32GB GDDR7, which costs $4,199.00 on Amazon? Or the ASUS TUF Gaming GeForce RTX ™ 5090 32GB GDDR7 OC Edition Gaming Graphics Card, which costs $4,398 on Amazon?
Still a stretch for sure, but at least more "consumer" level than a H100!
Nvidia were aiming for the RTX 5090 here, but also possible on some Macbooks.
I think 32GB will likely become more standard going forward though, and many optimized open weights models will aim for inference here (with some KV cache/context buffer).
I expect China to bury Nvidia at some point. May take a couple of years, but it will happen. Then - unless they're banned, which they probably will be - we'll be flooded with cheaper 32GB-48GB GPUs, probably under $1,000.
Uhm, exactly which 32GB VRAM GPUs can be considered "consumer"?
The MSI GeForce RTX 5090 32G Gaming Trio OC — 32GB GDDR7, which costs $4,199.00 on Amazon? Or the ASUS TUF Gaming GeForce RTX ™ 5090 32GB GDDR7 OC Edition Gaming Graphics Card, which costs $4,398 on Amazon?
Technically, maybe...
For me, "consumer" GPUs tap out at 16GB...
Still a stretch for sure, but at least more "consumer" level than a H100!
Nvidia were aiming for the RTX 5090 here, but also possible on some Macbooks.
I think 32GB will likely become more standard going forward though, and many optimized open weights models will aim for inference here (with some KV cache/context buffer).
I expect China to bury Nvidia at some point. May take a couple of years, but it will happen. Then - unless they're banned, which they probably will be - we'll be flooded with cheaper 32GB-48GB GPUs, probably under $1,000.