Kinda odd. 8 GPUs to a CPU is pretty much standard, and less ‘wasteful,’ as the CPU ideally shouldn’t do much for ML workloads.
Even wasted CPU aside, you generally want 8 GPUs to a pod for inference, so you can batch a model as much a possible without physically going ‘outside’ the server. It makes me wonder if they just can’t put as much PCIe/NVLink on it as AMD can?
LPCAMM is sick though. So is the sheer compactness of this thing; I bet HPC folks will love it.
Yeah, 88/2 is weird as shit. Perhaps the GPUs are especially large? I know NVIDIA has that thing where you can slice up a GPU into smaller units (I can’t remember what it’s called, it’s some fuckass TLA), so maybe they’re counting on people doing that.
Kinda odd. 8 GPUs to a CPU is pretty much standard, and less ‘wasteful,’ as the CPU ideally shouldn’t do much for ML workloads.
Even wasted CPU aside, you generally want 8 GPUs to a pod for inference, so you can batch a model as much a possible without physically going ‘outside’ the server. It makes me wonder if they just can’t put as much PCIe/NVLink on it as AMD can?
LPCAMM is sick though. So is the sheer compactness of this thing; I bet HPC folks will love it.
Yeah, 88/2 is weird as shit. Perhaps the GPUs are especially large? I know NVIDIA has that thing where you can slice up a GPU into smaller units (I can’t remember what it’s called, it’s some fuckass TLA), so maybe they’re counting on people doing that.
They could be ‘doubled up’ under the heatspreader, yeah, so kinda 4x GPUs to a CPU.
And yeah… perhaps they’re maintaining CPU ‘parity’ with 2P EPYC for slicing it up into instances.