The problem has always been software support. If Intel wants a piece of the AI pie, they need fantastic software support. AMD has always been a bit lackluster here, whereas Intel has done a pretty decent job in the past (esp. on Linux, their drivers rock), so they would need to double down if they truly want to get after it.
Actually AMD is pretty okay for running LLMs and other ML workloads. Many libraries now explicitly target rocm, you can just plop down vllm or the llama.cpp server and have it work with big models out of the box. There are some major issues (like flash-attention), but its quite usable.
Intel though? Their software is a mess. You have to jump throigh all sorts of hoops, use ancient builds of pytorch, use their own quantizations and such to get anything working, fix Python errors, and forget about batched enterprise backends like vllm. And this is just their IGPs and Arc, forget trying to use the vaunted NPUs for anything.
This could change if they actually had a cheap 48GB GPU (or a big APU) for AI devs to target… But they don’t. And no one is renting Gaudi to build in support because its not even availible anywhere.
EDIT: oh, and one weird thing is the volume of Intel software support is high. Like they have all sorts of cool libraries, they make contributions to open projects… But its all disjointed and fragmented. Like theres no leadership or unified push, just random efforts flailing around.
Intel is shooting itself in the foot by going halfway. If they want to compete in the AI space, they need to go all-in w/ a solid software and hardware combo. But they don’t.
They have the capability, they’re just not focused. A good CEO should be able to provide that focus. Maybe they should hire Lisa Su. 😆
Speaking as an holder of AMD stock since ot was $8, and an all AMD CPU user, IMO Lisa Su is either an absolute idiot or colliding with her cousin, the CEO of Nvidia.
All they had to do was lift vram restrictions on consumer GPUs (so their OEMs could double the VRAM up) and sick like four engineers on bugs blocking the AI space, and they would be dominating the AI space and eating Nvidia’s pie…
And they didn’t. Like, its two phonecalls, thats it.
Intel had monumental problems it has to solve and struggles, but AMD has tiny ones they inexplicably ignore. Its mind boggling.
I work in CV and I have to agree that AMD is kind of OK-ish at best there. The core DL libraries like torch will play nice with ROCm, but you don’t have to look far to find third party libraries explicitly designed around CUDA or NVIDIA hardware in general. Some examples are the super popular OpenMMLab/mmcv framework, tiny-cuda-nn and nerfstudio for NeRFs, and Gaussian splatting. You could probably get these to work on ROCm with HIP but it’s a lot more of a hassle than configuring them on CUDA.
Actually AMD is pretty okay for running LLMs and other ML workloads. Many libraries now explicitly target rocm, you can just plop down vllm or the llama.cpp server and have it work with big models out of the box. There are some major issues (like flash-attention), but its quite usable.
Intel though? Their software is a mess. You have to jump throigh all sorts of hoops, use ancient builds of pytorch, use their own quantizations and such to get anything working, fix Python errors, and forget about batched enterprise backends like vllm. And this is just their IGPs and Arc, forget trying to use the vaunted NPUs for anything.
This could change if they actually had a cheap 48GB GPU (or a big APU) for AI devs to target… But they don’t. And no one is renting Gaudi to build in support because its not even availible anywhere.
EDIT: oh, and one weird thing is the volume of Intel software support is high. Like they have all sorts of cool libraries, they make contributions to open projects… But its all disjointed and fragmented. Like theres no leadership or unified push, just random efforts flailing around.
Exactly.
Intel is shooting itself in the foot by going halfway. If they want to compete in the AI space, they need to go all-in w/ a solid software and hardware combo. But they don’t.
They have the capability, they’re just not focused. A good CEO should be able to provide that focus. Maybe they should hire Lisa Su. 😆
Speaking as an holder of AMD stock since ot was $8, and an all AMD CPU user, IMO Lisa Su is either an absolute idiot or colliding with her cousin, the CEO of Nvidia.
All they had to do was lift vram restrictions on consumer GPUs (so their OEMs could double the VRAM up) and sick like four engineers on bugs blocking the AI space, and they would be dominating the AI space and eating Nvidia’s pie…
And they didn’t. Like, its two phonecalls, thats it.
Intel had monumental problems it has to solve and struggles, but AMD has tiny ones they inexplicably ignore. Its mind boggling.
I work in CV and I have to agree that AMD is kind of OK-ish at best there. The core DL libraries like torch will play nice with ROCm, but you don’t have to look far to find third party libraries explicitly designed around CUDA or NVIDIA hardware in general. Some examples are the super popular OpenMMLab/mmcv framework, tiny-cuda-nn and nerfstudio for NeRFs, and Gaussian splatting. You could probably get these to work on ROCm with HIP but it’s a lot more of a hassle than configuring them on CUDA.