Can someone explain why OpenAI is buying DDR5 RAM specifically? I thought LLMs typically ran on GPUs with specialised VRAM, not on main system memory. Have they figured out how to scale using regular RAM?
OK, fair enough, but what are OpenAI doing buying production capacity rather than, say, paying NVIDIA to do it? OpenAI aren’t the ones making the hardware?
Just because Nvidia happily sells people discrete GPU's, DGX systems, etc., doesn't mean they would turn down a company like OpenAI paying them $$$ for just the packaged chips and the technical documentation to build their own PCBs; or, let OpenAI provide their own DRAM supply for production on an existing line.
If you have a potentially multi-billion dollar contract, most businesses will do things outside of their standard product offerings to take in that revenue.
> doesn't mean they would turn down a company like OpenAI paying them $$$ for just the packaged chips and the technical documentation to build their own PCBs
FWIW, this was the standard state of affairs of the GPU market for a long while. nVidia and AMD sold the chips they paid someone to produce to integrators like EVGA, PNY, MSI, ZOTAC, GIGABYTE, etc. Cards sold under the AMD or nVidia name directly were usually partnered with one of these companies to actually build the board, place the RAM, design the cooling, etc. From a big picture perspective, it's a pretty recent thing for nVidia to only really deliver finished boards.
On top of this, OpenAI/Sam Altman have been pretty open about making their own AI chips and what not. This might point to them getting closer to actually delivering on that (pure speculation) and wanting to ensure they have other needed supplies like RAM.
