From the source
OpenAI scaled Kubernetes clusters to 7,500 nodes to support infrastructure for large models like GPT-3, CLIP, and DALL·E, as well as for small-scale iterative research.
From the source
We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.
openai.com
