OpenAI is facing challenges with managing the allocation of graphics processing units (GPUs) internally. According to Greg Brockman, the President of the artificial intelligence startup, this is an exercise in “pain and suffering.” Speaking on the “Matthew Berman” podcast, Brockman said that deciding which teams will receive the crucial computing resource is an emotional and exhausting process, as the difficulty lies in choosing among multiple promising initiatives. “It’s so hard because you see all these amazing things, and someone comes and pitches another amazing thing, and you’re like, yes, that is amazing,” he noted.
How OpenAI divides its computing power internally
OpenAI divides its computing power between two primary areas: research and applied products. The overall split between the research and applied teams is determined by senior leadership, specifically CEO Sam Altman and the applications CEO, Fidji Simo. Decisions for allocations within the research side are made by the company’s chief scientist and research head. At the operational level, a small internal team focuses on shuffling GPU assignments, including Kevin Park, who handles redistributing hardware as projects wind down.“You go to him and you’re just like, ‘OK, like we need this many more GPUs for this project that just came up. And he’s like, ‘All right, there’s like these five projects that are sort of winding down,’” Brockman said. The internal GPU shuffle reflects the scarcity that OpenAI has warned about for months. Brockman said computing drives the productivity of entire teams, and the stakes are considerable.“People really care. The energy and emotion around, ‘Do I get my compute or not?’ is something you cannot understate,” he added.OpenAI has been vocal about its demand for computing power. “Every time we get more GPUs, they immediately get used,” OpenAI’s chief product officer, Kevin Weil, said on an episode of the “Moonshot” podcast published in August.Weil said the need for computing is straightforward: “The more GPUs we get, the more AI we’ll all use.” He noted that adding bandwidth enabled the expansion of video.Altman said last month that OpenAI is launching “new compute-intensive offerings.” Because of the costs involved, some features will initially be limited to Pro subscribers, while certain new products will have extra fees, he added.Other tech companies have also been direct about their appetite for GPUs. Mark Zuckerberg said on an episode of the “Access” podcast published last month that Meta is making “compute per researcher” a competitive advantage. He said the company is outspending rivals on GPUs and the custom infrastructure needed to power them.
About the Author
