OpenAI and Broadcom unveiled Jalapeño, OpenAI’s first custom AI accelerator, as Sam Altman’s chatbot maker pushes to control more of its data center chip stack and drive greater efficiency to lower inference costs.
The new chip is described as an “Intelligence Processor” and was designed from scratch for large language model inference, the compute-intensive process of serving AI products such as ChatGPT, Codex, and the OpenAI API.
“While OpenAI is still measuring final performance, early testing shows that Jalapeño will deliver performance per watt substantially better than current state-of-the-art,” OpenAI wrote in a press release.
Both companies moved from initial design to manufacturing tape-out in just nine months, supported by OpenAI’s proprietary models and Broadcom’s silicon expertise.
OpenAI said the chip’s architecture is designed to reduce data movement and better balance compute, memory, and networking resources, allowing workloads to run closer to peak performance. The company said the new chips are already powering machine learning workloads in a lab.
“Democratizing AI means making advanced models available, dependable, and affordable enough for more people to use every day,” OpenAI continued.
It is clear that OpenAI is forging ahead with new in-house chips to lower costs from expensive Nvidia GPUs while simultaneously expanding compute capacity.
Another key factor is control, as Altman wants greater command over OpenAI’s chip stack ahead of its push into physical AI. Improved cost efficiency protects margins and could drive profitability down the road. Internal projections estimate the profitability window opens in 2029-30.
Meanwhile, OpenAI has already struck deals involving Amazon’s Trainium chips, AMD hardware, and Cerebras systems as it diversifies beyond Nvidia.
Related:
Broadcom shares rose 1.5% after the news, while Nvidia shares remained flat.
How long until Altman finds a contract supplier to produce OpenAI-designed memory chips?