Thinking Machines Lab, the artificial intelligence research startup founded by former OpenAI CTO Mira Murati, has released its initial research focusing on achieving deterministic responses from AI models. The findings, published in a blog post titled "Defeating Nondeterminism in LLM Inference" on Wednesday, represent the first public insight into the lab's work since its establishment.
The research, authored by Thinking Machines Lab researcher Horace He, addresses the prevalent challenge of non-determinism in large language models (LLMs), where repeated queries often yield varied answers. He's analysis suggests that the root cause of this randomness lies in the orchestration of GPU kernels during inference processing, which are the small programs executed within Nvidia's computer chips. By implementing precise control over this orchestration layer, the research posits that AI model responses can be made more consistent.
The implications of reproducible AI responses extend to industrial and scientific applications. Horace He indicates that achieving determinism could lead to more reliable outputs for enterprises and scientists, where consistency is critical. Furthermore, the research suggests that consistent model responses could enhance reinforcement learning (RL) training by reducing data "noise," thereby making the RL process "smoother." Thinking Machines Lab has previously informed investors of its intent to leverage RL for customizing AI models for businesses, according to reports from The Information.
Thinking Machines Lab, which secured $2 billion in seed funding and is valued at $12 billion, is reportedly developing its first product for researchers and startups building custom models, with an unveiling anticipated in the coming months, Murati stated in July. The lab has also committed to frequent publication of its research, code, and other findings, aiming to benefit the public and foster its research culture. This initial blog post is part of the company's new "Connectionism" blog series, aligning with this public disclosure strategy.
