Training an AI Scientist with Feedback from Reality, w- Liam Fedus & Ekin Dogus Cubuk (from a16z)
Timestamps are as accurate as they can be but may be slightly off. We encourage you to listen to the full context.
Podcast Summary
In this episode of the a16z podcast, hosted by Anjane Mitte, listeners meet Liam Fetis (former VP of post-training research and co-creator of ChatGPT at OpenAI) and Ecken Dojeschubuk (former head of material science and chemistry research at Google DeepMind), who have co-founded Periodic Labs with a groundbreaking $300 million seed investment. (00:55) The company aims to build frontier AI research systems that can autonomously explore and understand the physical world, with the ambitious North Star goal of discovering a room-temperature superconductor. (16:23)
- Main Theme: Building an "AI physicist" that connects AI-generated hypotheses directly to real-world experiments using physical reality as the reinforcement learning signal, moving beyond digital-only AI systems to accelerate scientific progress through autonomous synthesis and characterization capabilities.
Speakers
Liam Fetis
Former VP of post-training research at OpenAI and co-creator of ChatGPT, Liam brings deep expertise in reinforcement learning from human feedback (RLHF) and language model development. He was instrumental in transforming raw autocompletion models into useful assistants through supervised learning and reinforcement learning against human preference-based reward functions.
Ecken Dojeschubuk
Former head of material science and chemistry research at Google DeepMind, Ecken has extensive experience in quantum mechanical simulations and physics research. He specializes in solid-state physics, material science, and chemistry at the quantum mechanical energy scale, bringing crucial physical science expertise to bridge AI capabilities with real-world experimentation.
Anjane Mitte
General partner at Andreessen Horowitz (a16z) who led the $300 million seed investment in Periodic Labs. He focuses on exploring the human and organizational dimensions of building AI for science companies, with particular interest in how frontier AI can accelerate scientific discovery and physical R&D.
Key Takeaways
Physical Reality Must Be the Reward Function
The fundamental limitation of current AI systems is that they optimize against digital reward functions like math graders and code graders, which don't capture the complexity of physical reality. (08:48) Liam explains that ChatGPT was created using reinforcement learning against human preferences, but "ultimately, science is driven against experiment in the real world." (11:24) Periodic Labs is pioneering the use of physical experiments as the actual reward function for AI systems, creating what they call "nature as our RL environment." This approach ensures that when simulators have deficiencies, the system always corrects against ground truth experimental results rather than potentially flawed theoretical models.
Iterative Scientific Method Is Essential for Discovery
Even the most advanced AI models cannot discover new science without iteration against real-world experiments. (14:14) As Ecken points out, "even the smartest humans try many times before they discovered the things they discovered," and the same principle applies to AI systems. The key insight is that LLMs need to learn "the method of scientific inquiry" - conducting simulations, theoretical calculations, experiments, getting results that are likely incorrect initially, and then iterating. (14:46) This iterative process cannot be replicated through text-only training, requiring actual physical experimentation in the loop.
Existing Scientific Data Is Insufficient for Training
Current scientific literature lacks the high-quality, comprehensive data needed to train effective AI models for physics and chemistry. (15:58) The literature spans "many orders of magnitude" for reported properties, contains predominantly positive results (creating bias), and lacks valuable negative results that are crucial learning signals. (16:06) Additionally, formation enthalpy labels in synthesis literature have such high noise levels that machine learning models trained on them aren't predictive enough for practical use. (27:52) This data scarcity problem means that the experimental data Periodic Labs wants to use "actually doesn't exist" in sufficient quality and quantity.
Cross-Disciplinary Team Building Requires No-Stupid-Questions Culture
Successfully building AI for science requires bridging the gap between ML researchers and physical scientists through active cultural integration. (40:41) Periodic Labs implements weekly teaching sessions where "LLM researchers teach how the RL loops work, how the data cleaning works, and then the physicists and chemists are teaching about different aspects of the science." (42:05) The company emphasizes that advanced degrees aren't required because "the amount that even our best physicist doesn't know about physics is much bigger than the amount that they know about physics," making the learning curve similar for all team members regardless of background. (44:02)
Commercial Applications Enable Maximum Scientific Impact
Building a successful commercial entity is essential for maximally accelerating scientific progress, not just an ancillary goal. (39:15) The company plans to serve as "an intelligence layer" for advanced manufacturing companies across space, defense, and semiconductors, helping them accelerate R&D workflows and reduce iteration times. (39:27) This commercial strategy allows Periodic Labs to scale their impact while generating revenue to support their fundamental research goals. The approach recognizes that "technology and capital are intertwined" and that widespread adoption of AI-powered scientific tools requires proven commercial value alongside scientific breakthroughs.
Statistics & Facts
- Periodic Labs raised a $300 million seed investment led by Andreessen Horowitz, representing one of the largest seed rounds in AI history. (00:28) This funding demonstrates unprecedented investor confidence in applying frontier AI to physical science research.
- The current best ambient pressure superconductor operates at 135 Kelvin, representing the benchmark that Periodic Labs aims to exceed. (16:43) Achieving even a 200 Kelvin superconductor would constitute a revolutionary breakthrough in understanding quantum effects at high temperatures.
- The team at Periodic Labs has grown to approximately 30 people, carefully balanced between ML researchers and physical scientists. (46:03) This represents a unique concentration of interdisciplinary expertise focused on AI-driven scientific discovery.
