Last week GV hosted our first AI Builders NYC event, bringing together our portfolio founders and the community of builders in New York. I invited Zach Gleicher, the longest standing PM at DeepMind, to join me to share his insights on LLM development, and what he thinks will be the next big opportunities for AI.
The following is a ‘Vibe-Scribed’ version of our talk (meaning, I fed the transcript to Gemini 2.5 Advanced and asked for the key takeaways from Zach).
Drawing on his experience from the pre-Transformer era to today, Zach started with a peek behind the curtain of how LLMs are made, and shared his perspective on talent, measurement and the biggest challenges and opportunities for LLMs.
1. Educating Artificial Intelligence
Zach likened a foundation model’s development to a person’s education and career. First is pre-training, the “undergraduate degree,” where the model gains foundational knowledge by learning from vast datasets. Next is post-training, the “first job,” where its style and personality are fine-tuned for specific tasks using human feedback. Finally, reasoning is like “showing your work,” where the model uses techniques like Chain of Thought to break down complex problems at runtime, leading to more accurate results. The biggest leaps in capability often come from pre-training, but all three areas are critical for building useful AI.
2. Building Frontier Models is a High-Stakes Game of Talent and Capital
Training state-of-the-art models costs hundreds of millions of dollars, making every training run a “billion-dollar question.” This intense, high-cost environment has sparked a “war for talent” for what Zach calls "100x engineers.” And he believes they might be worth the crazy salaries, as they possess rare intuition for what will work at scale.
3. Standard Benchmarks Don’t Tell the Whole Story
While industry benchmarks are important, it’s possible to “overfit” a model to perform well on a test without actually improving the user experience. Zach emphasized that internal, private testing is crucial. Real-world feedback is the only way to understand how a model truly feels to a user and to ensure that improving one skill (like coding) doesn’t accidentally degrade another (like creative writing).
4. The Next Wave is Agentic AI in Fields like Gaming and Finance
While coding was an early win, AI is now finding strong product-market fit in many new areas. Zach highlights significant pull from gaming (for creating dynamic NPCs), financial services (for complex analysis), and research (for synthesizing information). The key trend across these fields is a move beyond simple “co-pilots” toward more autonomous, agentic systems that users can delegate complex, multi-step tasks to.
5. Some of the Biggest Hurdles are Still Hallucination and Computer Control
Two major challenges stand in the way of the next leap forward. The first is hallucination— the model’s tendency to invent facts. This involves a difficult trade-off between ensuring factuality and preserving creativity. The second, and perhaps bigger, hurdle is computer control: enabling AI to seamlessly and reliably operate our user interfaces and applications. Solving this will be the key to unlocking the truly powerful, delegable AI agents of the future.