NextFin News - On December 3, 2025, at the NeurIPS conference in San Diego, Geoffrey Hinton and Jeff Dean joined Radical Ventures managing partner Jordan Jacobs for a fireside conversation recorded in front of a live audience. The discussion, part of Radical Talks, traced the technical and organizational history that turned neural networks from an academic curiosity into technologies running at massive scale.
The participants were Geoffrey Hinton (Nobel laureate and widely known as the “godfather of AI”), Jeff Dean (Google’s chief scientist and co‑lead of Gemini), and Jordan Jacobs (moderator and managing partner at Radical Ventures). The session was held during NeurIPS in San Diego and ran in the early evening of December 3, 2025.
On early insights: compute, parallelism and scaling
Both speakers returned repeatedly to the lesson that larger compute budgets and parallelism were decisive. Hinton recounted early experiments with parallel machines and emphasized missed lessons about scaling, saying that parallel computation experiments in speech should have signalled “if you just made them bigger, they work better.” Dean recalled his undergraduate and early research experiences with parallel algorithms and described early attempts at model‑ and data‑parallel training, noting the informal intuition that “if we could just get 32 times as much compute, we can make these amazing neural networks.”
We should have realized at that point that if you just made them bigger, they work better.— Geoffrey Hinton
AlexNet and the power of modest compute used well
Hinton described the AlexNet origin story: students experimenting on small images, a critical weight‑decay parameter change, and a strategic push to apply convolutional networks to ImageNet. He emphasized that the decisive training for AlexNet was conducted on two GPU boards in a student’s bedroom and recalled the practical support — Google paying for the boards and the student’s parents covering electricity.
It was two GPU boards in Alex's bedroom, which was in his parents' house. So the good news was we paid for the GPU boards but his parents paid for the electricity.— Geoffrey Hinton
The origins and aims of Google Brain
Jeff Dean described how early conversations with Andrew Ng and others led to a dedicated software abstraction for neural nets that supported both model and data parallelism across many machines. Dean recounted scaling runs that trained models far larger than had been done before and an unsupervised experiment on 10 million YouTube frames that used thousands of cores. He summarized the empirical pattern with a practical slogan: bigger model, more data, more compute.
On distillation, capsules, and working through early skepticism
Hinton reviewed several threads of his lab’s research history. He noted that some important ideas were initially rejected — for example, distillation being dismissed in peer review — and that work like capsules illustrated both the value and risk of long, determined efforts: success if the problem is well chosen, little return if not. He recounted that distillation was rejected by NIPS reviewers but later proved useful in practice.
When you distill into the student model it doesn't do any better than the teacher. So what's the point?— anonymous reviewer (recounted by Geoffrey Hinton)
Auction, acquisition and the early Google acquisition of DNN Research
Hinton and Dean recalled the auction for DNN Research at an early NeurIPS that took place in a casino setting. They described the practical decision to be acquired by Google because of the productive environment the Brain team offered and the benefits of joining a large engineering organization.
We decided we're going to be an acquisition. In the end we had an auction... I had sort of decided ahead of time that Google had to win because it was such fun working there in the summer and working in the brain team was just always great.— Geoffrey Hinton
TPUs and the case for custom hardware
Dean explained the cost calculus that led Google to design the TPU: inference at scale on CPUs would have required doubling Google’s data‑center fleet to roll out higher‑quality speech recognition, so the team pursued custom ASICs. He described starting with an inference card (TPUv1), deploying tens of millions in hardware investment, and then iterating toward multi‑chip training machines with high‑speed interconnects. He emphasized that TPUs delivered large performance and energy‑efficiency gains over contemporary CPUs and GPUs.
I sat down and we saw that we were making lots of progress on making much higher quality speech recognition models... the math was kind of scary... That essentially was the impetus for starting our own custom hardware.— Jeff Dean
Transformers, attention and algorithmic multiplicative effects
Both speakers discussed the rise of the transformer architecture and attention mechanisms. Hinton described his initial skepticism for a brain‑like mechanism that stores all state vectors but acknowledged transformers’ practical advantages: parallelism and the ability to attend to past states. Dean placed transformers alongside other algorithmic advances such as sparsely activated models, noting that these improvements multiply with hardware advances to produce orders‑of‑magnitude increases in effective compute.
The transformer paper showed that you could get the same quality results with like 10 to 100 times less compute or much higher quality with the same compute.— Geoffrey Hinton
Internal chatbots, Gemini and the response to ChatGPT
Dean and Hinton described Google’s internal chatbot efforts during COVID and the company’s cautious approach to external release because of factuality and hallucination risks. After ChatGPT’s public launch, Dean said he wrote a short memo arguing for consolidating efforts and pooling compute and people — a move that contributed to forming the Gemini effort.
We had an internal chatbot that was quite heavily used within Google... we wanted to work on factuality issues and hallucination issues more before we turned it into something we exposed external users to.— Jeff Dean
Publishing, competition and selective disclosure
On whether Google regrets publishing the transformer paper, Dean answered no and emphasised the value of publishing for community progress and talent recruitment. He added that for very large, commercially sensitive training recipes the company has become more circumspect in sharing exact details.
Future directions: long contexts, continual learning and new architectures
Looking ahead, both speakers identified several open technical directions. Dean stressed the importance of mechanisms to let models attend to vastly larger corpora — “billions or trillions of tokens” — which requires hardware and architectural innovation. Hinton highlighted the need for more flexible connectivity patterns and continual learning rather than large static training phases. Both raised tradeoffs between safety and online learning: continual learning may be powerful but is harder to verify and control.
How can we attend to, you know, billions or trillions of tokens rather than a million or a few million? That might really change what you can do with these models.— Jeff Dean
Societal impact: creativity, science, healthcare and jobs
Both Hinton and Dean reflected on social consequences. Hinton argued that large models compress vast amounts of knowledge and can uncover remote analogies, which may make them creative and useful across domains. Dean discussed possible transformations in healthcare and education while warning about employment disruptions and the political challenges of fair distribution of productivity gains.
The impact on society is not obvious: a lot of jobs are going to disappear. It's not clear it'll create jobs to replace that. This isn't AI's problem. This is our political system's problem.— Geoffrey Hinton
Anecdotes and human details
The conversation included personal anecdotes: Hinton recalled being a 64‑year‑old intern during a Google visit and the moment he met Dean for the first time over Vietnamese food in Palo Alto. Dean recounted asking his CFO for an early $50 million allocation to TPU deployment in 2013, and the pair reminisced about the early Brain team’s small office, shared desks and growing compute hunger.
References and related material
Episode recording and distribution pages:
- Event listing: Jeff Dean & Geoff Hinton in conversation (San Diego Tech Scene / Luma)
- NeurIPS 2025 conference information (NeurIPS)
- Event page: Jeff Dean & Geoff Hinton in conversation with Jordan Jacobs (San Diego Tech Scene)
- Radical Talks — podcast page (Audible)
- Video recording (YouTube)
Published audio/video of the session was released by Radical Ventures and distributed on major podcast platforms in mid‑December 2025.
Explore more exclusive insights at nextfin.ai.
