AI Models Are Grown Not Built

Video thumbnail: AI Models Are Grown Not Built
Oct 22, 20261m 6s video lengthLenny's Podcast

The Signal

Modern language models function through an architecture that turns words into numbers, performing complex, repeated internal computations before outputting a result. The central tension lies in the gap between successfully building these systems and actually understanding how they work. The speaker argues that the learned internal math is grown, opaque, and inherently biological in its complexity rather than a product of clean engineering design.

The Case

The Mechanism

  • Models operate by converting inputs into numerical representations, which then undergo repeated intermediate computations before being converted back into words.0:22
  • The foundational architecture provides the scaffold, but the actual internal formulas are grown through training data, making them opaque to developers.

The Research Agenda

  • The speaker’s team is attempting to move beyond analyzing final outputs by decoding intermediate states and learned formulas while the model is actively processing information.0:45
  • The field is characterized as being closer to biology than traditional engineering, suggesting the need to observe emergent, complex systems rather than rely on predictable design principles.

The Conflict

  • The speaker asserts that critics who simplify models as "just X or just Y" lack the experience of working directly within the models' internal logic.0:00
  • Whether these internal formulas are truly inscrutable or simply awaiting better interpretive methods remains an open question, as the speaker identifies them as currently unknown.

The 1 Minute Signal Take

The core takeaway is that our current proficiency in building language models has significantly outpaced our mechanistic understanding of them. If you assume these systems operate on straightforward, human-legible logic, you are likely missing the reality that their internal computations are essentially unmapped, emergent processes.

Pro Analysis

Why It Matters

The shift from 'engineering' to 'growing' models highlights a crisis of transparency. As systems become more powerful, our inability to interpret their internal logic creates a fundamental misalignment between human oversight and model capability. This transition from architecture to emergent behavior means we are deploying systems we do not fully comprehend, increasing the risk of unpredictable performance.

Strategic Implications

Organizations relying on AI for critical decision-making must assume these systems operate on non-intuitive logic. Strategies that assume total explainability are flawed; therefore, robustness testing should focus on empirical, edge-case behavioral analysis rather than theoretical verification of the model's 'understanding.'

Evidence & Hype Audit

This content is anecdotal and high-level. It offers a useful heuristic (the biology analogy) but lacks rigorous evidence or specific technical examples. It should be treated as a philosophical perspective from a practitioner rather than a documented research study.

Counterarguments

Critics might argue that the biological analogy is an excuse for poor engineering. If we cannot interpret these models, it suggests a failure in our current training architectures—not an inevitability of AI development. It is possible that we have simply not yet developed the correct mathematical frameworks for mapping these systems, rather than the internal formulas being inherently 'unknowable.'

Who Should Care

  • AI Researchers: To pivot toward mechanical interpretability.
  • Policy Makers: To acknowledge that current AI deployment involves significant, unquantifiable internal uncertainty.
  • Risk Officers: To update threat models to include behavior derived from opaque, non-linear internal states.

What to Do Next

  • Invest in mechanistic interpretability research that targets intermediate neuron activations.
  • Implement 'red-teaming' that focuses on process-traceability rather than output-validation alone.
  • Distinguish between model capability and model transparency in project scoping.
  • Advocate for standards that require documentation of the training dynamics rather than just the final model weights.

Share this

Written by: 1 Minute Signal Editorial Team