Why it Matters
This talk represents a significant shift in the 'agentic' hype cycle—a move from architectural complexity (many agents) t...
Lessons Learned Building Rippling AI | Interrupt 26
The Signal
Rippling recently centralized its AI infrastructure around a "flat agent" architecture, abandoning a multi-agent setup that struggled with context handoffs and coordination. By treating tool selection as a primary failure point, they moved toward generic, SQL-based data retrieval and a rigorous eval-first development process to manage the inherent stochasticity of LLMs.
The Case
Architecture and Data
- Rippling AI operates on an "employee graph," a centralized data model where employee information is synchronized across all internal systems.
- The team abandoned a complex multi-agent system in favor of a single, flat orchestration agent, finding that cross-agent handoffs, interrupts, and context sharing created unmanageable systemic complexity.
- To avoid hallucinations and performance bottlenecks, the agent does not ingest raw data into the prompt; instead, it uses the provided schema to generate SQL queries, which are executed against a cache to support iterative analysis.
- The engineers replaced a large catalog of bespoke tools with a limited set of generic, highly composable tools, applying a Unix-like philosophy to minimize tool selection errors.
Release Discipline
- Development is governed by "eval-driven development," where intuition is replaced by smoke evals on every commit and twice-daily health evals in pre-production.
- Eval design follows a rigorous tradeoff triangle—cost, uncertainty, and lag—where teams can optimize for only two of the three variables at any given time.
- Because LLM behavior is inherently stochastic, the team uses Wilson confidence intervals to quantify uncertainty, noting that even a perfect 3/3 pass rate holds a significant lower-bound failure risk in small samples.
The 1 Minute Signal Take
Engineering agent-based systems currently requires trading away architectural elegance for brute-force reliability through flat hierarchies, SQL-backed retrieval, and statistical validation. As long as model behavior remains stochastic, the most robust path is to remove abstraction layers between the model and its data while increasing the frequency of automated testing.
Pro Analysis
Full analysis always available on Pro.
Time saved: