Foundations of the Agent Loop

• An agent operates as a recursive cycle: the model emits a structured tool request, external application code executes the function, and the result is fed back into the conversation history, which serves as the agent's entire memory.2:16
• The loop terminates only when the model outputs plain text instead of a tool call; if the system requires constant user intervention between steps, it is not functioning as an agent.2:46
• Constrained decoding is the standard for reliable structured output, as it mathematically masks invalid next tokens during generation to prevent parse errors that break the loop.6:52

Managing Context and Complexity

• Context degradation is a mechanical failure mode where models prioritize the beginning and end of their context while losing track of middle content, especially when overwhelmed by 50,000+ tokens of tool logs.7:42
• Tool sprawl hurts reasoning; internal benchmarks suggest agents perform significantly better with a limited toolset, such as 19 tools versus 46, because excessive tool definitions consume reasoning budget.8:41
• Four engineering strategies define reliable agents: 'right' (storing fundamentals in persistent notes), 'select' (retrieving only relevant info), 'compress' (summarizing old history), and 'isolate' (delegating complex subtasks to separate agents).9:13

Measuring Reliability

• Individual step reliability compounds mathematically, meaning even a 95% success rate per step degrades to roughly 33% success over a 20-step task, making objective evaluation mandatory.10:48
• Effective evaluation requires running 20 to 50 real, failed tasks multiple times to account for non-determinism, using objective code-based graders where possible and calibrated LLM judges for qualitative checks.11:56
• Regression testing must track token counts, steps taken, and financial costs per task, as a prompt change that maintains success rates while doubling costs is an operational regression.13:02