Shipping your AI agent is actually the easiest part. The real question — the one almost nobody is talking about — is how to monitor AI agents in production once hundreds or thousands of real users start interacting with them every day. Unit tests won't save you. Regex filters won't save you. The only thing that saves you is closing the feedback loop fast, and building a system that watches your system. Here's exactly how to do it.
How Do You Monitor AI Agents in Production?
Most teams celebrate the launch and move on. But once your agent is live, you lose direct feel for what it's doing. You can't pretest every possible conversation. You can't enumerate every edge case. Customers will always do something you didn't anticipate — and the agent will respond in ways you never expected.
02:15
The moment the speaker defines 'the missing layer' — what happens after you ship an AI agent
Watch at 02:15 →
The foundation of production monitoring for AI agents is your logs and traces. Every conversation is a trajectory, and those trajectories are your source of truth. The challenge is that there are too many of them to read manually, and the failures are often subtle — not red-alert crashes, but quiet wrong turns that still leave users unhappy.
The solution is to build a meta-system: an agent (or set of agents) that watches your production agent, diagnoses problems, and closes the loop automatically.
Why Do AI Agents Fail Silently After Launch?
This is the part that keeps experienced builders up at night. AI agents fail silently for a few key reasons that don't exist in traditional software:
- Non-determinism: The same input can produce a different path every time. A slight change in phrasing can cause a completely different trajectory.
- Hidden recovery: An agent might struggle mid-task, get lucky, find a workaround, and complete the job — but your dashboard shows green. That recovered failure is an early warning sign you're almost certainly missing.
- Success ≠ helpfulness: An agent can finish a task and mark it complete without it actually being useful to the user. As Anthropic has noted, agents sometimes mark tasks as done without verifying they actually worked.
- Endless tool surface: Long-running agents use dozens or hundreds of tool calls — terminals, APIs, sub-agents, third-party services. Failures can hide anywhere in that chain.
A real-world example: a travel agent builds a full itinerary, runs successfully, and returns a result. But it used the wrong pricing service and made calculation errors throughout. Technically a success. Actually a failure. Your unit tests will never catch this — only production will.
07:40
Diagram of the end-to-end log monitoring loop: traces → agent diagnosis → PR → review agent → human
Watch at 07:40 →
How to Build a Tight Feedback Loop for AI Agents
The goal is to close the loop as fast as possible after launch. The tighter your feedback loop, the faster you can improve the product every single day. Here's the two-layer system that works in practice:
Layer 1: The Fast Loop (Log Monitoring Agent)
This agent runs every hour — or even every 15 minutes — and does the following:
- Reads recent logs and conversation trajectories
- Has access to the codebase
- Diagnoses whether something is a real bug or just noise
- Determines if an issue is a symptom or a root cause
- Sends a Slack alert for critical issues
- Opens a pull request with a fix for diagnosable problems
The PR itself should include a clear description, metadata, diagrams (Mermaid or ASCII), and artifacts that let a human — or another agent — understand the problem at a glance. The goal is to be able to understand a PR in under two minutes.
Layer 2: The Zoom-Out Loop (Session Analyzer)
The fast loop is great for catching specific bugs. But it doesn't give you a high-level sense of system health. That's what the session analyzer does. It runs less frequently — once or twice a week — and does the following:
14:20
Live demo of the session health dashboard showing score distribution, AI insights, and per-session breakdowns
Watch at 14:20 →
- Scores every conversation in the analysis window
- Detects patterns across hundreds or thousands of sessions
- Identifies logical failures, excessive tool calls, over-summarization
- Surfaces AI-generated insights with root causes and affected session counts
- Produces a health dashboard with success rates, sentiment analysis, and entity breakdowns
This is the tool that gives you back the feel for your own system — the intuition that disappears once you're past a few dozen users. It's not about fixing one bug. It's about understanding whether your product is getting better or worse over time.
How to Automatically Detect Bugs in Agent Logs
The most natural tool to analyze agent logs is — another agent. LLMs are excellent at reasoning over structured log data, differentiating real bugs from noise, and deep-diving into a trajectory to understand what actually went wrong.
Here's what the log monitoring agent needs to work well:
- Access to full conversation traces and trajectories
- Access to the production codebase
- Enough context to distinguish a symptom from a root cause
- A clear output format — PR or Slack alert — so the result is immediately actionable
The key insight is that operating an agent is itself an agent problem. You're not just writing scripts to grep through logs. You're building a reasoning system that understands your product deeply enough to diagnose it.
Should an AI Agent Review Its Own Pull Requests?
No — and this is a critical design decision. The same agent that diagnoses a problem and opens a PR is biased toward its own solution. It has the context of the bug baked into its thinking, which makes it a poor reviewer of its own fix.
The solution is a separate review agent with a fresh context window. This agent:
- Reads the PR without knowledge of the original bug investigation
- Critiques the solution from a different angle
- Scores the PR for quality and risk
- Runs focused tests against the change
- Either approves, requests changes, or closes the PR directly
In practice, the PR agent and review agent together send roughly 10x more PRs per day than the human team does. That's a good problem to have — but it means you need a clean system for handling the volume, or you become the bottleneck.
How to Analyze Thousands of Agent Conversations
Before agents, it was essentially impossible to meaningfully summarize what was happening across hundreds of daily user conversations. Now you can build a session analyzer that scores every conversation, surfaces the top failure patterns, and gives you a weekly health report — all automatically.
A well-built session health dashboard should show you:
- Overall health score and session count
- Success rate and score distribution
- AI-generated insights with root causes, affected session counts, and fix recommendations
- Sentiment analysis across conversations
- Tool call analytics — success rates, rejection rates, reasons for failure
- Per-session breakdowns: message count, tool calls, summaries, flags for human attention
Building this yourself is worth the effort, because you know exactly what you're looking for in your specific system. Off-the-shelf observability tools are useful, but they don't know your product the way you do.
Do You Still Need a Human in the Loop?
For now — probably yes, but less than you think. The PR agent and review agent together produce well-documented, artifact-rich pull requests that a human can evaluate in a few minutes. That's a reasonable trade-off while you're still calibrating the system's reliability.
The practical advice: close the loop first, then remove yourself. Get to the point where you are the bottleneck — where the agents are generating more fixes than you can review — and then automate the human step. Don't start by removing the human; start by making the human's job trivially easy, and then make it optional.
The teams shipping reliable production agents aren't just building better models or better prompts. They're building the meta-harness around those agents — the system that watches, diagnoses, and improves itself. That's the missing layer. And it's where the real work begins.
