Your AI agent just produced the wrong answer, and you have no idea why. Was it a bad prompt, a hallucinated tool call, or a fundamental misunderstanding of the context? Until now, debugging that black box of agentic reasoning has been a frustrating exercise in trial and error. You tweak the prompt, run it again, burn another dollar on API calls, and hope for a different result. Simon Willison’s LLM 0.32 release changes this by making the agent’s thought process visible, turning a black box into a glass box you can actually inspect.
The Agent Debugging Wall
Building reliable agents is hard because their internal logic is opaque. When they fail, they fail silently. Imagine you’ve tasked an agent with summarizing daily tech news. It’s supposed to use a search tool to find articles, read them, and synthesize a brief. But today, it returns an empty summary.
What happened? The agent could have hit a paywalled site. The search tool’s API key might have expired. Maybe the model formulated a bad search query that returned zero results and then gave up. Without a trace, all you see is the failed final output. You’re left guessing. This is the debugging wall: you spend more time diagnosing the agent’s invisible failures than building its capabilities. Each guess costs you API credits and development time.
This is where a reasoning trace becomes essential. Instead of just the final, wrong answer, you need a step-by-step log: the initial thought, the exact tool it decided to call, the parameters it used, the raw output from that tool, and its subsequent reasoning. That’s the difference between a broken product and a debuggable system.
What LLM 0.32 Actually Ships
The recent LLM 0.32 release, along with the llm-anthropic 0.26 plugin, introduces two features that directly address this: visible reasoning traces and server-side tools.
Visible Reasoning Traces
This is the core of the upgrade. By adding a single flag to your command, you can instruct the model to output its entire thought process. The model exposes its internal monologue, including which tools it considers and why.
Here’s the difference. Before, a query would just return the answer:
llm -m claude-3-opus "Summarize the latest post on Simon Willison's blog."
# Returns only the final summary...Now, with the --traces flag, you get the full story:
llm -m claude-3-opus --traces "Summarize the latest post on Simon Willison's blog."
# Returns a detailed trace with <thinking> and <tool_code> blocks
# followed by the final summary.The output now includes structured XML-like tags that show the model’s reasoning steps. You see it decide to use a tool, formulate the code for that tool, and process the result. No more guessing.
Server-Side Tools
Tool use isn’t new to llm. Version 0.26 introduced client-side tools, where the model would generate code (like Python) and ask your local machine to execute it. This was a good first step, but required a local Python environment and dependencies.
Server-side tools are different. The model asks the API provider, like Anthropic, to execute the tool on their end. For something like web search, this is a huge simplification. You don’t need to manage local browser automation or a search API client. You just tell the model the WebSearch tool is available, and Anthropic’s infrastructure handles the execution. This lowers the barrier to building simple, tool-using agents.
Tutorial: Building a Traceable Web-Searching Agent
Let’s build an agent that can answer questions about recent events using Anthropic’s server-side web search tool and see exactly how it works using traces. This entire workflow runs from your command line.
Step 1: Install the Tools
You need llm version 0.32 or newer and the llm-anthropic plugin at version 0.26 or newer. If you have them installed already, run the upgrade command.
pip install -U "llm>=0.32" "llm-anthropic>=0.26"Step 2: Set Your API Key
The llm tool needs your Anthropic API key. You only need to do this once.
llm keys set anthropicPaste your key when prompted. The tool stores it securely in a local file.
Step 3: Run a Query with a Tool and Traces
Now, let’s ask a question that requires external knowledge. We’ll use a Claude 3 model, enable the server-side WebSearch tool, and request reasoning traces.
This single command combines all the new capabilities:
llm -m claude-3-opus-20240229 \
--tool WebSearch \
--traces "According to Simon Willison's blog, what are the key features of LLM 0.32?"Step 4: Analyze the Output
The output will be verbose, which is exactly what we want for debugging. It will look something like this:
<thinking>
The user is asking about the key features of LLM 0.32, according to Simon Willison's blog.
I need to find the relevant blog post on Simon Willison's blog and then extract the key features of LLM 0.32.
I will use the WebSearch tool to search for "Simon Willison's blog LLM 0.32".
</thinking>
<tool_code>
print(WebSearch("Simon Willison's blog LLM 0.32 features"))
</tool_code>This first block is pure gold. You can see the model’s plan. It correctly identifies the need to search Simon’s blog and formulates a precise query. If it had generated a bad query, like llm features, you would see it right here.
Next, you’ll see the tool’s output returned from the API:
<tool_output>
[
{"title": "LLM 0.32: Traces and server-side tools - Simon Willison's ...", "url": "https://simonw.fyi/blog/2024/May/20/llm-0.32/", "content": "LLM 0.32 is out, with two big new features: traces for tool-using prompts, and support for server-side tools..."},
... more search results ...
]
</tool_output>This block shows the raw data the model is working with. If the search tool returned an error or irrelevant results, this is where you’d spot it. You can check the URLs and content snippets to verify the agent is on the right track.
Finally, the model synthesizes an answer based on the tool’s output:
<thinking>
The search results confirm that Simon Willison wrote a blog post about LLM 0.32. The top result provides the key features directly: traces for tool-using prompts and support for server-side tools. I can now answer the user's question.
</thinking>
Based on Simon Willison's blog, the two key features of LLM 0.32 are:
1. **Traces for tool-using prompts:** This feature provides a detailed log of the model's reasoning process when it uses tools, making it much easier to debug.
2. **Support for server-side tools:** This allows models to use tools executed by the API provider (like Anthropic's WebSearch) rather than requiring client-side execution.The entire chain of reasoning is laid bare. You went from a prompt to a final answer with full visibility into every intermediate step. The agent is no longer a black box.
What Verifiable Agency Means in Practice
This isn’t just a neat feature; it changes the calculus for building simple agents.
-
Real Reliability. When an agent fails, traces provide immediate root-cause analysis. Instead of just seeing a bad output and guessing at the prompt, you can pinpoint the exact failure point. Was it a malformed tool call? An unexpected API error from the tool? A logical leap where the model misinterpreted the tool’s output? The trace tells you where to fix the problem, which is a massive productivity gain.
-
Radical Simplicity. For a huge class of problems, you don’t need a complex orchestration framework. If your goal is to have an LLM answer a question using one or two tools, frameworks like LangChain or LlamaIndex are often overkill. They introduce their own layers of abstraction: chains, agents, parsers, and state management. With
llm, the agent definition is a single command. The complexity is gone. You can ship a useful, debuggable agent in five minutes. -
Meaningful Openness. This approach keeps you out of proprietary ecosystems. You’re not locked into a specific vendor’s SDK or an opaque cloud service that runs your agents for you.
llmis an open-source tool that works with models from OpenAI, Anthropic, Google, and local models. The traces are part of an open standard from the model providers. You own the workflow, and you can see how it works. This stands in sharp contrast to black-box commercial offerings that hide their internal logic.
Related
- Why Your AI Agent Is Bleeding Money in Production (And How to Fix It)
- Stopping Rogue Agents: Observability and Guardrails for Production AI
For developers building lightweight automations, this is a clear win. It’s a practical, low-overhead way to build more reliable AI-powered tools. The next time your agent breaks, you won’t have to guess why. You’ll just check the trace.