Day-5

Part-1

Now, let's discuss how the AI agents actually know how to analyze these traces. We have a directory filled with guide documents that the agents reference. These include guides for commands and methodology documents for best practices. We also have specific workflow guides for Deep Research and Case Handler. The agents are smart enough to identify the workflow type automatically. They do this by looking at the root node name of the trace. Once they know the type, they pick the right document to use. This ensures that the analysis is always relevant to the specific task. It allows the agents to give us specific, actionable feedback. This context-aware analysis is what makes the system so effective.

Part-2

Finally, I want to walk you through a typical usage scenario. First, you download the trace files using the command line tool. Then, you simply provide the file to the agent with a prompt. You might ask it to evaluate the relevance of the final answer. The agent reads the guides and runs the analysis commands for you. It generates a report that breaks down findings and suggests improvements. For example, it can tell you if the search strategy was effective. It even provides scores for different dimensions to help you prioritize. This entire process runs locally or via GitHub Actions for convenience. It significantly reduces the time we spend on manual trace reviews.