Monitoring AI Agents with Azure Application Insights

What Application Insights actually is

Application Insights is the application performance monitoring piece of Azure Monitor. It has supported OpenTelemetry, the vendor-neutral instrumentation standard, for regular web apps and services for years: traces, metrics, logs, an application map, live metrics, failure analysis.

What changed is that Microsoft added an entry point specifically for AI agents. The current Application Insights documentation lists 'AI Agents' as one of the supported instrumentation scenarios alongside web apps, Azure Functions and Kubernetes workloads, and it ships an 'Agents details' view described as a unified page for monitoring agents across Microsoft Foundry, Copilot Studio and third-party agents.

So this is not a new product bolted on the side. It is the same Application Insights backend that already existed, with agent-shaped telemetry and an agent-shaped view added on top of the general OpenTelemetry pipeline.

• Application Insights is Azure Monitor's APM feature, built on OpenTelemetry.
• AI Agents is now a documented, first-class instrumentation scenario, not a separate product.
• The Agents details view covers Foundry agents, Copilot Studio agents and third-party agents in one place.

How you actually wire it up

The setup path depends on where your agent runs. If you built it on Microsoft Foundry, you start from Foundry's own tracing setup and connect an Application Insights resource to the Foundry project, which routes telemetry there automatically.

If you are self-hosting and orchestrating the agent yourself, Microsoft's own path is the Microsoft Agent Framework, which emits OpenTelemetry data to Azure Monitor directly once it is pointed at a connection string. If you built the agent on something else, LangChain, LangGraph or the OpenAI Agents SDK, Microsoft documents an Azure AI OpenTelemetry Tracer specifically for those frameworks, so you do not have to hand-instrument every call.

Whichever path you take, the mechanics are the same: create the Application Insights resource, get its connection string, add the right OpenTelemetry distro or tracer for your language and framework, and give each agent a distinct name so you can tell them apart once telemetry starts arriving.

• Microsoft Foundry agents: connect an Application Insights resource to the Foundry project.
• Self-hosted agents: use the Microsoft Agent Framework, which emits OpenTelemetry to Azure Monitor.
• LangChain, LangGraph or OpenAI Agents SDK: use the Azure AI OpenTelemetry Tracer for that framework.
• Name each agent distinctly so the Agents details view can tell them apart.

What 'local' cannot mean here

One of the real queries behind this topic pairs 'application insights' with 'local', and it is worth answering directly: there is no local or offline mode for Application Insights. It is a cloud service. Telemetry is generated on your machine or server by an OpenTelemetry exporter, then sent to an Application Insights resource in Azure, where the processing, storage and every view described here actually happens.

You can develop and test against a local OpenTelemetry collector, or use something like the .NET Aspire dashboard to see traces on your own machine during development, but that is a different, Microsoft-adjacent tool, not Application Insights itself. If your goal is a fully local, nothing-leaves-the-machine way to watch an agent, Application Insights is the wrong tool regardless of how it is configured.

This matters for security too. The connection string that routes telemetry to your resource is itself a credential. Treat it the way you would any other secret that lets data leave a machine, not as a config value you can leave in a public repository.

• Application Insights requires a live Azure resource; there is no local-only mode.
• A local collector or the Aspire dashboard can preview traces during development, but that is not Application Insights.
• The Application Insights connection string is a credential and should be handled like one.

What the Agents view is built for

The Agents details view is aimed at agents that are deployed and running against real traffic, a customer-facing assistant, an internal workflow agent, something with a lifecycle beyond one person's terminal session. That is also where its evaluation features live: batch evaluations you run against a dataset during development, and continuous evaluations that run against live production traffic to catch quality regressions over time.

If you turn on the EnableSensitiveData flag in the Microsoft Agent Framework, Application Insights will capture full prompts, assistant responses, system prompts and tool usage, and you can read them back in the Search and Transaction Details views. That is powerful for debugging, and it is also a privacy decision: you are choosing to store conversation content in Azure Monitor Logs, so it needs the same access controls and retention thinking you would apply to any other store of user data.

None of this is scoped to a single developer watching their own coding session. It is built for a team operating an agent as a service, which is a different job than noticing that your own terminal has gone quiet.

• Batch evaluations run on demand against a dataset; continuous evaluations run against live traffic.
• EnableSensitiveData stores full prompts and responses in Azure Monitor Logs, which changes your privacy and retention obligations.
• The feature set targets a deployed, multi-user agent, not a single local coding session.

If what you actually want is a different job

The other real query behind this topic is 'local', and the honest answer is that watching your own coding agent while it works on your own Mac is a different problem than tracing a deployed agent fleet in Azure, and it calls for a different tool.

Forkbench is a desktop app that runs coding agents in real terminals on a Mac. Each tab has a pulse, a liveness signal driven by CPU use and output activity, with a flag when an agent looks stuck and is waiting on you. It is not a token meter and it is not telemetry you query later. It is a glance at whether the thing in front of you is still working, which is the job most developers actually have day to day.

That is a narrower and a different tool than Application Insights, on purpose. If you are running a production agent at scale, you want the Azure path. If you are the one person watching three terminals, a live pulse answers the question faster than a cloud dashboard built for someone else's job.

• Forkbench's pulse is a per-tab, CPU and output driven liveness signal, not a token meter.
• It flags a blocked agent so you know it needs you, instead of requiring you to read a trace after the fact.
• Use Application Insights for a deployed agent fleet; use a live pulse for your own terminal session.

A setup checklist

Work through these in order if you are instrumenting a real agent for the first time. Each step is small, and skipping one usually shows up as missing or unlabeled telemetry later rather than an outright failure.

The goal is telemetry you can actually use for an incident, not just a resource that technically has data in it.

• Create an Application Insights resource and copy its connection string.
• Pick the instrumentation path for how your agent is hosted: Foundry, Microsoft Agent Framework, or a third-party framework's Azure tracer.
• Give every agent a distinct name before telemetry starts flowing.
• Decide deliberately whether to enable sensitive data capture, and apply access controls to the resource if you do.
• Set up a continuous evaluation only once you have a baseline from batch evaluations to compare against.
• Treat the connection string as a secret, not a configuration value.