Microsoft Foundry Agent Architecture

The trap is seeing Foundry as an API for models. In reality it is a layered platform in which governance, development isolation, and connected services each have their own boundary. Anyone who does not separate these layers mixes production and experiment in one place.

Foundry uses a layered architecture: a top-level resource for governance, projects for development isolation, and connected Azure services for storage, search, and secrets, with distinct governance boundaries for Storage, Key Vault, and Azure AI Search. The Foundry Agent Service supports prompt agents, hosted agents, tools, observability, identity, RBAC, virtual network isolation, publishing, and security controls. The RBAC roles were recently renamed, which underlines that this is a fast-moving platform.

Think in three layers. The resource is the governance boundary: policy, roles, and network belong here. Projects isolate teams and workloads. Connected services (Storage for data, Key Vault for secrets, AI Search for RAG) each have their own access model. The Agent Service adds the runtime: tools, orchestration, and observability.

• Use projects to separate production, test, and experiment instead of putting everything in one resource.
• Put network isolation and private endpoints on the resource, not per project again.
• Treat connected services as their own governance boundaries: least privilege on Storage, Key Vault, and Search separately.
• Build observability in from the start; agents without tracing cannot be debugged or accounted for.

The distinct governance boundaries are an opportunity and a risk. An opportunity, because you can set least privilege tightly per service. A risk, because an overly broad role on the resource or on AI Search opens the door to all grounding data. Test RBAC against the current documentation after every rename.

Foundry cost follows the services used: model inference, AI Search, storage, and the agent runtime. Isolate cost per project so you can attribute consumption, and reconcile estimates against meter charges. See the research on agent cost for the cost model itself.

Teams scaling up from a single proof of concept often keep everything in one project. That works until the first production incident or audit question. Put the layer structure in place early; splitting afterward is expensive.

• Many small projects versus a few large ones: strong isolation versus more management overhead.
• Hosted versus prompt agents: more control versus less operational work.
• Tight RBAC versus speed: safer versus more friction for developers.

• Treating Foundry as standalone model access and cramming everything into one resource.
• Connecting connected services with too-broad permissions.
• Adding observability only once an agent already fails in production.
• Copying RBAC roles from an old guide after a rename.