The research question
How do you structure an enterprise AI platform on Microsoft Foundry so that governance, isolation, and observability hold up?
Why it matters
The trap is seeing Foundry as an API for models. In reality it is a layered platform in which governance, development isolation, and connected services each have their own boundary. Anyone who does not separate these layers mixes production and experiment in one place.
What the research says
The research question behind this topic is not which controls Foundry offers, but why agent systems fall over in production. Yao et al. (2022) showed with ReAct that a model alternating reasoning steps with real actions fabricates less than one that only reasons, because it can check against the outside world along the way. Xi et al. (2023) mapped in a broad survey how fast capabilities are growing. The measurement studies are more sober. Liu et al. (2023) found with AgentBench that agents on longer tasks get stuck repeating actions and compound errors; the problem is not that they cannot perform a step, but that they do not notice they are stuck. Kapoor et al. (2024) showed that much reported agent gain rests on benchmarks that do not control for cost and that are overfit, so part of the measured lead disappears once cost is weighed in. Wu et al. (2023) report with AutoGen that several cooperating agents handle complex tasks better than a single one.
The product layer underneath: Foundry uses a layered architecture with a top-level resource for governance, projects for development isolation, and connected Azure services for storage, search, and secrets, each with its own governance boundary. The Foundry Agent Service supports prompt agents, hosted agents, tools, observability, identity, RBAC, virtual network isolation, and publishing. The RBAC roles were recently renamed, which underlines that this is a fast-moving platform.
What the research does not prove
The line between what the studies actually establish and what we infer from them.
That multi-agent beats a single agent is not established. Wu et al. measure gains; Kapoor et al. dispute precisely that kind of measurement because cost is excluded and the benchmarks are overfit. No research shows multi-agent to be more reliable in production. Treat it as an open question, not an architectural rule.
The agent literature measures almost exclusively on benchmarks, not on business tasks with real data, real permissions, and real consequences. High benchmark scores say little about reliability under long-horizon instruction following: AgentBench and Kapoor et al. both find those two diverge sharply.
On Foundry itself there is no independent research. That prompt agents, hosted agents, or the project structure lead to better outcomes is product documentation plus our reasoning, not a measured result. The reason we recommend observability and evaluation from day one is that the literature shows agents fail without noticing it themselves, not that a study has tested that approach.
Technical context
Think in three layers. The resource is the governance boundary: policy, roles, and network belong here. Projects isolate teams and workloads. Connected services (Storage for data, Key Vault for secrets, AI Search for RAG) each have their own access model. The Agent Service adds the runtime: tools, orchestration, and observability.
Architecture implications
- Use projects to separate production, test, and experiment instead of putting everything in one resource.
- Put network isolation and private endpoints on the resource, not per project again.
- Treat connected services as their own governance boundaries: least privilege on Storage, Key Vault, and Search separately.
- Build observability in from the start; agents without tracing cannot be debugged or accounted for.
Security implications
The distinct governance boundaries are an opportunity and a risk. An opportunity, because you can set least privilege tightly per service. A risk, because an overly broad role on the resource or on AI Search opens the door to all grounding data. Test RBAC against the current documentation after every rename.
Cost implications
Foundry cost follows the services used: model inference, AI Search, storage, and the agent runtime. Isolate cost per project so you can attribute consumption, and reconcile estimates against meter charges. See the research on agent cost for the cost model itself.
Adoption implications
Teams scaling up from a single proof of concept often keep everything in one project. That works until the first production incident or audit question. Put the layer structure in place early; splitting afterward is expensive.
Trade-offs
- Many small projects versus a few large ones: strong isolation versus more management overhead.
- Hosted versus prompt agents: more control versus less operational work.
- Tight RBAC versus speed: safer versus more friction for developers.
Common mistakes
- Treating Foundry as standalone model access and cramming everything into one resource.
- Connecting connected services with too-broad permissions.
- Adding observability only once an agent already fails in production.
- Copying RBAC roles from an old guide after a rename.
For architects
Design Foundry as a layered platform: resource for governance and network, projects for isolation, connected services with least privilege, and the Agent Service with observability from day one. Revisit RBAC choices after every product update.
References
Grouped by source hierarchy. Research carries the reasoning, product documentation carries the implementation. Verify any of it yourself.
Methodology & confidence
The reliability question rests on agent research: ReAct for the reasoning-acting pattern, AgentBench and Kapoor et al. for the measurement critique, AutoGen for multi-agent, and the survey by Xi et al. for the overview (tier 1). Nearly all of that work is preprint, which is the customary publication route in this field but does mean peer review is thinner than for the other topics. That is why the evidence level is set to emerging. Microsoft Learn supplies the platform facts on resources, projects, the Agent Service, and RBAC (tier 3).
The conclusions are deliberately cautious: where the research contradicts itself, as with multi-agent, we say so instead of picking a side.
Peer-reviewed research
Journals, systematic reviews, meta-analyses, and reputable conference proceedings. This is the substantive basis.
- ReAct: Synergizing Reasoning and Acting in Language ModelsYao, S., Zhao, J., Yu, D. et al. (2022). ReAct: Synergizing Reasoning and Acting in Language Models. arXiv preprint (arXiv:2210.03629); presented at ICLR 2023
- AgentBench: Evaluating LLMs as AgentsLiu, X., Yu, H., Zhang, H. et al. (2023). AgentBench: Evaluating LLMs as Agents. arXiv preprint (arXiv:2308.03688)
- AI Agents That MatterKapoor, S., Stroebl, B., Siegel, Z.S. et al. (2024). AI Agents That Matter. arXiv preprint (arXiv:2407.01502)
- AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent ConversationWu, Q., Bansal, G., Zhang, J. et al. (2023). AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv preprint (arXiv:2308.08155)
- The Rise and Potential of Large Language Model Based Agents: A SurveyXi, Z., Chen, W., Guo, X. et al. (2023). The Rise and Potential of Large Language Model Based Agents: A Survey. arXiv preprint (arXiv:2309.07864)
Official technical documentation
How you build and configure it. Answers the implementation question, not the evidence question.
Continue across TechExplained
The same research, applied in other ways.