The research question
Which security controls does an enterprise AI platform genuinely need, and where in the architecture do they belong?
Why it matters
AI shifts the attack surface. Where classic apps lean on network and identity, agents and RAG solutions introduce new risks: prompt injection, data leakage through grounding, and agents that can do more than intended. These risks are architecture decisions, not an afterthought.
What the research says
The research frames prompt injection as a structural problem, not a bug. Greshake et al. (2023), at AISec co-located with CCS, showed that an attacker can take over an LLM application without touching the prompt: hiding instructions in content the system later retrieves is enough, so injection travels through the data layer. Zou et al. (2023) demonstrated that automatically discovered attack strings break alignment and transfer across models, which means filtering on known attack text is structurally behind. Wallace et al. (2024) counter with an instruction hierarchy, training the model to weight system instructions above retrieved text; they measure substantial improvement, but not closure. Zeng et al. (2024) found RAG cuts both ways: it reduces leakage from training data while introducing leakage from the retrieval corpus. Wang et al. (2023), in DecodingTrust, measured that GPT-4 follows instructions more precisely and is therefore more susceptible to misleading instructions than GPT-3.5.
The product layer follows from this. Microsoft 365 Copilot only works with content the user can already access through Microsoft Graph, and prompts, responses, and Graph data do not train the underlying models. Microsoft Purview provides the data layer on top with Purview DSPM, sensitivity labels, DLP, and Insider Risk Management, spanning Copilot, Foundry, and third-party AI apps. Defender for Cloud adds AI security posture management to its CNAPP, Sentinel provides detection, and OWASP and NIST supply the frameworks to test your controls against.
What the research does not prove
The line between what the studies actually establish and what we infer from them.
No published defense is simultaneously robust to adaptive, optimized attacks and practical at production latency. Wallace et al. measure improvement, not a guarantee, and the work of Zou et al. shows that a training-time defense can in principle be re-attacked. Presenting prompt injection as solved goes beyond the evidence.
The studies also measure model behaviour in research settings, not the effectiveness of Purview, Defender for Cloud, or Sentinel inside a real organisation. No independent research measures how many incidents those products actually prevent. The layered architecture we recommend below follows logically from the research findings, but has not itself been tested experimentally: that is our interpretation, not a measurement.
The finding by Wang et al. that a more capable model can be less robust was measured on 2023-era models. Treat it as a reason to test per model, not as a law.
Technical context
An enterprise AI platform has four security planes: identity (who or what calls the model), data (what grounding may see), model and agent (which tools can be invoked), and observability (what happened). Foundry separates these through a resource for governance, projects for isolation, and distinct governance boundaries for connected services such as Storage, Key Vault, and Azure AI Search.
Architecture implications
- Fix the identity boundary: agents run under a managed identity with least privilege, not under an administrator's rights.
- Isolate grounding data per project and label sources with sensitivity labels, so DLP and encryption rights travel with the answer.
- Place prompt-injection controls and excessive-agency limits (which tools, which scope) on the agent layer, not in the prompt.
- Close network access with virtual network isolation and private endpoints toward Storage, Key Vault, and Search.
Security implications
The biggest leaks are rarely the model itself. They come from oversharing (Copilot surfaces what was already shared too broadly), from agents with too wide a tool scope, and from grounding that pulls in confidential sources without a label. Treat Purview DSPM as the starting point: first see which sensitive data AI can reach, fix the permissions, then detect misuse with Sentinel.
Cost implications
Security carries a cost you must model up front. Sentinel cost depends heavily on ingestion volume, analytics tier, data lake tier, and retention; unfiltered AI logs can inflate the bill. Decide deliberately which signals go to the expensive analytics tier and which to cheaper storage.
Adoption implications
Do not start with detection but with data hygiene. Organizations that roll out Copilot without first solving oversharing get incidents that no security tool fixes, but governance does. Involve data owners early.
Trade-offs
- Least privilege versus speed: tight tool scopes slow agents down, but wide scopes are the core of excessive agency.
- Full logging versus cost: sending everything to the analytics tier is safe but expensive; tiering forces choices.
- Central versus federated controls: central Purview policies are consistent, but teams want per-project autonomy.
Common mistakes
- Trying to solve prompt injection with a better system prompt instead of controls on the tool layer.
- Rolling out Copilot without first running a Purview DSPM scan for oversharing.
- Running agents under broad administrator rights instead of a dedicated managed identity.
- Sending all AI telemetry unfiltered to the expensive Sentinel tier.
For architects
Design AI security in layers: identity and network as the foundation, Purview as the data layer, agent controls against prompt injection and excessive agency, and Sentinel as detection. Map your controls explicitly onto a recognized framework (OWASP LLM Top 10, NIST AI RMF) so your choices are testable.
References
Grouped by source hierarchy. Research carries the reasoning, product documentation carries the implementation. Verify any of it yourself.
Methodology & confidence
The risk analysis rests on peer-reviewed and widely cited research into prompt injection, adversarial attacks, RAG privacy, and model robustness (tier 1). The OWASP GenAI project and the NIST AI RMF supply the frameworks to test controls against (tier 2). Microsoft Learn is used for what the products factually do and how you configure them (tier 3), not as evidence that an approach works.
Note the status of the evidence: Greshake et al. appeared at the AISec workshop co-located with CCS and DecodingTrust at NeurIPS, so both are peer-reviewed. The attack and defense papers by Zou et al. and Wallace et al. are preprints that serve as canonical references in this field. Confidence strong for the risk categories, weaker for the effectiveness of individual defenses.
Peer-reviewed research
Journals, systematic reviews, meta-analyses, and reputable conference proceedings. This is the substantive basis.
- Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionGreshake, K., Abdelnabi, S., Mishra, S. et al. (2023). Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. 16th ACM Workshop on Artificial Intelligence and Security (AISec), at CCS 2023DOI 10.1145/3605764.3623985
- DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT ModelsWang, B., Chen, W., Pei, H. et al. (2023). DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models. Advances in Neural Information Processing Systems (NeurIPS 2023)
- Universal and Transferable Adversarial Attacks on Aligned Language ModelsZou, A., Wang, Z., Kolter, J.Z., Fredrikson, M. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv preprint (arXiv:2307.15043)
- The Instruction Hierarchy: Training LLMs to Prioritize Privileged InstructionsWallace, E., Xiao, K., Leike, R. et al. (2024). The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. arXiv preprint (arXiv:2404.13208)
- The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)Zeng, S., Zhang, J., He, P. et al. (2024). The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG). arXiv preprint (arXiv:2402.16893)
Academic and institutional
Research institutes and standards bodies such as NIST, ISO, IEEE, and ACM.
Official technical documentation
How you build and configure it. Answers the implementation question, not the evidence question.
- Data, privacy, and security for Microsoft 365 CopilotData, privacy, and security for Microsoft 365 Copilot. Microsoft Learn
- Microsoft Purview data security and compliance protections for AIMicrosoft Purview data security and compliance protections for AI. Microsoft Learn
- AI threat protection in Microsoft Defender for CloudAI threat protection in Microsoft Defender for Cloud. Microsoft Learn
- Microsoft Sentinel in the Microsoft Defender portalMicrosoft Sentinel in the Microsoft Defender portal. Microsoft Learn
Continue across TechExplained
The same research, applied in other ways.