Skip to main content
|
How-toLevel: Advanced

How to build an agentic application with Fabric and Foundry

From governed data in Microsoft Fabric to an agent in Microsoft Foundry that answers questions about it, with the decision that shapes the architecture at every step.

TechExplained 9 min readPublished: 28 August 2026Last updated: 28 August 2026
#agents#microsoft fabric#microsoft foundry#sql#governance#how-to
Three colleagues at a monitor with code; on the whiteboard behind them the architecture from Microsoft Fabric to Microsoft Foundry and on to the API and the web application
  1. 01

    Get the data ready in Microsoft Fabric

    The data layer is finished before the model appears. Ingestion, transformation and governance are ordinary data engineering, and it is that preparation which gives the agent a clean way in.

    Fabric has three places that speak T-SQL, and they differ on one point that matters here:

    ItemWritableWhat it is for
    SQL database in Fabricyes, full OLTPtransactional data the application also writes
    Warehouseyes, T-SQL DML and DDLanalytical models
    SQL analytics endpoint of a Lakehouseno, read-onlyquerying Delta tables

    Pick SQL database in Fabric and something happens that simplifies your architecture: the data is replicated into OneLake automatically and stored there in Parquet, ready for analysis. A SQL analytics endpoint is created for you at the same time, and that endpoint is read-only over those same mirrored files. So you don't need a copy pipeline to query operational data analytically.

    Decision
    does the agent look at the transactional endpoint or the read-only analytics endpoint? The second cannot break anything by definition, which makes it the boring, correct choice for anything that only has to report.
  2. 02

    Decide who runs the query

    This is the hinge, and the reason two teams using the same products end up with completely different security outcomes. There are two documented routes from a Foundry agent to Fabric data.

    Route A, the Fabric data agent. In Fabric you build a data agent over your Lakehouse, Warehouse, semantic models or KQL databases, publish it, and attach it in Foundry as a tool on your agent. The translation from question to SQL, DAX or KQL is built in. The important part is stated outright: the integration uses identity passthrough, on-behalf-of, and the tool runs queries under the identity of the signed-in user. Row-level security and permissions in Fabric keep applying, without you building anything for it.

    Three conditions come with it. This route is preview, so no SLA and not recommended for production. Every end user needs access to the data agent and to the underlying sources themselves, or the tool call fails. And service principal authentication is not supported: it has to be a user identity.

    Route B, your own tool. You write a function that runs SQL and register it with the agent. Full control, production-ready today, and it works with a service principal or managed identity. Except that every query then runs under that one identity. Row-level security still works, but on the rights of the service principal, not on those of the person asking. Two users get the same answer, even when they are allowed to see different data.

    Decision
    does the answer have to differ per user? If so, identity passthrough is not a detail but the requirement your architecture rests on, and you build authorization yourself if you take route B. If not, route B is simpler and carries no preview dependency.
  3. 03

    Build a data access layer, not database access

    This is where AI projects go wrong most often: the agent gets the connection string of the production database, which answers the question "what may it see" with "everything". Build a layer that is smaller than the database instead.

    Curated views are the cheapest form. One view carrying exactly the columns the agent needs, in a schema of its own, is easier to govern than twenty tables with joins between them:

    CREATE VIEW agent.SalesData AS
    SELECT
        t.TransactionDate,
        c.CustomerName,
        c.Country,
        p.ProductName,
        p.Category,
        t.Quantity
    FROM dbo.Transactions AS t
    JOIN dbo.Customers AS c ON t.CustomerId = c.CustomerId
    JOIN dbo.Products AS p ON t.ProductId = p.ProductId;
    

    Underneath sits the tooling Fabric already ships: row-level security filters rows per identity, object-level security keeps whole tables or columns out of reach, and dynamic data masking masks what may be counted but not read.

    Mind the order: masking and filtering belong in the database, not in the prompt. An instruction saying "do not show salaries" is a request to a model. A security policy is a rule the model cannot route around.

    Decision
    curated views with free SQL on top, or a handful of fixed tools such as get_sales_by_product? Views give flexibility, fixed tools give predictability. In production you usually see both: fixed tools for the questions that keep coming back, a curated view for the rest.
  4. 04

    Set up the Foundry project and pick a model that calls tools

    Create a project in Microsoft Foundry and put the model, the agent, the connections and the tools in it. Foundry Agent Service is generally available, as are tools and toolboxes, but the catalogue is mixed: each tool carries a label saying whether it is GA or preview. The Fabric data agent from step 2 is the example of that.

    The model has to support function calling, and not every model in the catalogue does. The limits page carries a matrix stating per model whether functions are enabled. Two numbers on that same page touch your design: an agent may register at most 128 tools, and rate limiting happens not on the agent but on the model deployment, so in tokens per minute.

    If you build in code, the Microsoft Agent Framework is the direction Microsoft points to for new work, as the successor to AutoGen and the Semantic Kernel Agent Framework. It is preview, and you install the packages as prerelease. In the same move, workflows in Foundry are being retired on 1 December 2026.

    Decision
    wait for GA or build on preview now? If you build on preview, write down what you do when the API changes, and never sell a preview date as a commitment.
  5. 05

    Write instructions that describe a role, and build the tool with guardrails

    Instructions are not a greeting but a job description. "You are a helpful assistant" leaves every decision to the model:

    You are an assistant for company data about sales, customers and products.
    
    Use the available data tools whenever a question asks for company data.
    Never invent data.
    If the information is missing, say explicitly what you are missing.
    When you state a number in a sentence, show how it was calculated.
    

    The tool around it gets hard limits, and you put those in code and not in the prompt. Refuse anything that writes (INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE), allow only the schema your view lives in, and cap the number of rows. Combine that with a read-only user in the database: the first limit is a check you wrote yourself, the second is a right the agent simply does not have.

    Decision
    what does the tool do when in doubt? A tool that explains why a query was refused gives the model something to work with. A tool that silently returns an empty list leads to a confident answer about zero sales.
  6. 06

    Sort out identity without writing down a single secret

    SQL authentication with a username and password does not exist here. SQL databases in Fabric rely on Microsoft Entra authentication, and that is not a recommendation but the only way in. A user, service principal or group also needs the Read item permission on the database.

    For the connection from your API there is a worked route from App Service to a SQL database in Fabric using a managed identity, system-assigned or user-assigned. That leaves no password anywhere in a configuration file.

    One step is nearly always forgotten and otherwise costs an afternoon: at tenant level, the setting for service principals using Fabric APIs has to be on. That is an administrator action, not a developer action, so ask that question early.

    Decision
    one managed identity for the whole application, or one per component? Separate identities cost some setup and return the ability to see in the trail who did what.
  7. 07

    Turn on tracing before the first user arrives

    When an agent does something unexpected, you want to know which tool it picked and which query came out. Application logs do not tell you that. Tracing in Foundry records the input, the output, tool usage, retries, latency and cost per run, via OpenTelemetry into Application Insights.

    Do check the label. Tracing is generally available for prompt agents and hosted agents; for workflow agents and external agents it is preview, and so is monitoring with dashboards and alerts. That difference belongs in your design, because it decides what you may lean on today.

    And consider what you are recording: prompts and answers are data themselves, often with company information in them. They belong under the same retention and access rules as the rest.

    Decision
    who may read the traces? That is a smaller group than who may use the application.

The architecture in one picture

Diagram laden…

The separation this picture enforces is the core: the frontend talks to an API, the API to the agent, the agent to tools, and only the tool to data. That is the same layering the n-tier pattern already prescribes, and the reason is not tidiness but measurability: every layer is a place where you can authorize, limit and log. A frontend that queries the database itself skips all three.

If you want to take this to production, you don't have to start from zero. The Baseline Microsoft Foundry Chat reference architecture describes exactly this setup with private endpoints, zone redundancy and outbound traffic through a firewall.

Common mistakes

MistakeWhy it goes wrongBetter
Agent on the whole databaseEverything in it is reachableCurated views in their own schema
Authorization in the promptAn instruction is not a ruleRLS, OLS and masking in the database
Skipping the identity questionEveryone sees the same answerChoose route A or B up front
The model as a data sourceFacts from weights are not factsModel reasons, platform delivers
Data model in the system promptExpensive, slow and quickly staleTools instead of context
Building the frontend firstThe bug is in the tool, not the buttonTest the agent alone, then the UI
Selling preview as GADelivery and trust both breakRecord status per capability
Tracing afterwardsUnexpected behaviour cannot be tracedTracing on before the first user

What to do next

The guardrails around the model itself, from prompt injection to agent identity, are in How to secure AI workloads and agents. If your agent draws knowledge from documents rather than tables, How to build a RAG solution is the next step. And what an agent costs per conversation is worked out in How to estimate the cost of an AI agent.

The process at a glance

Click a step for its key decision

Decision

Get the data ready in Microsoft Fabric

does the agent look at the transactional endpoint or the read-only analytics endpoint? The second cannot break anything by definition, which makes it the boring, correct choice for anything that only has to report.

Build an agentic application with Fabric and Foundry