Back to posts
Part 2 of 5 · Building Accurate Fabric Data Agents
Jun 23, 2026•10 min read

Writing Instructions That Make Fabric Data Agents Answer Accurately

Standing up a Fabric Data Agent takes an afternoon. Making it answer correctly takes iteration: agent instructions, data-source descriptions, example queries, and a small evaluation set you re-run after every change. Here is the unglamorous middle step nobody blogs about.

After wiring up service principal auth, I pointed my Data Agent at a real lakehouse and asked it a real question. The answer was wrong. Not broken, not an error: confidently, plausibly wrong. The gap between a Data Agent that runs and a Data Agent you can trust is instructions, and closing that gap is the least glamorous, most valuable work in the whole build.

This post sits between standing the agent up and embedding it anywhere. It covers where instructions actually live in the Data Agent configuration, why schema hygiene matters more than clever prompting, and the small evaluation loop that turns guesswork into progress.

  • 3 places instructions live. Agent-level instructions, per-data-source instructions, and example queries. Each solves a different class of wrong answer, and mixing them up wastes effort.
  • Under 10 tables an agent handles well. Not a hard limit, a practical one. Every extra table is another chance to join the wrong thing. Attach what the questions need and nothing else.
  • 15 questions in my eval set. A spreadsheet of question and expected-answer pairs, re-run after every instruction change. Small enough to run in minutes, large enough to catch regressions.

Why Agents Get It Wrong on Real Schemas

A Data Agent works by reading your schema metadata, generating a query (SQL against a lakehouse or warehouse, DAX against a semantic model), running it, and summarizing the result. Every wrong answer I have debugged traces back to one step: the generated query was reasonable for the schema the agent could see, and wrong for the business reality it could not.

Demo datasets hide this because they are curated. Real lakehouses have a sales table and a sales_v2 table and a sales_backup_march table. They have a date column that is actually the load timestamp, revenue in three currencies with no currency column, and soft-deleted rows nobody filters out. A human analyst learned these landmines over months. The agent gets one look at the metadata. Instructions are how you hand over that tribal knowledge.

Three Layers of Instruction, Three Different Jobs

The Data Agent configuration gives you three distinct places to steer behavior, and each one has a job. Putting the right guidance in the wrong layer is the most common mistake I see.

Agent instructions

Global behavior across all sources: which source answers which kind of question, business definitions, default filters, tone, and what to refuse. This is where "revenue always means net revenue" lives.

Data source instructions

Per-source guidance attached to a specific lakehouse, warehouse, or semantic model: which tables to prefer, which columns are traps, how the grain works. This is where "use sales_v2, never sales" lives.

Example queries

Question and query pairs stored with a source. The agent uses them as reference patterns when generating new queries. This is where correct join paths and gnarly filter logic get taught by demonstration.

A useful rule of thumb: prose instructions change what the agent decides, example queries change what it writes. If the agent picks the wrong table, fix it with instructions. If it picks the right table and writes the wrong join, fix it with an example query.

What Good Agent Instructions Look Like

Resist the urge to write a personality. The agent does not need to be told it is a helpful assistant. It needs the specific facts a new analyst would need on day one, stated flatly. Here is a trimmed version of the instructions block from a retail analytics agent I built, with three sources attached:

# Agent instructions: retail analytics data agent
## Routing
- Questions about sales, revenue, or orders: use the Sales semantic model.
- Questions about store locations, trade areas, or distances: use the Stores lakehouse.
- Questions about web traffic: use the Web Analytics warehouse.
## Definitions
- "Revenue" and "sales" always mean the [Net Revenue] measure, net of returns.
- "Last quarter" means the last complete fiscal quarter. Fiscal year starts July 1.
- "Stores" means active stores only (status = 'OPEN') unless the user asks about closed stores.
## Behavior
- All currency values are USD. State this in answers involving money.
- If a question is ambiguous about time period, assume the last complete month and say so.
- Do not answer questions about individual employees. Direct those to HR reporting.

Three sections doing three jobs: routing between sources, pinning down vocabulary, and setting defaults for ambiguity. Notice the last line is a refusal rule. Scope is an instruction too, and telling the agent what not to answer is as valuable as anything else in the block.

Clean the Schema Before You Write a Word

The cheapest instruction is the one you never have to write. The agent reads table names, column names, and descriptions, so every hour spent on metadata hygiene saves paragraphs of corrective prose. Before touching the instructions box, I do a hygiene pass on the source itself:

  • Deselect the junk: when attaching a source you choose which tables the agent can see. Leave out staging tables, backups, and anything suffixed _old, _tmp, or _v1. In a semantic model, hide technical columns (surrogate keys, ETL audit fields) so the agent never reasons about them.
  • Write column descriptions: lakehouse tables and semantic model fields both support descriptions, and the agent reads them. amt2 with the description "Order line amount in USD after discounts, before tax" is worth ten lines of agent instructions.
  • Fix hostile names where you can: in a semantic model, renaming is free and invisible to the underlying source. dt_ld_utc becomes "Load Date (UTC)". In a lakehouse, a view with sane column names over an ugly table achieves the same thing.
  • Define measures for anything computed: if "margin" is a calculation, make it a measure with a description rather than hoping the agent derives the formula. Agents are far more reliable selecting an existing measure than inventing arithmetic.

This is the same discipline that makes a semantic model pleasant for human report builders. A Data Agent is just a very literal-minded new hire, and it benefits from exactly the same tidiness.

Fewer Tables Beats More Tables

The temptation is to attach everything so the agent can answer anything. In practice accuracy falls as the attached surface grows, because every additional table adds candidate join paths and near-duplicate columns for the agent to choose between. My working approach: start from the question list, not the schema. Write down the twenty questions the agent actually needs to answer, attach the minimum set of tables that covers them, and stop.

When two audiences need genuinely different scopes, build two agents. A "store performance" agent with six tables and a "supply chain" agent with eight will each outperform one omniscient agent with thirty, and their instruction blocks stay short enough to maintain.

Example Queries: Teaching by Demonstration

For lakehouse and warehouse sources, the configuration lets you store example queries: a natural-language question paired with the SQL that correctly answers it. When a user asks something similar, the agent leans on the stored pattern instead of inventing its own join logic. This is the single highest-leverage fix for structurally wrong queries. One well-chosen pair fixed a join bug that three paragraphs of prose instructions had failed to fix:

-- Example query pair stored with the Stores lakehouse source
-- Question: "Which stores had the highest revenue per square foot last month?"
SELECT s.store_name,
SUM(f.net_amount) / MAX(s.sq_ft) AS revenue_per_sqft
FROM fact_sales f
JOIN dim_store s ON f.store_key = s.store_key -- join on store_key, NOT store_id
WHERE s.status = 'OPEN'
AND f.order_date >= DATE_TRUNC('month', CURRENT_DATE - INTERVAL 1 MONTH)
AND f.order_date < DATE_TRUNC('month', CURRENT_DATE)
GROUP BY s.store_name
ORDER BY revenue_per_sqft DESC

Choose examples that encode the decisions the agent keeps getting wrong: the correct join key, the status filter, the complete-month date logic. Three to five pairs per source covering your trickiest patterns beats twenty trivial ones. And keep them current: an example query referencing a renamed column silently becomes a lesson in doing it wrong.

The Evaluation Loop: Test Like You Mean It

Here is the discipline that separates an agent you demo from an agent you ship. Instructions interact in unpredictable ways, and a change that fixes one question can quietly break another. So before tuning anything, I build a small evaluation set: ten to twenty real questions with known correct answers, computed independently in SQL or an existing report. Include easy ones, ambiguous ones, and a couple the agent should refuse.

Then the loop is mechanical: run every question through the agent, record pass or fail, make one instruction change aimed at the worst failure, and run the whole set again. Not just the question you were fixing, the whole set. The Data Agent editor shows the generated query alongside each answer during testing, and checking that query is the difference between "the number looks right" and "the number is right". A wrong query returning a plausible figure is the failure mode that reaches production.

Mine lives in a spreadsheet with four columns: question, expected answer, agent answer, pass/fail. It is not sophisticated and it does not need to be. What matters is that it exists before you start tuning, because without it every instruction change is a vibe.

Failure Modes I Keep Seeing

Ambiguous measures

The model has [Revenue], [Net Revenue], and [Revenue LY], and the agent picks one arbitrarily per question. Fix: declare the default in agent instructions, and hide or clearly describe the alternatives in the model.

Date grain confusion

"Last month" becomes last 30 days, or a calendar month against a fiscal calendar, or order date when the user meant ship date. Fix: define the fiscal calendar and the default date column explicitly, and encode complete-period logic in an example query.

Synonym misses

Users say "locations", "branches", or "sites"; the table is dim_store, so the agent claims it has no location data. Fix: list the vocabulary mappings in the data source instructions. Users will not learn your schema's dialect, and they should not have to.

Silent double counting

A join across grains (orders to order lines, stores to daily snapshots) inflates a sum and nothing errors. Fix: state each table's grain in its description, and make your eval set include a question where double counting would show.

Where This Fits in the Series

If the earlier posts were about plumbing, this one is about trust. Standing up an agent proves the concept, and service principal auth makes it callable from real applications. But the moment an agent is embedded somewhere users cannot see the generated query, its answers get taken at face value. The instruction and evaluation work in this post is what earns that. Next up: a deep dive on example queries, the layer closest to the generated SQL, and how to import a whole library of them with the Python SDK.

None of this is glamorous. It is column descriptions, a spreadsheet, and re-running the same fifteen questions after every change. It is also the entire difference between a Data Agent that demos well and one that a regional manager can rely on without an analyst checking its homework.

Further Reading

Tags

Microsoft FabricAIData AgentsPrompt EngineeringSemantic Modeling