Example Queries: Teaching Fabric Data Agents to Write Correct SQL
Instructions change what a Data Agent decides. Example queries change what it writes. This post goes deep on the highest-leverage layer: what a good question-and-query pair looks like, how to develop them from real failures, and how to import and validate a whole library of them with the Python SDK.
In the last post I called example queries the single highest-leverage fix for structurally wrong queries, and then gave them one section. That was unfair. A well-built example query library does more for answer accuracy than everything else in the configuration combined, and building one well is a craft with its own rules.
This post is the deep dive: how the agent actually uses your examples, what separates a pair that teaches from a pair that confuses, a workflow for developing them from real failures instead of imagination, and the part almost nobody uses yet, importing and validating an entire library programmatically with the Fabric Data Agent Python SDK.
- 4 examples retrieved per question. The agent does not read your whole library. It retrieves the few most relevant pairs, typically the top four, and feeds them into query generation. Relevance is everything.
- 3 scores every example must pass. The SDK validator rates each pair on Clarity, Relatedness, and Mapping. An example is high quality only when all three are positive.
- 1 SDK call to import a library. A Python dictionary of question-and-SQL pairs and one call to
add_fewshots. Your examples become an artifact you version, review, and redeploy like code.
How the Agent Actually Uses Them
Example queries, also called few-shot examples, are stored question-and-query pairs attached to a data source. When a user asks something, the agent retrieves the most relevant pairs from your library and uses them as reference patterns while generating its own query. It is retrieval, not memorization: the agent is not looking up an answer, it is borrowing your structure, your joins, and your filter logic for a question that resembles one it has seen.
Three mechanics follow from this that shape everything else in the post:
- Only the top matches are used: typically around four per question. Twenty near-duplicate pairs do not give you twenty times the coverage, they compete with each other for the same retrieval slots.
- Every pair is validated against the source schema: an example that references a dropped column or fails validation is silently excluded. Your library can rot without ever throwing an error.
- Support depends on the source type: lakehouses, warehouses, and Eventhouse KQL databases take example queries. Semantic models do not, so for DAX sources your only levers are the model itself and prose instructions.
One debugging tool worth knowing before you write a single pair: the run steps view in the Data Agent editor shows exactly which examples were retrieved for a given question. If the wrong ones keep showing up, the fix is usually a clearer question on the example, not more examples.
Anatomy of a Pair That Teaches
A good example query encodes a decision the agent would otherwise get wrong. Here is one from my retail agent's library, and it is carrying four separate lessons:
The four lessons: which column means revenue, the correct two-hop join path to region, the fiscal quarter filter pattern, and, through the comments, where to substitute values. That last one matters more than it looks. The agent reads SQL comments and treats them as guidance, so a comment like "substitute the customer_id here" turns a static answer into a reusable template. Comments are how one pair generalizes to a family of questions.
The question side has its own craft. Write it the way a user would type it, not the way an analyst would specify it. Users ask "revenue by region last quarter", they do not ask "net_amount aggregated by region_name for fiscal quarter 2026-Q1". Retrieval matches the user's phrasing against your example's phrasing, so an example written in schema dialect can fail to surface for exactly the questions it was meant to fix. And keep each question specific and single-intent: a metric, a timeframe, a filter. Vague questions make weak retrieval keys and teach ambiguous lessons.
Develop Them from Failures, Not from Imagination
The tempting workflow is to sit down and write ten examples that feel representative. The problem: you will write examples for queries the agent already gets right, and miss the ones it gets wrong. The library ends up large, redundant, and aimed at nothing.
The workflow that works is failure-driven, and it plugs straight into the evaluation loop from the last post:
- Run the eval set, collect the failures: every question where the generated query was structurally wrong is a candidate. Wrong table and wrong routing failures belong to instructions, structural failures belong here.
- Write the correct query yourself: verify it returns the right answer against the source. This SQL becomes the example, with comments marking the substitution points and the non-obvious decisions.
- Pair it with the user's actual phrasing: take the question verbatim from the failure, not a cleaned-up version. That phrasing is exactly what future retrieval needs to match.
- Re-run the whole eval set: confirm the failure now passes, and check the run steps view to confirm your new example was actually retrieved. An example that fixes the target question but hijacks retrieval for unrelated questions is a net loss, and this is where you catch it.
Grown this way, the library stays small and every pair earns its place. My working target is still three to five pairs per source covering the trickiest patterns. When the count creeps past ten, it is usually a sign that overlapping examples are accumulating, and overlap is worse than absence: two pairs answering the same intent with different logic give the agent a coin flip.
Importing a Library with the SDK
Typing pairs into the portal one at a time is fine for three examples and miserable for thirty. It also leaves your library living only inside the artifact, invisible to code review and unrecoverable if someone edits it badly. The Fabric Data Agent Python SDK fixes both problems: the library becomes a dictionary in a notebook or repo, and import is one call.
The shape is deliberately simple: keys are the natural-language questions, values are the SQL. Which means the source of truth for your examples can be a JSON file sitting in the same repo as everything else, reviewed in pull requests, with the notebook reading the file and pushing it to the agent. When a column gets renamed, the fix is a commit and a re-import, not an archaeology session in the portal.
Validate What You Imported
Import is half the story. The SDK also evaluates the library you just loaded, and this is the feature that turns example queries from folklore into engineering. One call runs every pair through validation and hands back the results as DataFrames:
Each pair gets three scores, and a pair is high quality only when all three are positive. Clarity asks whether the question is specific and single-intent. Relatedness asks whether the SQL actually answers that question, the right metric at the right granularity. Mapping asks whether every literal in the question, every number, date, and category, appears explicitly in the SQL. Mapping failures are the sneakiest: a question that says "over 100 in March for the West region" paired with SQL missing the month filter looks fine at a glance and teaches the agent to drop filters.
After validation, the SDK runs conflict detection across the pairs that passed: two examples with the same intent hitting different tables, or computing the same metric with different aggregation logic, get flagged with a confidence rating. This catches the exact failure mode that grows silently as libraries age and multiple people add examples. One caveat: the evaluation utility currently covers SQL examples only, so KQL libraries still get validated by the schema check and your own eval set.
Failure Modes I Keep Seeing
Overlapping pairs
Two examples answer the same intent with different tables or different aggregation, and the agent alternates between them per question. Fix: run conflict detection after every import, keep one pair per intent, delete the loser.
Analyst-dialect questions
The question side is written in schema vocabulary, so retrieval never matches how users actually phrase things and the example sits unused. Fix: take questions verbatim from real failures, and confirm retrieval in the run steps view.
Silent rot
A rename or schema change invalidates a pair, validation quietly stops sending it, and accuracy degrades with no error anywhere. Fix: keep the library in source control and re-run evaluate_few_shots on a schedule, not just at import time.
Teaching the trivial
Twenty pairs of single-table SELECTs that the agent already generates correctly, crowding the retrieval slots that should surface the hard join. Fix: every pair must encode a decision the agent demonstrably got wrong without it.
Where This Fits in the Series
This closes out the accuracy arc. Five Things framed what separates a working agent from a demo, the instructions post covered the prose layers and the evaluation loop, and this post finishes the job at the layer closest to the generated query. The division of labor is worth repeating: instructions change what the agent decides, example queries change what it writes, and the eval set is how you know either one worked.
With the library imported and validated, one piece of the accuracy story is still manual: re-running the eval set yourself after every change. The next post automates it, turning the spreadsheet into a scheduled evaluation pipeline with the same SDK.
Further Reading
- Writing Instructions That Make Fabric Data Agents Answer Accurately: the previous post in this series, covering the prose layers and the evaluation loop this post builds on
- Data Agent example queries: Microsoft Learn documentation on few-shot examples, validation scores, and conflict detection
- Fabric Data Agent Python SDK: the management-plane SDK used for the bulk import and evaluation workflow in this post