In Short: A Translation Pipeline, Not Magic
Every natural language data answer in the Microsoft stack - Copilot in Power BI, Q&A visuals, Copilot in Teams via Fabric IQ - is produced by roughly the same pipeline: parse the question, match its terms against the semantic model's metadata, generate a query (DAX for semantic models), execute it under the user's permissions, and turn the result into a visual or a sentence. No stage involves the model "knowing" your data; every stage involves reading what your metadata says about it.
Understanding the pipeline matters because it tells you exactly where answers go wrong and exactly what to fix. The failures are rarely mysterious once you know which stage produced them.
Stage by Stage
Parsing. The question is decomposed: metrics, groupings, filters, time frames. "Top five products by margin in Germany last quarter" becomes a metric (margin), an entity (products), a ranking (top five), a filter (Germany) and a period (last quarter). Modern models handle this stage well; it is rarely the problem.
Matching. Each parsed term must bind to something in your model: margin to a measure, Germany to a value in a country column, products to a table. This is where most wrong answers are born. The matcher works with names, descriptions, synonyms and endorsement signals, which is why a measure named Adjusted Revenue with a description beats Rev_adj_v3 every time, and why preparing the model is the highest-leverage work in this whole area.
Generation. A DAX query is written against the matched objects. Generation quality has improved to the point that, given correct matches and correct relationships, the query is usually right. Given a mistaken many-to-many or a bidirectional filter set for convenience, the query faithfully reproduces the modelling error at conversational speed.
Execution and narration. The query runs under the asking user's identity - row-level security applies, as it does in any report - and the result is rendered or narrated. Narration adds its own risk: emphasis and interpretation the data does not strictly carry. Numbers come from your model; adjectives come from the language model.
Where It Breaks, and What Each Break Looks Like
- Vocabulary gap - the user says turnover, the model says revenue; the matcher guesses or fails. Fix with synonyms and business-language naming
- Ambiguity - three measures all plausibly called margin; the matcher picks one, silently. Fix by consolidating duplicates and endorsing the authoritative model
- Homonyms in data - Georgia the country versus Georgia the US state; ambiguous values need well-named, well-described columns
- Aggregation traps - averaging a ratio, summing a percentage: explicit measures with correct aggregation logic prevent the generator improvising
- Relationship errors - the query multiplies or drops rows exactly as a visual would; the fix is star schema discipline, not prompt engineering
A useful diagnostic habit: when an answer is wrong, ask which stage failed. Wrong number with the right terms is usually relationships or aggregation. Right number for the wrong thing is matching. Refusals are vocabulary gaps.
Making It Reliable
Reliability is a testing discipline, not a hope.
- Build a question bank: the real questions your business asks, harvested from tickets, chat logs and analysts
- Run the bank against the model before enabling anything for viewers, and score answers pass or fail
- Fix failures in the model - names, synonyms, descriptions, relationships - never by coaching users to rephrase
- Retest on every metadata change, because matching behaviour shifts when names and synonyms do
- Watch live usage once enabled: failed and oddly-answered questions are a continuous free audit
Teams that do this treat the question bank the way engineers treat a test suite, and it shows: their Copilot rollouts survive contact with sceptical users because the first answers people try actually land.
Sources and Further Reading
Frequently asked
In stages: the question is parsed, its terms are matched against the semantic model's metadata (names, synonyms, descriptions), a DAX query is generated against the matched objects, the query executes under the user's permissions, and the result is rendered as a visual or narrated answer. Every stage depends on the model's metadata quality.
Usually a matching failure: the user's term matched the wrong measure or column, or an ambiguous term matched something plausible but incorrect. The query then executes perfectly against the wrong target. Better naming, descriptions and synonyms fix most of these; the rest are usually relationship or modelling errors.
Yes. Generated queries execute under the asking user's identity, so RLS and workspace permissions filter the result exactly as they would a report visual. That makes RLS correctness more important, not less, because a misconfiguration surfaces as a fluent answer containing data the user should not see.
The practice of teaching the model the business's vocabulary: synonyms for tables, columns and measures, so that revenue, sales, turnover and income all resolve to the measure you intend. It is the highest-leverage tuning available for natural language quality.
Collect the twenty or thirty questions the business genuinely asks, run them against the model, and score the answers. Fix failures at the model level (names, synonyms, relationships), not by teaching users to phrase questions differently, then retest. Repeat after every model change that touches metadata.
