The last dashboard
Report and dashboard building is ending. Not because nobody wants reports, but because a grounded search bar beats the report to the answer. What the accuracy research really shows, and what has to be true underneath for the answers to be right.
◆A dashboard is a bet placed months in advance
Every dashboard is a pre-answered question. Someone guessed, a quarter ago, what a finance leader would want to know today, and built a page around the guess. When the real question arrives and the page does not answer it, the request goes into a queue, a report writer picks it up, and the answer comes back in weeks. The organization has paid that wait as a standing tax for twenty years.
The next analytics layer, the one the market calls conversational analytics or natural language to SQL, removes the guess. The interface is a search bar. The question is asked in plain words, the answer comes back in seconds, in the shape the question implied: a number when the question asked how much, a trend when it asked over time, a table when it asked for a list, and a written explanation with named drivers when it asked why. Every answer carries its receipt, the exact query that produced it and a drill from the figure to the journal lines behind it.
◆Why bolting AI onto raw tables fails
The obvious objection is that language models get SQL wrong, and the published research says the objection is right, in a specific and useful way. On the long-standing academic benchmark, where databases are small and clean, leading systems land near 91% execution accuracy.1 On a benchmark built to look like real data, with messy content and domain knowledge, a top general model scores near 52%, the best purpose-built systems reach the low seventies, and human experts sit above ninety.2 On the benchmark built from real enterprise warehouses, with hundreds of columns per database and multi-step questions, accuracy falls to roughly 21%.3
The models are not the variable. The ground is. Point a model at hundreds of raw tables and the work of finding the right join across an undocumented schema becomes combinatorial, and the model guesses. Point the same model at a small curated tier where every metric is defined once, every join path is declared, and every table means what its name says, and the guessing surface disappears. That curated tier is what the industry calls a semantic layer, and it decides more than accuracy. The entire industry has converged on this conclusion: the warehouse vendors' native assistants each require an authored semantic model before they answer anything, and each leaves the authoring to the customer's data team.4 The modeling is the product. The chat window is the easy part. We did not invent this pattern and would not trust it if we had: it is the path the platform documentation itself recommends,4 and the layering underneath it is standard, published practice.5 We follow it exactly, with one difference turned to the customer’s advantage: the authored model lives in the customer’s repository, not inside a vendor’s walls.
◆The dataflow, end to end
One question moves through eight stages. Two of them, routing and grounding, decide both the accuracy and the cost of everything downstream.
Most teams treat data modeling and AI cost as two problems, and they are one: a well-modeled semantic tier is what makes the context small, and a small, precise context is at once why the answers are right and why the bill is low. The same artifact does both jobs.
| Lever | How the modeling cuts the cost |
|---|---|
| Small context | The model receives a retrieved slice of the semantic tier, the few views and metric definitions the question touches, not the schema of the warehouse. That is the difference between a few hundred tokens per question and tens of thousands, paid on every single question. |
| Zero-token path | The most-asked questions match saved, verified queries and never reach a model at all. The cheapest token is the one never sent, and in steady state this path should carry most of the traffic. |
| Right-sized model | The router classifies the question and sends lookups to a small fast model, comparisons to a mid one, and diagnostic whys to the strongest. The router itself is the small model. Paying frontier prices for a single-view lookup is the most common waste in production systems. |
| Cached prefix | The stable part of the context, the standing rules and the semantic manifest header, is identical on every call, so it is cached and billed once, not re-sent at full price per question. |
| Results, not rows | The narration model sees aggregates and a bounded summary of the result set, never a raw row dump. The warehouse did the arithmetic; shipping the rows back into the model would pay tokens to re-do work already done correctly. |
| Escalate on failure only | A generation that fails verification retries once, then escalates one tier. Nothing defaults to the strongest model; the strong model earns its invocations. |
◆Not everything has to be predefined
The defined metrics are the floor, not the ceiling. If the base columns exist in the tier, the engine composes: a new report shape, the same view cut by a different ledger, an added column, a year-over-year variance nobody wrote in advance. No developer opens a configuration screen, because the warehouse does the arithmetic on governed columns and the verifier holds the guardrails either way. And when the base data genuinely is not held, the engine says so, plainly: we do not hold that data, and here is who to ask. The refusal is logged, and enough people asking the same missing question is how the next column earns its place in the tier.
◆The loop that makes it more accurate every week
A system like this is not shipped accurate. It is shipped honest, and made accurate by use. One picture, ten steps: the pipeline, the answer path, the outcomes, and one return arrow. The return arrow is the whole point.
Confirmed answers move traffic onto the saved path, so the system gets cheaper as it is used. Corrections repair the definition, so one fix covers every future question that touches it. Refusals decide what gets modeled next, and the test bank reruns on every change, so accuracy only moves one direction.
◆What has to be true underneath
| Layer | The requirement |
|---|---|
| Structure | Data lands raw and unchanged, is conformed with history kept, and surfaces as star schemas with metrics defined once, in code, under version control. The top tier is the semantic model, and it is the only thing the answer engine can see. |
| Trust | Every metric is tied to the source ledger before anything goes live, and control totals re-verify the tie nightly. An answer engine on unreconciled data is a fast way to be confidently wrong. |
| Restraint | The engine is read-only by credential, not by promise. It selects from the semantic tier and can do nothing else. Every generated query is verified before it runs: read-only, allowed views only, sane cost, the user's own data access applied. |
| Division of labor | The database computes every number. The model writes sentences about numbers it was handed, and each figure in the draft is checked against the result set before display. A model that is never asked to do arithmetic cannot get the arithmetic wrong. |
| Routing | A lookup goes to a fast model, a comparison to a mid one, a diagnostic why to the strongest available. Most traffic should skip the model entirely and hit a saved, verified query. |
| Proof | A bank of known-answer questions runs as a release gate. Accuracy is measured as executed results against expected results, reported by question type, on the curated model. A generic accuracy percentage with no stated ground is a marketing number, not a measurement. |
◆What it runs on
Every component is a managed service on the customer's existing cloud; the only custom pieces are code, versioned in the customer's repository. The table names the Google Cloud build. Every row has a direct equivalent on AWS and Azure, and the code does not change when the cloud does.
| Component | Job | On Google Cloud | Elsewhere |
|---|---|---|---|
| Landing | Raw extracts, append-only, immutable | Cloud Storage | S3 · ADLS |
| Warehouse | The semantic tier and every computation | BigQuery | Databricks or Snowflake on any cloud |
| Transform | Models, metrics, tests, docs, lineage | dbt, cloud-neutral, in Git | |
| Orchestration | Extract, load, test, reconcile, on schedule | Cloud Composer | Managed Airflow · Data Factory |
| Models | The routed tiers, small to strong | Vertex AI | Bedrock · Azure OpenAI |
| Retrieval | Matching a question to its slice of the tier | Vector search in the warehouse | Same, warehouse-native |
| App | The search bar, the receipts, the two buttons | Cloud Run | Fargate · Container Apps |
| Identity | Sign-in and per-user data access | Customer's SSO, row-level security enforced in the warehouse | |
| Audit | Question, SQL, verify, run, answer, judgment | Warehouse query history plus app log, retained for the audit period | |
◆One tier over every system
The reason to build the tier once, properly, is that every enterprise system the finance team touches lands in the same place and answers through the same bar. Each source keeps its own sanctioned exit, and the tier is where they stop being separate systems.
That last line is the question no single-system tool can take: what did the reorganization actually do to cost per entity, planned against actual against payroll. It needs the HCM system, the ledger, and the plan in one model with shared dimensions. Once the tier exists, that question is one join, and the answer layer treats it like any other.
◆What actually dies, and what survives
Scheduled report production dies, because the question beats the report to the answer. The report-request queue dies with it. What survives is small and permanent: a handful of watched tiles, and the semantic model underneath, which becomes the most valuable analytics asset the company owns. Teams that treated dashboards as the deliverable will spend years rebuilding. Teams that treated the model as the deliverable have the hard part done, and the search bar is a thin layer on top.
One more thing survives: ownership. The semantic tier, the metric definitions, the prompts, and the test bank are code. Code can sit in the customer's repository, on the customer's cloud, with no license running back to anyone. Or it can sit inside a vendor platform, rented. That choice is being made right now, mostly by default, and it will be expensive to remake later.
◆The questions this paper gets asked
Is the dashboard actually dead?
The production of dashboards is what ends: the request queue, the report writer, the guess about next quarter's question. A handful of watched tiles survives, and the semantic model underneath becomes more valuable, not less.
How accurate is text-to-SQL, really?
On clean academic schemas, near 91%. On realistic data, near 52% for a top general model. On real enterprise warehouses, roughly 21%. Grounding generation in a curated semantic layer is the one fix the published research and every vendor's own documentation agree on, which is why accuracy should only ever be claimed against a stated, curated question bank.
Do we need a vendor platform for conversational analytics?
No. The vendor assistants and the owned build both require the same authored semantic model before they answer anything; the platform is the thin part. Build the model once, in your repository, and it can serve an owned search bar and the vendor tools alike.
What is a semantic layer, and why does the AI need one?
A small curated tier where every metric is defined once, every join is declared, and every table means what its name says. It is why the answers are right, and because it makes the model's context a few hundred tokens instead of a schema dump, it is also why the AI bill stays low.
- Spider 1.0 benchmark leaderboard, exact-match and execution accuracy on academic cross-domain schemas. Yu et al., 2018; leaderboard at yale-lily.github.io/spider.
- BIRD benchmark, execution accuracy on large realistic databases; figures as reported for leading general models and purpose-built systems, with human expert performance published alongside. Li et al., 2023; bird-bench.github.io.
- Spider 2.0, enterprise text-to-SQL over real warehouse schemas and multi-step workflows; multi-step agentic accuracy as reported by the benchmark authors. Lei et al., 2024; arxiv.org/abs/2411.07763.
- Vendor documentation for the warehouse-native assistants describes the same grounding this paper uses: an authored semantic model with declared join paths, verified example queries, and a curated table set, with the authoring left to the customer's team. Practitioner accounts consistently identify that authoring as the accuracy lever.
- The warehouse layering here is the widely published standard: raw, conformed, and consumption tiers (the medallion pattern, Databricks documentation) with Kimball dimensional modeling and dbt-defined, version-controlled metrics.
- Benchmark figures are not directly comparable across evaluations; metrics and settings differ. They are cited here for the shape of the result, a steep fall from clean schemas to real ones, which is the finding this paper rests on. Verified July 2026.