What is a semantic layer, and why do agents need one?
A semantic layer holds what a business means by its metrics, entities and relationships, apart from the tables that store them. For dashboards it is a convenience. For an agent writing its own SQL it is the difference between a number and a guess.
A semantic layer is the place a business keeps what it means — its metrics, its entities and the relationships between them — separately from the tables that physically store the data. "Net revenue" is defined once, with its filters and its grain, and every tool that reports it computes it the same way. In the BI world this has existed for decades under several names: Looker's LookML, dbt's metrics, Cube, AtScale, and the old OLAP cubes before any of them.
For dashboards it is a convenience and a consistency measure. For an agent it is closer to a safety control, and the reason is worth being precise about.
Why an agent changes the stakes
A dashboard built without a semantic layer is wrong in a visible way. Somebody notices that two charts disagree, asks which is right, and the disagreement gets resolved — slowly, annoyingly, but in the open.
An agent writing its own SQL is wrong in an invisible way. Asked "how many active merchants do we have?", a capable model will find a merchants table, find a status column, pick the value that looks like it means active, and return a number. The query is valid. The number is plausible. It is also not your figure, because your business excludes test accounts, or counts a merchant active only once it has transacted, or measures it at month end rather than today. Nothing in the answer distinguishes the two, and the person reading it has no way to tell. A revenue metric answered faster is worth nothing if the number is wrong.
That is the asymmetry: the faster and more fluent the thing answering, the more a missing definition costs, because fluency is what removes the signal that something should be checked.
What a semantic layer holds
Four things, in rough order of how often they are missing:
- Metric definitions. What the figure counts, how it is computed, its grain, its unit, and the exclusions that make it the company's own. The exclusions are the part that is never written down anywhere and the part that most often explains why two systems disagree.
- Entities and their identity. What a "customer" is, and which records across which systems are the same customer. See entity resolution.
- Relationships and join paths. How an order reaches a merchant, so a query does not invent a path that happens to produce rows.
- Who owns each definition. Who to ask when the rule looks wrong, and who agreed it in the first place.
Certified, and why that word matters
The useful distinction in any semantic layer is between a definition a person agreed and one a machine proposed. Modern layers can generate candidate definitions from metadata — read the schema, see a numeric column called settled_amount on a table called transactions, and suggest a measure. That is genuinely useful, and it is also a guess about the most consequential thing in the layer.
A guess presented as the company's definition is worse than an empty layer, because every answer afterwards is confidently wrong in the same direction. So the two states have to be visibly different, and what separates them should be provenance rather than a flag somebody remembers to set: a rule a person stated, or two independent sources agreeing on one, is settled; a rule read off a column name waits for somebody to say it is right.
Where disagreement lives
The expensive failure is not an undefined metric — that announces itself the moment somebody asks. It is two definitions of one name, each defensible, living in different systems, with nobody aware there are two. The CRM counts a merchant active on a flag somebody set by hand; the ledger counts one that transacted in the last thirty days. Both are right in their own system.
A layer that silently picks one has destroyed the only evidence that there was a disagreement. The honest behaviour is to keep both, say which system each comes from, and give the figure each way until somebody chooses. That is unsatisfying as an answer and far more useful than a single number nobody can reproduce.
The limits
Worth saying, because the rest of this is an argument for having one:
- A semantic layer is not a warehouse, and does not replace one. Questions that scan history or join six systems for a board pack belong in a warehouse, modelled deliberately. This is the layer that says what the numbers in it mean.
- It has to be maintained, and it goes stale quietly. A definition certified in March can be uncomputable by September because somebody renamed a column — with no error, because nothing runs the expression until a question needs it. Whatever holds the layer has to check that its definitions still match the data they claim.
- It cannot tell you what you have not defined. The most valuable thing a layer can do early on is name the figures people keep asking for that nobody has ever defined, rather than present a tidy list of the ones somebody got round to writing.
- A definition is not a permission. Knowing what a figure means says nothing about who may see the data behind it.
How Memtro approaches it
Memtro's measures register is a semantic layer built for the agent case rather than the dashboard one. It proposes the figures a business already keeps — from the numbers its people quote across systems, and from the catalogue of its connected databases — holds the rule behind each in words as well as, optionally, as an executable expression, and keeps both definitions where two systems disagree.
A rule a person states in a conversation is settled where they say it; one read off a column name waits. Settled rules ride in the assistant's prompt as a line each, the rule itself is fetched when a question turns on that figure, and every answer is graded on how much of what the question named the company actually has a definition or a memory for — so a gap is named rather than filled in.