What is structured vs unstructured data?
Structured data has a schema declared before the data arrives; unstructured data does not. The point that matters is that most valuable business questions need both, and the join between them is identity rather than format.
Structured data has a schema declared before the data arrives: rows and typed columns, each field with an agreed meaning. A ledger entry, an order line, a deal with an amount and a close date. You can filter it, total it and join it, because something decided in advance what each field means.
Unstructured data has no such declaration. The meaning is in the prose, and nothing in the storage knows what it says. A support thread, a meeting transcript, a contract, a long argument in a channel.
Semi-structured data sits between the two: shape, but not a fixed schema. A JSON webhook payload, an email with reliable headers and an unconstrained body; a spreadsheet export whose free-text notes column holds the important part.
The split is less about content than about where the meaning lives. An invoice PDF is a highly structured document stored as unstructured data: every field is present — number, date, net, VAT, total — and nothing in the system holding it knows any of them, so it can file the invoice and cannot add it up.
The claim that 80 or 90 per cent of company data is unstructured is an industry estimate repeated more often than measured; the proportion changes no decision anybody faces.
The questions that need both
Has this customer paid, and why are they unhappy?
The first half is a number in the ledger. The second is a reason in a mailbox, a transcript or a ticket. Neither half is an answer, and the question is the ordinary shape of commercial work:
- Should we renew at the discount they are asking for? Contract terms, usage figures, and the tone of the last six months.
- Is this churn risk real? Support volume, payment history, and what somebody was told on a call.
Each needs a figure you can rely on and a reason in prose, and neither store answers it alone: a search index cannot do arithmetic and a warehouse cannot read a sentence.
The join is identity, not format
The hard step is not reading the prose or querying the table; both are solved problems. It is knowing that the row and the thread are about the same customer.
Structured systems join on keys. Prose has no keys. The mailbox has an email address, the ticket has a name as somebody typed it, the invoice has a billing entity with "Ltd" on the end, and the warehouse has an account id that appears nowhere else. Nothing in that list is a shared identifier, and resemblance does not stand in for one: two records about one customer may share no words, while two about different customers may share many, because both are complaints about delivery.
Which makes entity resolution — deciding which records refer to the same real thing — the step that turns "we hold both kinds of data" into "we can answer questions that span both".
Why the usual architecture cannot answer them
The conventional split sends structured data to a warehouse and unstructured data to a search or vector index. Each is good at its half and blind to the other's, and the boundary is organisational as much as technical: one team owns the warehouse, somebody else owns the documents, and nobody owns the join.
So the join is done by a person with two browser tabs. Putting the documents in a vector store next to the warehouse does not fix it: that produces two retrieval systems with no shared notion of who the customer is, and a model left to infer from a sentence which rows and threads concerned the same account. That inference is where confident wrong answers about company data come from.
Resolving entities across both sides removes the guess: once the row, the thread and the invoice hang off one node, the figure and the reason arrive together.
Where this stops
Resolving identity makes the join possible. It does not make the two halves equal evidence. A number in a ledger is an assertion with an owner and an audit trail. A support thread is one person's account of one moment, often wrong about the facts and still the best source for how somebody felt about them. An answer that blends them should say which part came from where.
The merging direction stays dangerous, and this shape of answer hides it: two customers collapsed into one give you a figure from one and a reason from the other, told as a single coherent story.
A system spanning both kinds spans only what is connected. If no mailbox is connected, the why half of the question has no source at all, and an answer that does not say so is overstating itself.
Where Memtro fits
Memtro's connectors read both kinds: records from CRM, support, finance and work-management systems, prose from mail, chat, meeting transcripts and wikis, and read-only SQL over databases and warehouses — 146 systems in all. Results are anchored into the memory graph deterministically, on external ids and email rather than a model's guess, so a CRM contact and a mailbox correspondent are one node. The extractor also lifts the measurable half onto the entity it belongs to — an amount with its unit, a date, a count — because which deals are over £50k cannot be answered by reading sentences and hoping you found them all.