Knowledge Graphs
By the end of this module
- Separate a mention from an entity, and say why the distinction does the work
- Judge whether two mentions refer to the same thing, and why strings cannot decide it
- Explain what putting validity dates on a fact buys you
Definition
The previous module ended on persistent memory and the facts worth keeping in it. Storing those facts as text works until you need to answer a question that spans several of them, at which point you want structure: entities, the relations between them, and when each was true.
A knowledge graph is that structure. Its value is not the diagram. It is that a question like "which merchants does this accountant act for, and since when" is a traversal rather than a search over prose that may or may not contain the answer.
Mentions and Entities
One distinction carries most of the difficulty. A mention is a piece of text referring to something. An entity is the thing itself. Many mentions, across many documents and conversations, point at one entity.
| Piece | What it is |
|---|---|
| Mention | A piece of text referring to something: a name, an identifier, or a phrase like "the merchant". |
| Entity | The thing itself, with one canonical record however many ways it gets written. |
| Relation | A typed link between two entities. Supplier of, employed by, acts for. |
| Type | What kind of thing an entity is, which constrains what relations are even possible. |
| Confidence | How sure the pipeline is, carried on entities and relations alike rather than thrown away. |
| Validity | When the fact started being true, and when it stopped. |
Extraction turns text into mentions and is the easy half. Deciding which mentions are the same entity is the hard half, and it is where a graph either becomes useful or quietly fills with duplicates and false merges.
Resolving Mentions
Six pairs. For each, decide whether they refer to the same thing. The tally at the end compares your answers with what matching on the text alone would have produced.
Do these two mentions refer to the same thing?
ACME LIMITED
Acme Ltd.
Exact matching fails in both directions at once, which is what makes this hard rather than merely fiddly. Two different things share a name, and one thing is written six ways. Fuzzy matching fixes the second direction and makes the first worse, because the names most similar to each other are often the ones most important to keep apart.
So resolution runs as a pipeline rather than a comparison. Coreference first, settling what things like "the merchant" refer to within their own context. Then linking each mention against what is already in the graph, by identifier where one exists, by similarity where none does, and filtered by type so a company is never matched against a person. Then merging, which looks for records that should never have been separate. Each stage carries a confidence rather than a verdict, and the low-confidence ones are the queue a person should be reviewing.
Temporal Validity
The thing most often left out is time, and leaving it out is what makes a graph age badly.
A fact stored without dates is a claim about now, forever. Jane Smith is CFO. Acme is a supplier. The merchant's mandate is active. Each was true when written, each will be retrieved with full confidence long after it stopped being true, and nothing in the retrieval marks the difference.
Putting a validity window on every entity and every relation fixes this cheaply. A fact carries when it started being true and, if it has ended, when it stopped. An open window means current. Three useful things follow, and the third is the one that changes how a team works.
| What it gives you | Why it matters |
|---|---|
| Correct answers about the past | Who approved this in March is answerable, rather than being answered with who holds the role today. |
| Contradictions that resolve | Two facts that disagree are often both right about different periods, so time turns a conflict into a sequence. |
| Corrections without deletion | A fact that stopped being true gets its window closed rather than being removed, so what was believed and when is still recoverable. |
A graph records 'Jane Smith is CFO of Acme'. She leaves in June. What is the least damaging way to handle it?
Discussion
Pick an entity your team's systems all refer to: a customer, a product, an account. Count how many systems hold a name for it and how many hold an identifier. The gap between those two numbers is the size of your resolution problem, and it exists whether or not anyone is working on it.