Most of us here have shipped at least one retrieval layer that worked beautifully in the demo and then quietly failed in production. This is a writeup of the specific failure mode I keep running into, and the extraction step that actually fixes it.
The failure
Store this sentence in your vector index:
we migrated the checkout flow from Stripe to Braintree last quarter
Three months later somebody asks:
what payment processor do we use
The two embeddings are not close enough, because neither the word payment nor the word processor appears anywhere in the stored text. The fact is in your database. The vocabulary is not. No amount of chunk size tuning fixes this, because chunking was never the problem.
Entity extraction turns that sentence into structured facts before you store anything:
- checkout flow uses Braintree (current)
- checkout flow previously used Stripe (historical)
- a migration event occurred, with a date
Now the question resolves by looking up an entity and its type, not by hoping the wording overlaps.
NER and LLM extraction solve different halves
Traditional NER is fast and cheap. SpaCy's en_core_web_trf hits 89.8% F1 on OntoNotes and processes thousands of documents per second on a single CPU. Fine-tuned BERT models reach 92 to 93% on the same benchmark. What both give you is the standard label set: PERSON, ORG, GPE, DATE, MONEY.
The moment your domain invents its own entity types, and it will (microservice, API endpoint, deployment environment, database table), a pretrained NER model simply does not see them. You either fine-tune, which costs 200 to 500 labeled examples per entity type, or you describe the types in a prompt and let an LLM extract them.
The cost difference is the whole argument. Running 100,000 documents through an LLM extraction prompt costs roughly $300 to $1,500 depending on the model. The same corpus through SpaCy costs under a dollar.
So most production pipelines now run tiered extraction. The small model goes over everything, and the LLM only touches passages where domain-specific entities or ambiguous relations are likely, using the NER output as context so nothing gets extracted twice. That cuts LLM spend 60 to 80 percent while keeping accuracy where it matters.
Entities on their own are a list. Relations are what make it traversable, and relation extraction is the harder half, because it needs the semantic connection rather than just the mention.
John mentioned Redis during the architecture review contains two entities and a very weak relation. John mentioned it. He does not maintain it, own it, or depend on it. A pipeline that treats co-occurrence as a relation fills your graph with edges that mean nothing, and every traversal after that inherits the noise.
The one that bites everybody eventually is predicate normalization. If uses, utilizes, relies on, is built on and depends on all land as separate predicates, traversal silently misses most of the graph. It presents as a retrieval bug and it is really a schema bug.
The full writeup, with the pipeline stages, the tiered cost model and the section on measuring extraction quality, is here: Entity Extraction and NER for AI
If you have built one of these, I am curious which half broke first for you, the entities or the relations.