Knowledge Graph Evaluation: Scoring 12 Real Decisions

Knowledge Graph Evaluation: Scoring 12 Real Decisions

●1 ●12 ●44
calendar_today • schedule1 min read
— Originally published at renezander.com

Three weeks ago I started scoring my knowledge graph against the answer I would have reached without it. It earns its place: 6 of 12 decisions came out sharper, 2 errors never reached real work, and 73 minutes of research time saved.

The Metric That Cannot Fail

Retrieval told me none of that. 21 of 21 queries, 100% recall, 8s median. Recall and latency describe the index, not the work. No input makes them come back bad.

Two dials: retrieval pinned at the top of a narrow 90 to 100 percent scale, decision quality reading 6 of 12 on a full one

Freeze Your Answer First

None of that score exists unless the old answer is written down first: the options, the criteria, the recommendation, a confidence number. Then query. Write the baseline afterwards and you rebuild a past self who conveniently agreed with whatever came back.

A locked answer with a measurable gap to the post-read position, against a recalled answer that slides to meet it

If your retrieval has never contradicted you, nobody has checked.

What would it take for your own retrieval layer to return an answer that changes your mind, and would you be able to prove it did?


I write field notes from real builds: AI integration, automation, and the parts that break in production. New posts every two weeks. Use the RAG requirements template to audit the source, access and action boundaries in your own context layer.

1 Comment

0 votes
🔥 Join developers growing publicly
Share your knowledge, build in public, and grow your developer presence with a global community.

More Posts

Sovereign Intelligence: The Complete 25,000 Word Blueprint (Download)

Pocket Portfolio - Apr 1

AAIF Hits 10 Projects. Its CTO on What to Adopt Now and the Security Gaps Left Open

Tom Smithverified - Oct 23

Architecting a Local-First Hybrid RAG for Finance

Pocket Portfolio - Feb 25

The Privacy Gap: Why sending financial ledgers to OpenAI is broken

Pocket Portfolio - Feb 23

RAG for Small Businesses: AI That Searches Your Company Knowledge

floxolab - Nov 6
chevron_left
749 Points • 57 Badges
Berlin, Germany • renezander.com
21Posts
7Comments
14Connections
Enterprise AI Architect | Production-Ready AI Agents for CRM & ERP

Related Jobs

Commenters (This Week)

3 comments
2 comments
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!