Budget-Exact Context Packing: Render First, Then Charge

●1 ●11 ●39
calendar_today ago • schedule1 min read

How TokenCap v3.1 fills a token budget:

  1. Render each file, then charge the tokens it actually produced. No up-front estimates.
  2. Large files ship the lines around the task's rarest terms, weighted by how rare each term is inside that file.
  3. A matching function of up to 60 lines is shown whole, using Tree-sitter boundaries.
  4. Files under 15% of the best score are listed as references, not code.
  5. In the companion, code has its own budget, separate from the primer and focus brief.
tokencap ask "cookie fallback" --budget 8000

Benchmark caveat: spans cover 73% of edited lines on average, lower on very large files. Numbers and limits are in HONEST-NUMBERS.md.

See tokencap.vansharora.app

Want more deep dives?

Join 5,951+ developers sharing real benchmarks, code breakdowns, and projects. Free forever.

More Posts

Local-First: The Browser as the Vault

Pocket Portfolio - Apr 20

Everyone says DeepSeek is cheaper, but I got tired of guessing the exact math. So I built a calculat

abarth23 - Apr 27

Architecting a Local-First Hybrid RAG for Finance

Pocket Portfolio - Feb 25

Should Exact Search Matches Always Come First?

WAQAS AHMAD - Oct 30

CData's New AI Gateway Puts Context at the Center — Fixes the Three Reasons Enterprise AI Stalls

Tom Smithverified - Sep 28
chevron_left
1.7k Points • 51 Badges
Jaipur, India • t.co/BRmkFprjYq
17Posts
8Comments
11Connections
tech freak

Related Jobs

Commenters (This Week)

1 comment
1 comment

Contribute meaningful comments to climb the leaderboard and earn badges!