Great update with practical improvements to Pocket Portfolio, especially the mock-trade lab and quick CSV import. How do you plan to expand the price fallback system to handle larger market data loads in real time?
Today’s ship: Mock-Trade Lab, 5-min CSV Import, and bulletproof Price Fallbacks
Pocket PortfolioLeader
posted
Originally published at dev.to
2 min read
0 Comments
Pocket Portfolio
•
Thanks, Ben — great point. “Never 0.00” is easy for a handful of symbols; the real work is scale + burstiness. Here’s how we’re extending the fallback system to handle larger, real-time loads without turning into a mini-HFT shop.
Architecture at a glance (client → edge → providers)
Symbol fan-out & coalescing (edge)
- Each region runs a lightweight coalescer per symbol bucket.
- Ingest from 2–3 providers (WS + HTTP backup), buffer for 100–250 ms, emit freshest per frame (5–10 Hz max).
- If no tick in
rotateMs→ rotate provider; if none instaleMs→ serve LKG withstale:true.
Snapshot + delta
- On connect, clients receive a compressed snapshot (last-known price + ts).
- Then we stream tiny deltas
{sym, price, ts, src, stale}. Cheaper, faster, cache-friendlier.
Tiered cache
- Hot KV (per region) for LKG + last 60–120 s.
- Cold object store for minute snapshots (optional analytics).
- Edge functions read KV first; providers only when needed.
Backpressure & quotas
- Per-client rate cap (e.g., max 5 Hz) + symbol limit per session.
- Adaptive frames: if fan-out spikes, coalescer steps from 100 → 200 ms automatically (humans can’t read 20 Hz anyway).
Health + circuit breakers
- We count timeouts/NaNs per provider, per region.
- When error rate > threshold, open circuit, exclude for
cooldownMs, log one concise line (no PII).
Clear UX
- If
stale:true, show a chip: “Last 2.4s • p2 fallback”. - If >10s, gray cell + tooltip. We always prefer known-stale to silently wrong.
- If
Tiny edge loop (sketch)
const frameMs = env.FRAME_MS ?? 200; // coalesce window
const rotateMs = env.ROTATE_MS ?? 1000; // rotate provider if quiet
const staleMs = env.STALE_MS ?? 2000; // mark stale
const buf = new Map<string, Tick>(); // latest within frame
const lkg = new Map<string, Tick>(); // last-known-good
setInterval(() => {
const now = Date.now();
for (const [sym, latest] of buf) {
const fresh = (now - latest.ts) <= staleMs;
const out = fresh ? latest : (lkg.get(sym) ?? latest);
const stale = !fresh;
pushToClients(sym, { ...out, stale }); // SSE/WS
lkg.set(sym, out);
if (!fresh && (now - (lkg.get(sym)?.ts ?? 0)) > rotateMs) rotateProvider(sym);
buf.delete(sym);
}
}, frameMs);
Why this scales
- Real-time in, human-rate out (frame coalescing) keeps fan-out manageable.
- Snapshot + delta slashes payload size.
- Regional KV removes cross-region chatter.
- Deterministic policy (rotate, stale, LKG) makes behavior predictable.
Open questions / collaboration invites
- Fairness: best strategy to allocate symbol budgets across many tenants?
- Crypto specifics: per-exchange consolidation vs. VWAP across venues?
- Compression: worth moving to SSE + zstd for high-symbol dashboards?
- Observability: minimal counters that actually help tune policy (no log firehose).
If you or anyone wants to riff, I’ll spin up a tiny OSS coalescer + SSE demo we can benchmark together (bring your provider keys). Appreciate the nudge—this is exactly the design pressure we want at this stage.
Please log in to add a comment.
Please log in to comment on this post.
More Posts
chevron_left