Extracting Provider Logging into a Shared Module

Question

Devlog: Extracting Provider Logging into a Shared Module (so both quote and health APIs stay in sync)

Stack: React + Vite (SPA), Firebase Auth/Firestore, Vercel Edge (quote + health functions).
Story: “As a developer I want the provider logging logic in a shared module so that both the quote API and health API can reuse it.”

The problem we set out to solve

Our quote pipeline queries Yahoo, Chart API, and Stooq in parallel under a strict time budget. It already records successes/failures and whether a fallback was used. But that logging lived inside the quote function, and the health API implemented its own version. Result: duplicated code, inconsistent field names, and the occasional “red vs amber” disagreement when someone changed thresholds in one place.

We needed a single, typed logging module that could be called by:

• the quote function each time a provider succeeds/fails, and
• the health endpoint when aggregating/reading metrics.

We also wanted this to work locally without Upstash while seamlessly persisting on Vercel with Redis/Upstash when env vars are present.

Assumptions (stated so we can ship):

• Server time is UTC via Date.now() from Edge; the client converts to locale.
• Health metrics are operational (not financial) so currency/CSV quirks don’t affect logging; however, we record the portfolio base currency with each event for future symbol-level rollups.
• When network or Redis is down, the module must degrade to in-memory and never block quote responses.

How we scoped it

We cut the work into three PR-sized pieces:

1. Create @pp/shared-health (single folder in src/shared/health/) exporting the contract + read/write functions with a storage adaptor (Upstash|Memory).
2. Refactor quote to call recordSuccess/recordFailure and remove all inline logging.
3. Refactor health endpoint to call getAll, returning the same types the logger writes.

We deliberately excluded symbol-level freshness and alerts (separate stories) to keep the surface focused.

Design decisions

• Small, stable schema. Four fields per provider give us enough signal: lastSuccess, lastFailure, failureCount, activeFallback. We added optional baseCcy for future metrics.
• Adaptor pattern. A Storage interface supports UpstashKv and InMemory. The rest of the code never imports @vercel/kv directly.
• Idempotent writes. record* merges with the current row, so a retries or duplicate calls are safe.
• No await in hot paths. Quote writes are fire-and-forget (void record…()); health reads await.

---

Code sample 1 — Shared module (type & adaptor + record/get)

// File: src/shared/health/index.ts
export type Provider = "yahoo" | "chart" | "stooq";
export type ProviderHealth = {
  provider: Provider;
  lastSuccess?: number;     // epoch ms UTC
  lastFailure?: number;
  failureCount: number;     // rolling window
  activeFallback: boolean;  // last quote resolved via fallback
  baseCcy?: "USD" | "GBP" | "EUR"; // optional operational tag
};

export interface Storage {
  hget<T>(key: string, field: string): Promise<T | null>;
  hset<T>(key: string, map: Record<string, T>): Promise<void>;
  hgetall<T>(key: string): Promise<Record<string, T> | null>;
}

// Upstash adaptor (enabled by env), else memory
import { kv } from "@vercel/kv";
const memory = new Map<string, any>();
const KEY = "pp:health:v1";

export const store: Storage = process.env.UPSTASH_REDIS_REST_URL
  ? {
      async hget<T>(k, f) { return (await kv.hget(k, f)) as T | null; },
      async hset<T>(k, m) { await kv.hset(k, m); },
      async hgetall<T>(k) { return (await kv.hgetall(k)) as Record<string, T> | null; }
    }
  : {
      async hget<T>(k, f) { return memory.get(`${k}:${f}`) ?? null; },
      async hset<T>(k, m) { for (const [f, v] of Object.entries(m)) memory.set(`${k}:${f}`, v); },
      async hgetall<T>(k) {
        const out: Record<string, T> = {};
        for (const [kf, v] of memory.entries()) if (kf.startsWith(`${k}:`)) out[kf.split(":")[1]] = v;
        return Object.keys(out).length ? out : null;
      }
    };

function merge(oldRow: ProviderHealth | null, patch: Partial<ProviderHealth>): ProviderHealth {
  return { failureCount: 0, activeFallback: false, provider: patch.provider ?? oldRow?.provider as Provider, ...oldRow, ...patch };
}

export async function recordSuccess(provider: Provider, opts?: { baseCcy?: ProviderHealth["baseCcy"], asFallback?: boolean }) {
  const now = Date.now();
  const current = await store.hget<ProviderHealth>(KEY, provider);
  const next = merge(current, { provider, lastSuccess: now, activeFallback: Boolean(opts?.asFallback), baseCcy: opts?.baseCcy });
  await store.hset(KEY, { [provider]: next });
}

export async function recordFailure(provider: Provider) {
  const now = Date.now();
  const current = await store.hget<ProviderHealth>(KEY, provider);
  const next = merge(current, { provider, lastFailure: now, failureCount: (current?.failureCount ?? 0) + 1 });
  await store.hset(KEY, { [provider]: next });
}

export async function getAll(): Promise<ProviderHealth[]> {
  const all = await store.hgetall<ProviderHealth>(KEY);
  return all ? Object.values(all) : [];
}

Diff-able suggestion: colocate thresholds in src/shared/health/rules.ts so the card and health tests import the same “RAG” rules.

---

Code sample 2 — Using the module from both APIs

// File: api/quote/route.ts (Edge) — snippet inside the per-provider attempt
import { recordSuccess, recordFailure } from "../../src/shared/health";

// when yahoo succeeds:
void recordSuccess("yahoo", { baseCcy: "GBP", asFallback: false });

// when yahoo times out or non-200:
void recordFailure("yahoo");

// when we fall back to stooq to satisfy the time budget:
void recordSuccess("stooq", { asFallback: true });

// File: api/health-price/route.ts (Edge)
import { getAll } from "../../src/shared/health";
export const runtime = "edge";
export async function GET() {
  const providers = await getAll();
  return new Response(JSON.stringify({ providers }), {
    headers: { "Content-Type": "application/json", "Cache-Control": "max-age=5, s-maxage=5" },
  });
}

Why fire-and-forget on quote? We can’t risk blocking the hot quote path on Redis hiccups. If Upstash is down or the network blips, the quote still returns; logging catches up on the next call.

---

Diagram (textual)

Providers (Yahoo/Chart/Stooq) → Quote Engine (budgeted parallel fetch) → Shared Logger (recordSuccess/recordFailure to Upstash|Memory).
Health API reads from Shared Logger via getAll() → Health Card (RAG badges). Single schema shared by both sides.

---

CSV quirks, timezones, currency — why they still matter here

The logger itself doesn’t parse CSV, but CSV imports feed the dashboard that sits next to the Health Card. Two practical choices:

• We never infer provider health from CSV payloads; the source of truth is runtime events.
• We tag each record with optional baseCcy. That keeps future symbol-level rollups sane in multi-currency portfolios and prevents foot-guns like mixing GBP/ USD ages.

All timestamps are epoch ms UTC. UI renders “age” using the user’s locale/timezone. We added a test to ensure negative ages (client behind/ ahead) clamp to “Unknown”.

---

What we learned

• One module > two implementations. Extracting the logger removed an entire class of bugs where fields drifted between endpoints.
• Adaptors buy portability. With a tiny Storage interface we can swap to Firestore/SQLite for self-hosters without touching the quote path.
• Fail open beats fail closed. Quotes must return even if logging can’t; our fire-and-forget write plus retries made incident timelines simpler.

---

Acceptance Criteria (testable)

• A shared module src/shared/health exports recordSuccess, recordFailure, and getAll.
• Both quote and health routes import the module; no duplicate logging code remains.
• With Upstash present, logs persist across edge instances; with no env vars, logs work in-memory.
• Quote path does not await logging; health endpoint does await reads.
• Timestamps are epoch ms UTC; no thrown errors propagate to the quote response when storage is unavailable.
• Unit tests cover: merge semantics, failureCount increment, and storage fallbacks.

Definition of Done (broader than AC)

• All Acceptance Criteria pass.
• Threshold rules live in one file and are imported by the Health Card tests.
• Lint/type-check clean in CI; edge bundle size unchanged ±5KB.
• Docs updated (/docs/health.md); example env vars added.
• Feature flag remains unchanged; behaviour is identical in prod.

Test notes

• Simulate Redis down (unset env) and confirm quotes still return while health reads from memory.
• Inject a clock skew (±2 minutes) and verify ages never go negative in the UI (“Unknown” instead).
• Hammer /api/health-price with 2× polling frequency and confirm no rate-limit errors bubble to the UI.

Documentation / Release notes

• New: shared provider health module at src/shared/health.
• APIs: recordSuccess(provider, { baseCcy, asFallback }), recordFailure(provider), getAll().
• Env: UPSTASH_REDIS_REST_URL, UPSTASH_REDIS_REST_TOKEN (optional; falls back to in-memory).
• Impact: no user-visible change; enables the Health Card and simplifies future features (symbol-level freshness, outage alerts).